Vbeyond
VBeyond Corporation || PARTNERING FOR GROWTH
Experience: 7+ years in platform / DevOps / MLOps engineering, including production LLM or ML workloads.
You turn a working pipeline into a production system. The existing toolchain needs to be fully wired into CI/CD. You will enhance it with a proper evaluation and guardian pattern, and make the whole thing observable, auditable and affordable.
Responsibilities
Productionise the existing RAG and scanner toolchain through CI/CD — connecting the pipeline end to end so scans, dispositions and remediations flow without manual intervention.
Build the guardian / evaluation agent: an automated check that runs on every sub-agent deliverable, replacing the current brute-force knowledge-capture approach with a best-practice evaluation pattern.
Implement the deterministic assertion layer as a programmatic gate — automatically rejecting any disposition that contradicts its own evidence, before a human ever sees it.
Own AgentOps: trace capture, prompt / rule / model versioning, evaluation-in-CI, regression harnesses, and drift detection.
Build the observability the team watches daily: pending burn-down, auto-disposition rate, accuracy against the gold set, human-minutes per item, assertion-rejection rate and cost per item.
Own FinOps for the AI workload: model routing, delta-scoped runs (re-processing only items whose evidence changed), caching, and a per-cycle token budget tracked as a service-level objective.
Guarantee provenance and auditability for a regulated environment — every decision reproducible from its evidence snapshot, rule/prompt/model version and human verdict.
Qualifications
Python — production-grade.
CI/CD automation for application and ML/LLM workloads; release automation and test gating.
AgentOps / LLMOps — tracing, prompt versioning, evaluation in CI, regression harnesses, drift detection.
Observability — OpenTelemetry, distributed tracing, metrics and logging; building dashboards operators actually use.
AWS; containerisation; infrastructure-as-code (Terraform).
FinOps for AI workloads — token accounting, model-routing economics, cost dashboards.
Guardrails and policy-as-code; secure handling of regulated data.
Working knowledge
Kubernetes; LangGraph; AWS Bedrock Guardrails.
SQL; Informatica; evaluation-harness construction.
Drop a email : AbihaaranL@VBeyond.com
To apply for this job email your details to AbihaaranL@VBeyond.com