Forward Deployed Engineer – LLMOps

About the role

The Forward Deployed Engineer – LLMOps owns the production operations for large‑language‑model (LLM) and agentic workloads. You will ensure reliable serving, cost efficiency, and observability for systems that are far less predictable than traditional ML pipelines.

Key responsibilities

• Design, operate, and scale inference infrastructure, load balancing, and caching for LLM/agentic workloads.
• Monitor and control inference costs, including token usage, retry loops, and model routing decisions.
• Build observability for LLM‑specific failure modes such as hallucination rates, latency spikes, and prompt drift.
• Manage model and version rollout strategies, including canary releases, fallback models, and A/B testing.
• Lead incident response for production issues affecting client‑facing LLM services.
• Partner with GenAI Engineers and Agentic AI Architects on production‑readiness reviews.
• Explain token‑cost dynamics to finance and business stakeholders.
• Support the definition and enforcement of cost‑governance policies for LLM workloads.

Required profile

• 4–6 years of platform/MLOps engineering experience with hands‑on LLM/GenAI production.
• Deep understanding of LLM inference economics (token costs, batching, caching, model routing).
• Experience building canary and rollback strategies for probabilistic systems.
• Comfortable operating under the higher unpredictability of agentic workloads.
• Strong communication skills to convey cost impacts to non‑technical stakeholders.
• Calm under pressure during live incidents.

Required skills

• Azure AI Foundry
• AWS Bedrock
• Google Vertex AI
• Self‑hosted LLM serving frameworks (vLLM, TGI)
• LLM inference cost analysis and token‑cost economics
• Batching, caching, and model routing techniques
• Observability tooling (tracing, evaluation pipelines, prompt/version management)
• Canary releases and rollback mechanisms
• Incident response for AI production systems