Forward Deployed Engineer – LLMOps

About the role

The Forward Deployed Engineer – LLMOps owns the end‑to‑end production operation of large language model (LLM) and agentic workloads. You will be responsible for serving, cost management and observability of systems that are far less predictable than traditional ML pipelines.

Key responsibilities

• Own production serving and scaling for LLM/agentic workloads (inference infra, load balancing, caching)
• Monitor and control inference cost – token usage, retry/loop cost, model routing decisions
• Build observability for LLM‑specific failure modes: hallucination rate, latency spikes, prompt drift
• Manage model/version rollout strategy (canary releases, fallback models, A/B testing)
• Lead incident response for LLM production issues
• Partner with GenAI Engineers and Agentic AI Architects on production‑readiness reviews
• Explain token‑cost dynamics to finance and business stakeholders
• Support cost‑governance policy for LLM workloads

Required profile

• 4–6 years of platform or MLOps engineering experience with hands‑on LLM/GenAI production
• Deep understanding of LLM inference economics, including token costs, batching, caching and model routing
• Experience building canary and rollback strategies for probabilistic systems
• Ability to communicate cost implications clearly to non‑technical stakeholders
• Calm under pressure during live incidents affecting client‑facing systems

Required skills

• MLOps tooling and pipelines
• LLM inference infrastructure (load balancing, caching)
• Token‑cost monitoring and optimization
• Observability tooling for LLMs (tracing, evaluation pipelines, prompt/version management)
• Cloud model hosting platforms: Microsoft Azure AI Foundry, AWS Bedrock, Google Vertex AI
• Self‑hosted open‑source inference servers: vLLM, Text Generation Inference (TGI)
• Canary release and A/B testing frameworks
• Incident response and root‑cause analysis