LLMOps
The operational discipline of running LLM-based applications in production — prompt versioning, model routing, evaluation pipelines, cost governance, observability, incident response. Distinct from traditional MLOps because model weights are largely opaque and prompts are the primary "code".
Related terms
- AI Observability
The practice of instrumenting LLM applications to capture prompts, completions, tool calls, latencies, token counts, costs, feedback, and quality metrics — then analyzing that stream for drift, regressions, and abuse. Distinct from traditional APM; tools include Langfuse, Arize, Datadog LLM Obs, Braintrust.
- MLOps
Operational practices for machine-learning systems — data pipelines, model training pipelines, feature stores, drift monitoring, retraining. Precursor discipline to LLMOps; still relevant for traditional ML systems that run alongside GenAI.
Related on this site
Where this fits
LLMOps is part of the Data & Infrastructure vocabulary used in the Generative AI Maturity Framework. See the full glossary for the complete set of 149 defined terms, or take the free maturity assessment to see where your organisation stands.