MLOps and LLMOps: A Systematic Survey of Deploying, Monitoring, and Operating Large Language Models in Production
Year: 2026
Type: preprint
Study design: Systematic Review
Abstract: Machine learning operations (MLOps) emerged to close the gap between a model that works in a notebook and a system that works in production, and the arrival of large language models (LLMs) has forced the field to evolve again. Language models introduce new deployment problems — enormous model sizes, autoregressive inference, prompt-dependent behavior, and outputs that cannot be checked against a fixed ground truth — that classical MLOps tooling was never designed to handle. This paper presents a systematic survey of MLOps and its language-model successor, LLMOps. It traces the principles of MLOps from continuous integration and delivery for machine learning through model registries, data validation, and monitoring; it presents a taxonomy of production machine-learning systems organized by deployment pattern, serving architecture, and operational risk; it examines how LLM inference is actually served, from memory-efficient attention and paged key-value caches to quantization and speculative decoding; it analyzes monitoring and evaluation for systems whose outputs are free text, including drift detection, LLM-as-a-judge evaluation, retrieval-augmented generation evaluation, and tracing; it surveys the operational layer of LLM applications — prompt versioning, retrieval pipelines, and autonomous agents; and it discusses cost, energy, safety, governance, and compliance as first-class operational concerns. The survey concludes that LLMOps is not a replacement for MLOps but an extension of it: the classical disciplines of versioning, testing, and monitoring remain necessary, and the new work is to extend them to artifacts — prompts, retrieved context, and agent trajectories — that have no counterpart in classical machine learning.
Author: Mahmoudpour, Aref
Institution: Islamic Azad University
Language: English
Subfield: Computer Networks and Communications
Field: Computer Science
Domain: Physical Sciences
Keywords: MLOps, LLMOps, large language models, production deployment, large language model inference, model monitoring, drift detection, retrieval-augmented generation, speculative decoding, model quantization, LLM-as-judge evaluation, autonomous agents, continuous integration and delivery, model registry, data validation
Open Access status: green
OpenAlex ID: W7234360066
