MLOps Is the Other 95 Percent: An Interactive Map of Machine Learning System Debt

Teams often speak as if a machine learning product were mainly a model. In practice, the training script or notebook is only a small part of what must be built, operated, and maintained. The larger share is the machine learning platform: the systems that collect data, check quality, engineer features, serve predictions, monitor behaviour, manage process and resources, and keep configuration under control.

This idea is not new. In 2015, Sculley and colleagues at Google published Hidden Technical Debt in Machine Learning Systems at NeurIPS. Their central figure shows a tiny box labelled machine learning code, surrounded by much larger boxes of supporting infrastructure. Accounts of that work often summarise the split as roughly five percent model code and ninety-five percent platform. Whether or not those exact percentages fit every organisation, the visual argument remains accurate: if you change “only the model”, you are still touching most of the system.

This post restates that argument in plain language, then offers an interactive map of the same layout. Select any box to see what that part of the platform is for, what forms of debt tend to accumulate there, and which tools teams commonly use today. The tool lists are a current illustration. They are not claims from the original paper.

The tiny box in the middle

Machine learning code is the part that trains, fine-tunes, and evaluates a model. It is also the part that attracts attention in demos, papers, and hiring conversations. That attention is understandable. The model is what makes the product distinctive.

It is not, however, what usually decides whether the product survives contact with production. Production requires a reliable path from raw data to a prediction that someone or something can consume, with enough monitoring to notice when that path goes wrong. When those pieces are weak, teams discover that replacing a model still forces a rewrite of pipelines, features, serving paths, and configuration. The “small change” was never small.

[[MlPlatformTiles]]

What a machine learning platform must do

If you look at the map above, the capabilities fall into a few clear groups.

Data must enter the system and remain trustworthy. Collection brings events, documents, labels, and feedback into storage. Quality checks catch schema breaks, stale batches, and distribution shifts before training or serving trusts the data. Analysis helps people decide which problems are worth modelling and which slices of the population fail more often than the average metric suggests.

Features and models must stay aligned. Feature engineering turns raw inputs into signals that training and serving both understand. For classical systems that often means offline and online feature pipelines. For language-model systems it often means retrieval, prompt assembly, and tool results. The model code sits in the centre of that work, but it depends on everything around it.

Predictions must be delivered and watched. Serving infrastructure handles online and batch inference, authentication, rate limits, canaries, and routing across models. Monitoring covers latency, errors, drift, cost, and feedback loops. Without deliberate monitoring design, the system can train on its own past outputs and slowly reinforce the wrong behaviour.

People and machines must be able to run many projects without chaos. Process management orchestrates extract–transform–load jobs, training, evaluation, and promotion. Resource management schedules compute, enforces quotas, and controls cost. Configuration management versions hyperparameters, routes, prompts, model identifiers, and rollouts so that yesterday’s result can be reproduced tomorrow.

A useful platform also supports more than one project at a time, with isolation between users and teams. Without that isolation, a change that should affect one experiment ends up rewriting ninety-five percent of a shared stack.

Debt that hides outside the model

Sculley and colleagues argued that machine learning systems create forms of technical debt that ordinary software also has, plus forms that are specific to learning systems. A few patterns are especially useful to keep in view when you look at the map.

Changing anything can change everything. A small shift in a feature, a label definition, or an upstream data join can move metrics in unexpected ways. Aggregate dashboards often hide slice-level failure.

Consumers of predictions are easy to leave undeclared. Once a score is published, other systems may depend on it without a clear contract. Serving then becomes difficult to change safely.

Pipelines grow into jungles. Features are appended because they help a little. Paths multiply. Training and serving slowly diverge until nobody can explain why yesterday’s notebook no longer matches today’s online path.

Configuration can outgrow code. Feature flags, model identifiers, prompt versions, and rollout rules accumulate until the system’s behaviour lives more in configuration than in the training script.

Glue code freezes choices. Thin wrappers around libraries and services feel temporary, then become the only way the organisation knows how to ship. The model stays easy to discuss; the glue becomes hard to replace.

None of these smells live primarily in the tiny machine learning code box. They live in the surrounding platform.

What to invest in first

You do not need every box to be world-class on day one. You do need honesty about which boxes are load-bearing for your product.

A practical order for many teams is:

1. Data collection and quality — without a trustworthy input path, model work is theatre.
2. Serving and monitoring — without a safe way to deliver and observe predictions, research never becomes a product.
3. Configuration and process — without versioned configuration and repeatable pipelines, every improvement is a one-off.
4. Feature engineering and resource management — deepen these as scale, latency, and cost begin to hurt.
5. Model code — keep improving it, but treat it as one component inside a larger factory rather than as the factory itself.

The point is not to build a platform for its own sake. The point is to stop pretending that the model is the system.

Closing

Machine learning code is what makes a learning system stand out. According to the experiences described by Sculley and colleagues, it still amounts to only a small fraction of the total system. The rest is the machine learning platform: data, features, serving, monitoring, process, resources, and configuration.

If your organisation is debating which model to buy or train next, it may be asking the wrong first question. Ask instead which parts of the surrounding map are missing, fragile, or owned by nobody. That is usually where delivery stalls — and where debt compounds quietly while the tiny white box in the middle keeps getting all the attention.

References

Sculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., Chaudhary, V., Young, M., Crespo, J.-F., & Dennison, D. (2015). Hidden Technical Debt in Machine Learning Systems. In Advances in Neural Information Processing Systems (NeurIPS 2015). https://proceedings.neurips.cc/paper/2015/file/86df7dcfd896fcaf2674f757a2463eba-Paper.pdf