The failure mode for enterprise AI is not a model that does not work. It is a model that works, demos well, and never reaches a workflow — because no one owns the decision it was built to support, and no one budgeted for keeping it accurate.
Four questions before any pilot
- Whose decision does this change? If the answer is a committee, the use case has no owner and will not be adopted.
- What happens when it is wrong? A use case with no acceptable error mode needs a different solution, not a better model.
- Does the data already exist as a byproduct of operations? Data that has to be created for the model will stop being created the moment attention moves.
- What is the current cost of the decision being made badly? If nobody can estimate it, the return cannot be evidenced afterwards either.
Data readiness is a workflow question
Teams assess data readiness by volume and quality. The more predictive measure is whether the data is produced by a process someone depends on. Operationally load-bearing data stays clean because breaking it breaks someone's day. Data collected for reporting drifts quietly and takes the model with it.
Decide the evaluation before the build
An evaluation set drawn from real cases, labelled by the people who currently make the decision, is the single highest-return artifact in an AI project. It converts 'the model seems better' into a number, makes regressions visible, and — critically — survives a change of model or vendor. Build it first; it outlives everything else in the project.
A durable evaluation set is worth more than the first three models you run against it.
The costs that arrive after launch
- Monitoring for drift, with a defined owner and a defined response, not a dashboard nobody opens.
- Periodic re-labelling, because ground truth moves as the business does.
- Human review capacity for low-confidence cases — a real staffing line, not an assumption.
- Inference spend, which scales with adoption and therefore rises exactly when the use case succeeds.
Durable use case shapes
The use cases that last tend to share a profile: a high-frequency decision, a tolerable error cost, an owner who feels the pain today, and data produced as exhaust from a process that will keep running regardless. Document classification in operations, triage and routing in support, anomaly detection in finance and infrastructure, and retrieval over internal knowledge all fit that profile. Use cases that promise to replace judgment in low-frequency, high-stakes decisions almost never do.
Start where the data is already load-bearing
It is the least exciting selection criterion and the most reliable one. A modest use case on operational data outperforms an ambitious one on data that exists because someone was asked to produce it for a pilot.
