· By Sophie Laurent
- ai
- genai
- governance
Over the past eighteen months the question our clients ask has changed. It is no longer "does this work" but "why is our prototype not reaching production". The answer is rarely the one they expect.
The model is not the bottleneck
The models available today comfortably exceed what most internal use cases demand. A document search assistant, a drafting aid or a classifier for incoming requests do not require the state of the art. They require clean input data and a clear definition of what a good answer is.
Three recurring obstacles
Input data. A real document corpus contains obsolete versions, partial duplicates and contradictory documents. A retrieval-augmented system plugged into it faithfully reproduces that incoherence. Cleaning and document governance come before the AI project; they are not replaced by it.
Evaluation. Many teams validate their prototype by testing it by hand on fifteen chosen questions. That is not an evaluation, it is a demonstration. Without a test set built with the business, against explicit criteria, there is no way to say whether a change improved or degraded the system.
Accountability. When the system produces a wrong answer that someone acted on, who answers for it? As long as that question has no named owner, legal departments block — and they are right to.
What does work
The deployments we see hold share three traits: a restricted, well-understood document scope, an evaluation set maintained like code, and explicit human validation on consequential uses. It is not spectacular. It is what reaches production.