Why Your AI Pilot Won’t Reach Production

Most organisations now have an AI pilot. A good number have several. Far fewer have something running in production that a business process depends on, and the gap between those two states is where most of the money currently being spent on AI is going.

It is tempting to read that as a technology problem — the models aren’t good enough yet, the tooling is immature, next year will be different. The published research does not support that reading, and it is worth looking at what it actually says, because the diagnosis determines what you should do next.

What the research found

The RAND Corporation interviewed 65 data scientists and engineers, each with at least five years of experience building machine learning systems in industry or academia, and published the results as The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed .

Their headline figure is that more than 80% of AI projects fail, roughly twice the failure rate of IT projects that don’t involve AI.

Separately, MIT’s NANDA initiative published The GenAI Divide: State of AI in Business 2025 , based on interviews with business leaders, an employee survey, and an analysis of 300 public AI deployments.

Its finding was starker and narrower: around 95% of generative AI pilots produced no measurable impact on profit and loss, against an estimated $30–40 billion of enterprise spend.

Two studies, different methods and different populations, converge on the same conclusion: failing or stalling is the default outcome, and the causes are mostly not technical.

The five root causes, and the thread through them

RAND’s five causes can be summarised as follows.

1. The problem was never agreed

Leadership and the technical team have different understandings of what is being solved, so nobody can say what success would look like.

RAND found that leadership-driven issues were frequently identified as the primary cause of failure — an uncomfortable finding, but also one of the most actionable.

2. The data wasn’t adequate

The quality, availability and governance work was underestimated. Not absent — underestimated, which is worse, because the project is often already committed by the time the scale of the issue becomes visible.

3. The technology came first

The project started from a desire to use a particular capability rather than from a problem worth solving.

These pilots often work in the sense that they do what they were built to do, and then nobody adopts them.

4. Deployment infrastructure was underfunded

Getting from a working prototype to something reliable in production is harder and less interesting than building the prototype, and it is consistently the part that isn’t adequately budgeted.

5. The problem was beyond the current state of the art

This is the genuinely technical cause on the list — and, importantly, the least common.

A use case is often chosen without knowing whether the data required to support it exists, is current, is governed, or means what people believe it means.

The prototype is then built using a hand-assembled extract, which works because a hand-assembled extract almost always works.

Production requires that same information on a schedule, with access control, agreed definitions, accountability and monitoring. That is a different project, usually a larger one, and frequently it was not included in the original plan.

What this predicts

The pilot’s success tells you almost nothing about production viability

A prototype demonstrates that a model can perform the task. It does not demonstrate that your data pipeline can feed it, that your permission model can constrain it, or that anyone will notice when output quality degrades.

These are separate questions with separate failure modes, and the pilot was not designed to answer them.

The expensive discoveries arrive late by default

Stale data, unexplained permissions, inconsistent metric definitions and integration constraints frequently remain invisible during prototyping.

They surface during integration, when budgets are already committed and reversing course becomes much more expensive.

More pilots will not fix it

If the constraint is the state of the data layer, running another pilot produces another demonstration — not another production system.

The instinct when pilots stall is often to try another use case. The research suggests that the use case itself was rarely the only problem.

What to do instead

Find out what state you are actually in before committing to a build. Concretely, that means answering four questions with evidence rather than assumptions.

What data do you actually have?

Where does it live, how fresh is it, who owns it, and what is quietly broken?

Which candidate use cases are actually feasible?

Evaluate use cases against the data you have today, rather than the data you would like to have.

What is the honest effort estimate?

Include the pipeline, integration, governance, permissions, monitoring and operational work — not just the model work.

Which ones should you not build?

This is the question many assessments avoid, and often the one that saves the most money.

A prioritised roadmap with nothing on the don’t build this list has usually not been written honestly.

Readiness before build

This is a two-week question, not a six-month one. It requires access to representative data, a small number of stakeholder conversations, and someone willing to document an unwelcome answer if that is what the evidence supports.

It is also one of the highest-leverage things an organisation can do before spending money on a build, because four of the five documented failure causes are decided at this stage, and only one of them is primarily about the technology.

Our AI Readiness Assessment is a fixed two-week engagement designed to answer those questions and give you a roadmap to keep, whether or not you continue working with us afterwards.

References

• RAND Corporation — The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed
• RAND — full research report and methodology.
• MIT NANDA — The GenAI Divide: State of AI in Business 2025