When Not to Use AI

A material share of the use cases put in front of AI consultancies are better solved with a rules engine, a fixed report, a form, or a conversation between three people.

Saying so costs revenue in the quarter.

It is also the most useful thing most firms in this market could tell their prospects, and almost nobody does, so here is the list.

This is not an argument that AI is overhyped. The capabilities are real and some of the applications are transformative.

It is an argument that the current default — reach for a language model first and check afterwards whether the problem needed one — produces a predictable category of expensive disappointment.

RAND’s study of AI project failure lists applying AI to problems beyond the current state of the art as one of five root causes, and separately lists chasing the technology rather than solving a problem as another.

Those are two different mistakes and both are avoidable at the point of choosing the use case.

1. The rules are known, written down, and stable

If the logic can be expressed as a decision table — if this, then that, unless the other — then a decision table is the better implementation.

It is deterministic, testable, auditable, cheap to run, and modifiable by the person who owns the policy rather than by an engineer adjusting a prompt.

The temptation is that the rules feel complicated.

Two hundred rules across nine branches looks like a machine learning problem.

It usually isn’t. It is a large rules problem, and large rules problems have well-understood tooling that has existed for decades.

A language model applied here introduces non-determinism into a process whose entire value was that it was deterministic, and then requires an evaluation harness to measure whether it is still doing what the rules already did correctly.

2. The answer is a lookup

“How many orders shipped late last month?” is a query.

“What is this customer’s credit limit?” is a field.

“Which invoices are overdue?” is a report.

Wrapping a natural-language interface around these is sometimes worthwhile as a convenience, particularly at scale across many such questions.

But the frequent version of this project is a chatbot built over a warehouse to answer twelve specific recurring questions.

That is twelve dashboard tiles with extra latency, extra cost, and a new failure mode where the answer can be confidently wrong.

If the question set is small and known, build the answers.

3. There is no tolerable error rate

Some processes have a genuine zero-tolerance requirement — regulatory filings, payment instructions, safety-critical calculations, anything where being wrong once is materially worse than being slow.

A language model is a probabilistic system.

Its error rate can be measured, constrained, and reduced. It cannot be made zero.

Putting a human reviewer in front of the output helps less than the design may imply, because a reviewer checking plausible-looking output has a non-trivial probability of missing errors, and that risk rises with volume and familiarity.

Where the requirement is genuinely zero tolerance, the honest answer is that this is the wrong architecture, not that it needs another guardrail.

4. The process itself is broken

This is the most common one and the hardest to say out loud in a meeting.

An organisation has a process that takes eleven days and involves four handoffs, three spreadsheets and a weekly reconciliation meeting.

The proposal is to automate parts of it with AI.

Almost always, the eleven days are mostly waiting — for approvals, for a batch job, for someone to come back from leave — and the AI would compress the two days of actual work while leaving the nine days of waiting untouched.

Automating a broken process makes it a faster broken process.

It also makes the process considerably harder to fix afterwards, because the automation now encodes the current shape and changing that shape means rebuilding the automation.

5. The data isn’t there

A model cannot infer a fact that does not exist in the data you have.

If the answer to the business question depends on information that was never captured, captured inconsistently, or captured in a free-text field that three teams filled in differently over six years, then no model will reliably produce it.

This is the cause that RAND and MIT both put near the top, and it is the one most often reframed as a model problem.

A system producing unreliable output because its inputs are unreliable looks identical, from the outside, to a system producing unreliable output because the model is weak.

The remedies are completely different and cost different amounts.

6. The volume doesn’t justify it

A workflow that runs forty times a month and takes fifteen minutes each time is ten hours of work.

Automating it well — including evaluation, monitoring, error handling and a maintenance owner — is considerably more than ten hours, recurring, forever.

This calculation is skipped constantly because the prototype takes an afternoon.

The prototype is not the cost.

The cost is the evaluation suite, the alerting, the runbook, and the person who fixes it when a source system changes.

7. Nobody has agreed what success means

If the people sponsoring the project and the people building it have different definitions of success, the project will produce something and then argue about whether it worked.

RAND found this to be a leading root cause of failure, with 84% of its industry interviewees pointing to leadership-driven issues.

The specific version that matters here: if “working” cannot be stated as a number before the build starts, there is no way to know afterwards whether a change made things better or worse.

That is not a reason to skip AI.

It is a reason to resolve the definition first, on a sample of real cases, with whoever owns the answer in the room.

What’s left

Everything above is a filter, not an argument against the technology.

What passes through it is a real and substantial category: problems where the input is unstructured, the rules resist enumeration, the volume is high, an error rate in the low single digits is acceptable with review, and the data to support the task actually exists.

Document extraction and classification at volume.

Retrieval across a large corpus where the questions are open-ended.

Drafting that a person then edits.

Reconciliation across systems that do not share identifiers.

Triage that routes rather than decides.

Summarisation where the reader can check the source.

These are good applications and they are worth doing properly.

We have talked prospects out of projects, out of larger scopes, and out of AI entirely, and we expect to keep doing it.

It costs revenue in the quarter.

It is also the only version of this business we would want to run, and the reasoning behind that is on our About page .

References

• The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed — RAND Corporation. The source for the five root causes referenced throughout, including problem definition and technology-chasing.
  RAND Corporation
• Full report PDF — methodology and the finding on leadership-driven causes.
  RAND Research Report
• The GenAI Divide: State of AI in Business 2025 — MIT NANDA initiative, on integration and the learning gap rather than model capability.
  State of AI in Business 2025 Report
• Why AI Projects Fail — RAND presentation format, useful to send to a non-technical stakeholder.
  RAND Presentation