[ Finance ]
Why AI Pilots Fail in Finance: What the 95% Statistic Misses
The model is an easy target when a finance AI pilot stalls. The harder problem may be the missing system around it: governed data, encoded logic, context, controls, and ownership.
On this page
The number that escaped a 2025 Project NANDA preliminary report was 95%. The report did not find that 95% of finance AI pilots technically failed. It found that only 5% of task-specific enterprise GenAI tools reached its threshold for sustained productivity or P&L impact.
A finance AI pilot can fail when a compelling demo is mistaken for a production system. The model can draft a fluent answer on prepared data. Production has to reconcile source systems, apply the company’s financial definitions, survive a period change, show its work, and stop when judgment belongs to a human.
Appetite alone is not enough. Architecture and operating choices decide whether a demo survives.
What does the 95% statistic actually measure?
The 95% figure is a warning about the pilot-to-production gap, not a census of broken models. Project NANDA reviewed more than 300 public AI initiatives, interviewed representatives of 52 organizations, and surveyed 153 senior leaders. Its v0.1 report covered January through June 2025 and defined a successful task-specific implementation as one producing a marked and sustained productivity or P&L effect. The authors called the deployment figures directional; some came from interviews rather than official company reporting, and category samples and success definitions varied.
NANDA’s threshold is high, and uneven. A tool can work technically, save an analyst ten minutes, and still miss it; a production deployment may also leave enterprise profit unchanged. The defensible conclusion is narrower: many custom enterprise GenAI initiatives stall before they create sustained, measurable value.
A separate RAND study interviewed 65 experienced AI practitioners and found that leadership mistakes and inadequate data were the two dominant causes they volunteered. Among the 50 industry practitioners, 84% cited at least one leadership-driven cause, such as choosing the wrong problem or optimizing the wrong metric. RAND did not measure an 80% project failure rate, despite that statistic often being attached to the study.
Why does demo magic disappear in production?
A demo proves that a model can respond under controlled conditions. Production proves that a finance team can rely on the whole system under changing conditions. Those are different tests.
Imagine a pilot that explains a gross-margin decline from one clean spreadsheet. The answer sounds right. At month-end, the same question crosses an ERP, CRM, billing system, budget file, and a foreign-exchange table. One source uses invoice date, another uses accounting period. “Revenue” means booked revenue in one report and recognized revenue in another. The model did not suddenly become less eloquent. The job became financial.
Complexity compounds quickly. In the 2026 FinMathBench study, GPT-4o with chain-of-thought prompting scored 72.9% on one-formula questions and 14.0% on four-formula questions. The benchmark was synthetic and did not test tool-enabled production systems, so the exact scores should not be generalized. The direction is still useful: a clean calculation demo says little about a chained finance workflow with dependencies and edge cases.
Which layer is missing: data, logic, or context?
Pluvo’s diagnostic groups recurring pilot gaps into three layers: unified governed data, encoded financial logic, and persistent business context. Better prompting can disguise a gap for a demonstration. It cannot reconcile the next close, carry a changed forecast assumption, or answer a board follow-up with evidence.
1. Unified governed data
Unified data does not mean copying every table into one large prompt. It means resolving entities, periods, currencies, dimensions, permissions, and source freshness before analysis starts. The system needs to know which records are approved and which version of the budget is active.
2. Encoded financial logic
Financial logic is the company’s definition of the number: how ARR treats one-time revenue, which entities enter the consolidation, how intercompany activity is eliminated, and whether the forecast uses transaction date or accounting period. If those rules live only in an analyst’s workbook or memory, the pilot has no stable operating policy.
3. Persistent business context
Context records what relates to what and why: a customer to a contract, a contract to an invoice, an invoice to a GL entry, and a GL entry to cash. A durable business ontology makes those relationships explicit instead of asking the model to rebuild them for every question.
Pluvo’s approved internal evaluation offers a directional illustration. On the same financial-analysis test, the same model scored 16.7% without structured context and 54.2% with it. The task count, model version, dataset, scoring rubric, and test date are not available in an auditable source artifact, and the result has not been independently reviewed. These figures are not a market benchmark. They show only that structured context materially changed this model’s score on this test.
What should a finance AI pilot prove before production?
A finance AI pilot should prove a bounded operating claim: with named sources, definitions, controls, and owners, the system can complete one recurring workflow to an agreed acceptance standard. “The demo was impressive” is not an acceptance standard.
A 2024 Bank of England and FCA survey of 118 UK financial-services respondents found that 75% were using AI, yet only 2% of use cases involved fully autonomous decision-making. Four of the five highest perceived current risks were data-related. The survey shows broad reported use alongside a narrow appetite for full autonomy.
The NIST AI Risk Management Framework 1.0 organizes risk work around govern, map, measure, and manage. NIST notes that the functions are neither an ordered sequence nor a checklist, and a revision is in progress. Applied to a finance pilot, Pluvo translates them into named accountability, a documented use case, testing against known ground truth, and monitoring after deployment.
What stays human after the pilot works?
Humans retain the decisions that turn a calculation into accountability. Finance leaders decide which definition is authoritative, whether an exception is material, when an assumption has changed, and what can be sent to an auditor, executive, or board.
A production system should reduce manual inspection without creating review theater. Deterministic refreshes can move to exception-based review after they earn trust. Ambiguous mappings, policy choices, assumption changes, journal-impacting actions, and external reporting need an explicit human owner.
What does production-ready architecture look like?
Production-ready finance AI separates language from calculation. Approved systems supply the data. Financial definitions govern the computation. Context connects the entities. The model explains the result. Source lineage lets a reviewer reproduce it, while controls define what the system may do and when it must stop.
Production-ready finance AI also needs an operator. A finance engineer owns the reusable system behind the analysis, while finance leaders retain judgment and approval. The role prevents a pilot from becoming an orphaned notebook that only its builder understands.
How should a failed finance AI pilot be restarted?
Restart with the failure evidence, not a new model shortlist. The Finance AI Pilot Architecture Diagnostic below turns the first visible symptom into the next test. It is Pluvo’s editorial framework, not a validated scoring instrument.
| Observed symptom | Likely missing layer | Evidence to request | Next pilot test |
|---|---|---|---|
| Answer changes when the file or period changes | Unified governed data | Source list, period rules, freshness checks, reconciliation | Run the same question across two closed periods and reconcile totals to source |
| A plausible number uses the wrong business definition | Encoded financial logic | Metric owner, formula, inclusion rules, version history | Test one metric with two known edge cases and require the approved definition |
| The team must re-explain the business every session | Persistent context | Entity map, relationships, assumptions, prior decisions | Repeat the workflow without restating known relationships and compare outputs |
| Reviewers cannot reproduce the answer | Lineage and controls | Source records, transformations, prompt/model record, approvals | Trace one material figure from answer to source and back |
| The demo works, but nobody owns production | Operating ownership | Named finance owner, technical owner, escalation path, service level | Run a period change and a source failure with the operating team |
| Users like it, but leadership sees no value | Outcome design | Baseline time, error rate, cycle time, decision or cost measure | Measure one repeated workflow against the pre-pilot baseline |
For more field notes on rebuilding AI finance workflows around data, logic, context, and controls, subscribe to the Finance Engineering newsletter.
A failed pilot is useful if it reveals which layer was absent. Replacing the model before reading that evidence risks repeating the same failure.
Frequently asked questions
Do 95% of AI pilots fail?
A preliminary, directional 2025 Project NANDA report found that only 5% of task-specific enterprise GenAI tools met its threshold for marked and sustained productivity or P&L impact. That is not a finance-specific technical failure rate.
Why do AI pilots fail in finance?
Common failure modes include missing governed source data, encoded financial definitions, persistent business context, source lineage, controls, or a named operating owner.
Is the AI model usually the problem?
Sometimes. Model limits matter, especially as financial calculations become more complex. But leadership choices, data quality, workflow fit, definitions, infrastructure, and controls often determine whether a capable model creates durable value.
What should a finance AI pilot measure?
Measure one recurring workflow against a pre-pilot baseline. Track reconciliation accuracy, exception rates, cycle time, reviewer effort, traceability, and the business outcome the finance owner approved.
What finance AI decisions should remain human?
Humans should retain authority over metric definitions, materiality, ambiguous mappings, policy choices, assumption changes, journal-impacting actions, and information sent to auditors, executives, or boards.
