[ Finance ]
The 7 AI Failure Modes Finance Teams Should Test First
Diagnose seven AI finance failure modes by symptom, five-minute test, stop condition, architectural root cause, and accountable human owner.
On this page
Click the 18.4% gross-margin figure in an AI-generated finance report. Ask for the source rows. If the system returns a citation and a polished explanation but no path to the ledger, the evaluation is over.
At Pluvo, we test seven AI finance risks first: missing run history, unverifiable outputs, confident invented numbers, lost session continuity, business context rebuilt inside prompts, permissions broader than the task, and shallow variance analysis. These are first tests, not a claim about incident frequency.
The test is evidence, not eloquence. Run the seven checks below on a live workflow with your data and permissions. On June 10, 2026, the Financial Stability Board issued a consultation for financial institutions whose proposed practices include documenting agent steps, limiting permissions, and preserving human accountability. It is nonbinding guidance, not a rule. A vendor that cannot show the evidence before signature is unlikely to produce it afterward.
What are the seven AI finance risks teams should test first?
Each failure begins with a visible symptom. The test should take about five minutes; failure should stop the workflow.
| Failure mode | What the user sees | Five-minute test | Stop if |
|---|---|---|---|
| 1. No audit trail | The answer exists, but the run, user, time, configuration, and review do not. | Replay the last completed run and inspect who initiated, changed, reviewed, and distributed it. | A screenshot or prompt transcript is the only record. |
| 2. Unverifiable output | The figure has a citation or chart, but no path to its source population and calculation. | Choose one material number and trace it to source rows, filters, joins, formula, currency, and period. | The system cannot reproduce the number from the cited records. |
| 3. Confident hallucination | A plausible number or driver appears without a deterministic calculation. | Recalculate a figure outside the language model, then change one input and rerun. | The figure changes without a traceable change in inputs or logic. |
| 4. No memory between sessions | A repeated workflow forgets prior assumptions, exceptions, corrections, or review decisions. | Close the session, reopen the same workflow, and retrieve the prior approved run and exception history. | Continuity depends on one person's chat history. |
| 5. Context re-entered every time | Every prompt restates revenue policy, entity mappings, fiscal calendar, and metric definitions. | Ask where the current revenue definition lives, who approved it, and which reports use its version. | The answer is another prompt or an analyst's memory. |
| 6. No role-based permissions | The agent can read payroll, customer, or entity data because the connection can. | Run the same request as two roles, inspect the agent identity, and review every tool and dataset touched. | Access is all-or-nothing or the agent has no attributable identity. |
| 7. Surface-level variance analysis | Commentary says costs rose or margin fell, then hands the investigation back to finance. | Require a reconciled driver bridge to transactions and leave any unexplained residual visible. | Named drivers do not tie to the reported variance. |
| Failure mode | Likely architectural cause | Recommended owner |
|---|---|---|
| 1. No audit trail | No durable run log or version history. | Finance process owner |
| 2. Unverifiable output | Narrative generation is detached from financial data lineage. | Controller and data owner |
| 3. Confident hallucination | Financial figures are accepted without a deterministic, reproducible calculation trace. | FP&A owner |
| 4. No memory between sessions | Conversation state is mistaken for durable workflow state. | Workflow owner |
| 5. Context re-entered every time | No governed semantic or ontology layer holds business meaning. | Metric owner |
| 6. No role-based permissions | The agent sits outside role-based access control and least privilege. | Security and data owner |
| 7. Surface-level variance analysis | The system summarizes a chart instead of tracing supported driver relationships. | FP&A owner |
Do not collapse these failures into trust. A prompt log cannot prove a number. Chat history cannot govern the definition of revenue. A manager's approval cannot authorize data access after the fact. Each defect needs its own owner and repair.
Why are an audit trail and a verifiable output different?
A run log tells you what happened. Verifiability tells you whether the result can be reproduced from evidence. A transcript may show that an analyst asked a question at 10:04 a.m. It cannot prove that the answer drew from the full ledger population, approved account mapping, and correct exchange rate.
Public Company Accounting Oversight Board (PCAOB) Auditing Standard 1105 offers a useful analogy: audit evidence must be relevant and reliable. When auditors use company-produced information, the standard requires them to test its accuracy and completeness, or test the controls over those attributes, and assess whether it is sufficiently precise and detailed. The rule applies to auditors. Here, it is a design test, not a claim of compliance.
Finance needs both records. The run log shows who acted, when, and what changed. Lineage shows the source records, definitions, and transformations behind the figure. Pluvo's 15-control AI governance checklist covers the broader control layer.
Why do confident financial numbers survive review?
The U.S. National Institute of Standards and Technology (NIST) Generative AI Profile calls confidently false output confabulation. NIST explains that confabulation follows from how generative models predict likely tokens, and warns that a model can invent the logic or citation used to justify a wrong answer. Confidence is presentation, not evidence.
Enter 18.4% actual gross margin and 17.1% plan. The answer is 1.3 percentage points, or 130 basis points. Change actual to 18.6%. The answer should move to 150 basis points, and the trace should identify the changed input. If the prose engine produces the math, the reviewer is checking a guess.
A bare LLM generates text by predicting tokens; do not treat its arithmetic output as a deterministic calculation. The Pluvo guide to hallucinated financial numbers explains the two-system control pattern. The test here is narrower: the figure must come from reproducible computation, not the sentence generator.
How are lost memory and missing business context different?
Session memory preserves interaction history. Governed business context preserves what the company means. The first remembers that an analyst excluded a one-time charge yesterday. The second knows which account qualifies, which entities the rule covers, who approved the definition, when the definition changed, and which workflows inherit it.
The two failures produce different bugs. Without session continuity, the workflow repeats exceptions and forgets corrections. Without governed context, the workflow remembers a wrong or stale instruction with perfect fluency. A longer context window can reduce repetition. It cannot turn an unowned prompt into company policy.
The 2024 Bank of England and Financial Conduct Authority (FCA) survey received 118 responses. Among firms using or planning to use AI over the next three years, 46% reported partial understanding of the technologies they used; 34% reported complete understanding, with respondents linking much of the gap to third-party models. The sample covers UK financial services, not corporate finance at large. It is evidence of an accountability gap, not a prevalence estimate for this audience.
Run two checks. Reopen an approved workflow and retrieve its exceptions. Then locate the current metric definition, approver, and downstream reports. One tests continuity; the other tests governance. The Pluvo guides to why AI pilots fail in finance and the ChatGPT control boundary show the larger architecture.
What breaks when a finance agent has the wrong permissions?
Give the same agent two identities: an FP&A analyst and a payroll-restricted user. The second request should fail before retrieval, and the log should name the agent, user, role, entity, dataset, and tool. Applying role-based access control only to the employee who launched the agent leaves the agent itself ungoverned.
The FSB's June 2026 consultation proposes least privilege for agents and subagents, plus records of intermediate steps, tools, and data accessed. It is not a binding international standard. It is a useful design check: the system should show both the denial and the attempted path.
Human approval does not cure unauthorized assembly. A reviewer cannot retroactively grant an agent permission to read data it should never have touched. First enforce the boundary. Then review the permitted output.
Why can AI variance analysis stop at what changed?
Surface-level variance commentary restates the chart: operating expense rose, gross margin fell, revenue missed plan. A finance-grade explanation must reconcile the total movement to named drivers supported by source records across systems and leave the unexplained residual in view. A reconciled bridge supports attribution; a causal claim needs additional evidence.
The five-minute test is a bridge. Ask the system to decompose a reported variance into named drivers, sum the drivers back to the total, open one driver to transactions, and show what remains unexplained. A plausible story that does not tie is not analysis. It is a lead for an analyst.
The Pluvo variance analysis runbook covers the operating workflow. The pass condition here is unforgiving: tie the drivers to the total and the transactions, or label the cause unknown.
What can AI not own, and what must stay human?
AI should not be assigned final accountability for materiality, policy, judgment, or distribution. A system can calculate a variance and retrieve a policy. A finance professional must set materiality, approve definitions, accept or reject unexplained residuals, and decide who may receive the answer.
The FSB consultation defines meaningful oversight by the human's ability, authority, and incentive to intervene; it also warns that a click can become rubber-stamping. Federal Reserve Board General Counsel Randall D. Guynn made the same boundary explicit in March 2026 testimony about supervisory AI tools: judgment and decisions remain with subject-matter experts.
| Human owner | Decision the person owns | Evidence needed |
|---|---|---|
| Finance process owner | Scope and execution of the run | Run history, exceptions, configuration, and change record |
| Metric owner | Definitions, mappings, assumptions, and versions | Governed definition and impact of changes |
| Data or security owner | Access remained within approved systems and records | Identity, role, scope, tool-access log, and denial events |
| Business reviewer | Output is decision-relevant and uncertainty is visible | Driver bridge, source links, caveats, and unresolved items |
What does an architecture that passes the seven tests look like?
Passing all seven tests requires three things to remain distinct: what the business means, where the number came from, and who was allowed to act. Put definitions and relationships in a governed record. Give every figure a reconstruction path. Give every person and agent a scoped identity, a log, and a gate for sensitive work.
Pluvo says its public architecture implements those records through a finance ontology for definitions and relationships, Lineage for source-to-figure reconstruction, and Controls for identity, scope, and review. Other architectures can pass. The requirement is the evidence, not the product name.
For one practical finance-systems lesson each week, join The Finance Engineering Newsletter.
A fluent answer wins the demo. The source rows decide whether it survives review.
Frequently asked questions
What are the biggest AI risks for finance teams?
The seven AI finance risks in this framework are missing audit trails, unverifiable outputs, confident hallucinations, lost session continuity, business context rebuilt in prompts, unscoped permissions, and variance analysis that names movement without proving cause.
Is an AI audit trail the same as financial data lineage?
No. An audit trail records who or what ran, when, under which configuration, and who reviewed it. Financial data lineage connects the reported figure to source records, definitions, transformations, and calculations. A production workflow needs both.
Is chat memory the same as governed business context?
No. Chat memory preserves conversation history. Governed business context stores approved metric definitions, entity relationships, account mappings, fiscal periods, versions, and policy outside a single conversation.
Should a language model calculate financial figures?
A language model can explain or draft around a figure, but the figure should come from a deterministic calculator or governed compute layer. The same inputs should reproduce the same result, with a trace to the source.
Who should approve AI-generated finance analysis?
Ownership should be explicit. The process owner confirms scope and execution, the metric owner approves definitions, the data or security owner confirms access, and the business reviewer decides whether the explanation is fit to distribute.
