September 15, 2026

Getting Your Model Validation Team to Yes on AI

Axle Team

Key takeaways

Model validation for AI compliance tools is a documentation and controls problem, not a math problem. Validators say no when they cannot see inputs, logic, outputs, and limits.
The April 2026 revised model risk management guidance (SR 26-2 / OCC Bulletin 2026-13) explicitly leaves generative and agentic AI out of scope, and puts the burden on your own governance to decide what applies.
Reframe the ask: validate the decision workflow and its controls (human review, sampling, outcome analysis), not the language model in isolation.
Bring the validation team a package: conceptual soundness memo, data lineage, back-test results against historical dispositions, ongoing monitoring plan, and vendor diligence.
Regulators have been consistent since 2018 that pilots will not by themselves draw supervisory criticism, and FinCEN's 2026 program rule proposal names AI use as a factor in enforcement decisions.

What model validation of AI compliance tools actually requires

Model validation for AI compliance tools means demonstrating to an independent model risk function that the tool is conceptually sound, that its outputs can be tested against ground truth, that its limitations are documented, and that ongoing monitoring will catch drift. For AI agents in AML, that typically means validating the end-to-end decision workflow and its controls rather than the underlying language model alone.

That last sentence is where most AI compliance projects stall. The compliance team wants a tool that clears alerts faster. The model validation team hears "black box making SAR-adjacent decisions" and reaches for the same framework it uses on a credit scorecard. Neither side is wrong. The job is to translate.

What changed in April 2026, and why it matters for your pitch

For fifteen years the operative text was SR 11-7 / OCC 2011-12, plus the April 2021 interagency statement clarifying how it applied to BSA/AML systems. On April 17, 2026, the Federal Reserve, OCC and FDIC replaced all of that with revised guidance (SR 26-2 for Fed-supervised institutions, OCC Bulletin 2026-13 for national banks). The OCC bulletin rescinds OCC 2011-12 and OCC Bulletin 2021-19 on BSA/AML model risk management outright.

The revised guidance restates the core definition in plainer language:

"Model validation evaluates whether models perform as expected and includes an assessment of a model's reliability and its limitations. The nature and rigor of validation generally align with the model's approach, use, and materiality."
— Federal Reserve, Revised Guidance on Model Risk Management (SR 26-2 attachment), April 2026

Two other points in the new text matter directly for AI in AML compliance.

Generative and agentic AI are explicitly out of scope

Footnote 3 of the revised guidance says:

"Generative AI and agentic AI models are novel and rapidly evolving. As such, they are not within the scope of this guidance. Nonetheless, a banking organization's risk management and governance practices should guide the determination of appropriate governance and controls for any tools, processes, or systems not covered in this document."
— Federal Reserve, Revised Guidance on Model Risk Management, April 2026

Do not read this as "AI agents are exempt from validation." Read it as: the agencies are handing the framework decision to your governance committee. A validation team that insists on running an LLM-based agent through a scorecard template is applying a framework the regulators themselves declined to apply. A compliance team that argues the agent needs no review is ignoring the second sentence. The productive middle is a bespoke control framework, approved by model risk, that borrows the parts of SR 26-2 that fit.

The guidance is principles, not requirements

The OCC bulletin states plainly that the guidance "does not set forth enforceable standards or prescriptive requirements; accordingly, non-compliance with this guidance will not result in supervisory criticism against a banking organization," and that it is "expected to be most relevant to banking organizations with over $30 billion in total assets" (OCC Bulletin 2026-13). If you run compliance at a community bank or a fintech partnering with one, your validation team has real latitude to size the review to the risk.

What validators need to say yes: the package

Validation teams reject AI compliance tools for the same handful of reasons. Each one has a concrete deliverable that answers it.

"We can't validate a black box" — What it really means: No visibility into inputs and reasoning. Deliverable that resolves it: Per-decision rationale logs; full data lineage from source systems to prompt to output.
"The vendor won't share the model" — What it really means: SR 26-2 vendor section applies. Deliverable that resolves it: Vendor documentation on conceptual soundness, design, and performance; your own outcome testing.
"Output isn't deterministic" — What it really means: Same alert could get a different narrative. Deliverable that resolves it: Define the decision variable (disposition, risk tier) and test its stability across reruns, not the prose.
"No ground truth" — What it really means: Nothing to back-test against. Deliverable that resolves it: Historical alert dispositions and QA overturn logs as the benchmark set.
"How do we know when it degrades?" — What it really means: No ongoing monitoring plan. Deliverable that resolves it: Sampling rate, overturn thresholds, drift triggers, and a re-validation cadence.

The vendor row deserves emphasis. The revised guidance is direct that proprietary components do not excuse you from validation:

"An important element of model risk management is the validation of vendor products, either by internal or outside parties. Sound practice includes developing an understanding of the vendor model, including its conceptual soundness, design, development data, and performance."
— Federal Reserve, Revised Guidance on Model Risk Management, April 2026

Any vendor that cannot produce that documentation is going to fail your validation review no matter how good the demo looked. Ask for it before the pilot, not after. Our vendor evaluation questions list covers what to request.

A step-by-step path to approval

Scope the model. Write down exactly what the AI decides and what a human decides. If the agent recommends a disposition and a reviewer approves it, the "model" is the recommendation engine plus the review control. Validate both.
Pick the decision variable. For alert investigation, that is usually the disposition (close / escalate) and the risk rationale category. Prose narratives are an output, not the variable you test.
Build the benchmark set. Pull 12–24 months of historical alerts with final dispositions, including QA overturns and any SARs filed. Stratify by typology and risk tier so the sample is not dominated by low-risk closures.
Run a shadow pilot. Let the agent process the benchmark set, or run it in parallel to live L1 review, with no production effect. Compare agreement rates, and more importantly, read every disagreement.
Document limitations. Where does the agent underperform? Thin-file customers, novel typologies, foreign-language documents? Write it down. Validators trust a limitations section more than a results section.
Write the ongoing monitoring plan. Sampling percentage, overturn-rate thresholds that trigger review, retraining or prompt-change control, and re-validation triggers.
Get effective challenge on record. The revised guidance describes effective challenge as "critical analysis conducted by objective experts who evaluate model risk and effect appropriate changes throughout the model lifecycle." Have model risk sign off on the framework, not just the result.

The practitioner insight: validate the workflow, not the model

Here is the workflow observation that changed how we approach these conversations. When compliance teams bring model risk an "AI model" to validate, the review goes in circles because there is no stable object to test. When they instead bring a documented decision workflow (alert in, data gathered, rationale produced, human review, disposition out, QA sample) with the AI as one component, the validation team already has a mental model for it. It looks like any other operational process with a system dependency.

The trade-off is real: framing it as a workflow means you cannot skip the human-review control in the first year, even if the agent is clearly good enough to auto-close low-risk alerts. That control is what lets model risk say yes. Earn the right to loosen it with monitoring data, not with a vendor slide. We wrote more about this in human-in-the-loop AML and AI audit trails for compliance.

Regulators have already said pilots are safe

Validation teams sometimes treat the pilot itself as a supervisory risk. The 2018 joint statement from the Fed, FDIC, FinCEN, NCUA and OCC addressed that directly, and it has never been withdrawn:

"Pilot programs undertaken by banks, in conjunction with existing BSA/AML processes, are an important means of testing and validating the effectiveness of innovative approaches. While the Agencies may provide feedback, pilot programs in and of themselves should not subject banks to supervisory criticism even if the pilot programs ultimately prove unsuccessful."
— Federal Reserve, FDIC, FinCEN, NCUA, OCC, Joint Statement on Innovative Efforts to Combat Money Laundering and Terrorist Financing, December 2018

The direction of travel has only strengthened. FinCEN's April 7, 2026 AML/CFT Program Rule NPRM proposes that, in deciding whether to pursue enforcement or a significant supervisory action against a bank, the Director would consider "whether the bank is employing innovative tools such as artificial intelligence that demonstrate the effectiveness of the bank's AML/CFT program" (FinCEN Program NPRM Fact Sheet). It is a proposal, not a rule, and the comment period closed June 9, 2026. But it tells your validation team which way the agencies are leaning.

How Axle supports the validation conversation

Axle's agents are built for exactly the package described above: every alert decision carries a structured rationale, source data references, and a full audit log, and the workflow keeps a human reviewer in the loop by design. That gives model risk a reviewable object rather than a demo. If you are sizing a pilot, the AI readiness playbook and the ROI calculator are useful inputs for the business case that goes alongside the validation memo.

FAQ

Does SR 11-7 still apply to AI compliance models?

No. SR 11-7 and OCC 2011-12 were superseded on April 17, 2026 by revised model risk management guidance (SR 26-2 / OCC Bulletin 2026-13). The 2021 interagency statement on BSA/AML model risk was rescinded as well. The revised guidance excludes generative and agentic AI from scope and directs institutions to set appropriate governance themselves.

Do AI agents in AML need independent model validation?

The regulators have not mandated a specific framework for agentic AI. Sound practice is to treat the agent-plus-human-review workflow as a model-like process, document its conceptual soundness and limitations, back-test against historical dispositions, and put an ongoing monitoring plan in place, with sign-off from an independent model risk function.

How long does model validation of an AI compliance tool take?

It depends on how complete your package is when you walk in. Teams that arrive with a benchmark set, a shadow-pilot readout, and a monitoring plan tend to move quickly; teams that arrive with a vendor deck spend months answering questions the deck should have answered.

Ready to build the validation package for your own AI pilot? See how the Axle platform is designed for auditability and human review: https://www.axleruns.com/platform

Getting Your Model Validation Team to Yes on AI

More from the blog