September 13, 2026

Data Requirements for AI Transaction Monitoring Pilots

Axle Team

Key takeaways

Most AI transaction monitoring pilots fail on data, not on models. The binding constraint is usually historical alert dispositions — your labels — not transaction volume.
Regulators have said pilots themselves should not draw supervisory criticism; what they will scrutinize is data quality and lineage, per SR 11-7's model development standards.
Scope the pilot's data to five domains: transactions, customer/KYC, alert and case history, screening results, and reference data — and document gaps rather than papering over them.
Disposition labels inherit the bias of the rules that generated them. A pilot that only replays old alerts can prove efficiency gains, but not detection gains.

What data does an AI transaction monitoring pilot need? Five domains: transaction records with counterparty detail, customer and KYC profiles, historical alerts with their investigation outcomes, sanctions and screening results, and reference data that ties them together. The scarcest input is usually well-documented alert dispositions — they are the labels the system learns from and is evaluated against.

Most teams scoping a pilot start by asking which model or vendor to test. The better first question is whether your own data can support an honest evaluation, because AI transaction monitoring pilot data problems — not model problems — are what quietly kill these projects.

The data an AI transaction monitoring pilot actually needs

Transactions — What it includes: Amounts, dates, channels, originator/beneficiary detail, cross-border flags. Why the pilot fails without it: The core signal; missing counterparty fields blind the system to network patterns.
Customer & KYC — What it includes: Risk rating, occupation/business type, expected activity, account opening data. Why the pilot fails without it: "Unusual for this customer" is undefined without an expected-activity baseline.
Alert & case history — What it includes: Historical alerts, dispositions, escalations, SAR decisions, narratives. Why the pilot fails without it: These are your labels — without them you cannot measure precision or SAR yield.
Screening & watchlist — What it includes: Sanctions/PEP screening results, prior matches and resolutions. Why the pilot fails without it: Investigative context an agent needs to disposition an alert the way an analyst would.
Reference data — What it includes: Entity resolution keys, account-to-customer mapping, org hierarchies. Why the pilot fails without it: Fragmented identity data turns one customer into five, and five risks into none.

Two practical notes on scope. First, depth matters more than breadth: a full year of history captures seasonality and recurring behavior in a way three months cannot. Second, completeness matters more than polish — a pilot can tolerate messy fields it knows about, but not silent gaps discovered mid-evaluation.

What regulators expect from pilot data

The good news first. The 2018 interagency statement from FinCEN, the Federal Reserve, FDIC, OCC, and NCUA removed the biggest perceived blocker:

"Pilot programs undertaken by banks, in conjunction with existing BSA/AML processes, are an important means of testing and validating the effectiveness of innovative approaches."
— Joint Statement on Innovative Efforts to Combat Money Laundering and Terrorist Financing (Dec. 2018)

And, critically:

"While the Agencies may provide feedback, pilot programs in and of themselves should not subject banks to supervisory criticism even if the pilot programs ultimately prove unsuccessful."
— Joint Statement (Dec. 2018)

The scrutiny lands elsewhere: on whether the data feeding the system is fit for purpose. If your institution follows SR 11-7 (or your model validation team does), expect the pilot's data package to be judged by its standard for model development:

"The data and other information used to develop a model are of critical importance; there should be rigorous assessment of data quality and relevance, and appropriate documentation."
— Federal Reserve SR 11-7, Supervisory Guidance on Model Risk Management (2011)

The FATF's 2021 report on opportunities and challenges of new technologies for AML/CFT makes the same point from the international side: new analytical tools are only as good as the data they run on, and it identifies data quality and data standardization as central operational challenges to adoption.

The practical translation: document your data lineage before the pilot starts. Where each field comes from, what its known gaps are, which proxies you used and why. That documentation is the difference between a pilot your validation team can approve and one they will make you rerun — and it is the first thing to have ready before evaluating any agentic compliance platform.

Labels beat volume — and your labels are biased

Here is the observation that separates teams who have run one of these from teams scoping their first: everyone over-invests in cleaning transaction data and under-invests in the quality of disposition history. But dispositions are the labels. An evaluation that says "the AI agreed with analysts on 9 of 10 closures" is only as meaningful as those original closures — if your historical QA found inconsistent dispositioning across analysts, that inconsistency is now baked into your benchmark.

The subtler trap is label bias. Your historical alerts exist because your current rules fired. A pilot evaluated only on that population can demonstrate efficiency — faster investigation of the same alerts, better alert dispositioning — but it cannot demonstrate improved detection, because activity your rules never flagged has no labels at all. If detection uplift is a pilot goal, budget for analyst review of a sample of below-threshold or never-alerted activity the system surfaces. That review time is a real cost; put it in the plan, not in the fine print.

A pre-pilot data readiness checklist

Pull 12 months of transactions, alerts, and dispositions into a static, versioned extract — evaluations against a moving dataset are unreproducible.
Reconcile record counts against source systems and document every known gap (missing counterparty names, unparsed wire memos, legacy system cutover dates).
Verify entity resolution: one customer, one key, across accounts and products.
Have a second reviewer re-disposition a sample of historical alerts to measure label consistency before you treat dispositions as ground truth.
Define evaluation metrics up front — precision, recall proxy, SAR yield, investigation time — and get model validation's sign-off on the design before the first result exists.
Settle data handling with your vendor in writing: environment, retention, access, and whether your data trains anything beyond your deployment.

FAQ

How much historical data does an AI transaction monitoring pilot need?

Enough to cover your major typologies and a full seasonal cycle — in practice, teams aim for around a year of transactions and alert dispositions. A shorter window can work for an efficiency-focused pilot, but it weakens any claim about detection performance across customer behavior cycles.

Can you run a pilot on synthetic or masked data?

Synthetic data is useful for integration testing and security review, but a credible performance evaluation needs your real alert population and real dispositions — tokenized or pseudonymized where needed. Synthetic data rarely reproduces the messy correlations that make transaction monitoring hard.

Will a pilot create regulatory exposure if it finds activity we missed?

The 2018 joint statement addressed this directly: when banks test AI-based monitoring and identify suspicious activity that existing processes missed, the agencies said they will not automatically assume the existing processes are deficient. You still need to disposition (and where warranted, report) what you find — plan analyst capacity for that.

If you are scoping a pilot, see how Axle's platform runs agentic investigation on top of your existing monitoring stack, and use the ROI calculator to size the efficiency case before you pull a single extract.

Data Requirements for AI Transaction Monitoring Pilots

More from the blog