September 14, 2026

TM Alert Quality Metrics: Precision, Recall, and SAR Yield

Axle Team

Key takeaways

Alert volume and backlog age measure workload, not effectiveness. The transaction monitoring metrics that withstand examiner scrutiny are alert precision, estimated recall, and SAR yield.
FFIEC examiners test whether your monitoring system "effectively detects unusual or suspicious activity" — and they trace deficiencies back to inappropriate filters, weak risk assessment, and poor decisioning.
SR 11-7 outcomes analysis applies squarely to TM scenarios: compare what the model flagged against what was actually suspicious, on an ongoing basis, not once a year.
Precision gains are easy to fake by raising thresholds. Never approve a tuning change on conversion-rate evidence alone; pair it with below-the-line testing in the same cycle.
SAR yield extends past the filing. FinCEN's FY2025 Year in Review shows SARs remain a primary trigger for federal investigations, which is the outcome the whole pipeline exists to produce.

Transaction monitoring metrics that actually measure effectiveness

The transaction monitoring metrics that matter are alert precision (the share of alerts that become SARs or confirmed true positives), recall (the share of genuinely suspicious activity your scenarios catch, estimated through below-the-line testing), and SAR yield (whether the filings you produce are usable downstream). Volume metrics — alerts per analyst, backlog age, time-to-close — measure workload, not effectiveness.

Most TM dashboards are built around the workload metrics because they are easy to pull from the case manager. But no examiner has ever concluded a monitoring system was adequate because alerts were closed quickly. The FFIEC BSA/AML Examination Manual is explicit that testing targets detection, not throughput:

"Transaction testing of suspicious activity monitoring systems and reporting processes is intended to determine whether the bank's policies, procedures, and processes are adequate and effectively implemented."
— FFIEC BSA/AML Examination Manual, Suspicious Activity Reporting examination procedures

The same procedures direct examiners to sample accounts independently, identify unusual activity themselves, and then check whether your system caught it:

"From the results of the sample, determine whether the transaction or surveillance suspicious activity monitoring system effectively detects unusual or suspicious activity. Identify the underlying cause of any deficiencies in the monitoring systems (e.g., inappropriate filters, insufficient risk assessment, or inadequate decision-making)."
— FFIEC BSA/AML Examination Manual

That is a recall test, run against your program from the outside. If you are not running the equivalent test internally, the exam is the first time you learn the answer.

The three metrics, defined precisely

Alert precision — What it measures: Share of alerts worth generating. How to compute it: SARs filed (or confirmed true positives) ÷ total alerts, by scenario and segment. What to watch: Trend by scenario; a scenario whose precision is near zero for several cycles is a tuning or design problem.
Estimated recall — What it measures: Share of suspicious activity actually caught. How to compute it: Below-the-line testing: score activity under current thresholds and sample it for disposition; add lookback and 314(b)/subpoena-triggered SARs missed by monitoring. What to watch: Any SAR whose origin was a subpoena, negative news, or fraud referral — not a TM alert — is a recall miss.
SAR yield — What it measures: Whether filings are useful downstream. How to compute it: Law-enforcement inquiries and keep-open requests tied to your SARs; continuing-activity reviews that escalate. What to watch: A rising SAR count with zero downstream signal suggests defensive filing, not effectiveness.

Precision and recall trade off against each other through thresholds — which is why reporting either one alone is misleading. Report them as a pair, per scenario, per customer segment.

Your tuning program is a model — treat the metrics as outcomes analysis

Scenario thresholds, segmentation logic, and any ML-based alert scoring all fall under model risk management expectations. The Federal Reserve's SR 11-7 names the discipline directly:

"The third core element of the validation process is outcomes analysis, a comparison of model outputs to corresponding actual outcomes. … Outcomes analysis should be conducted on an ongoing basis to test whether the model continues to perform in line with design objectives and business uses."
— Federal Reserve SR 11-7, Supervisory Guidance on Model Risk Management

For a TM program, "model outputs" are alerts and "actual outcomes" are investigator dispositions and SAR decisions. Precision and recall are not vendor marketing language; they are the outcomes analysis SR 11-7 expects. This framing also settles a recurring argument with model validation teams: monthly precision/recall reporting by scenario is not gold-plating, it is the ongoing monitoring the guidance describes. If you are evaluating AI in this stack, our AI readiness playbook covers how to present agentic tooling within an SR 11-7 framework.

SAR yield: the metric almost nobody closes the loop on

Filing volume keeps climbing. FinCEN's Year in Review for FY 2025 reports SAR filings of 4.3 million in FY2022, 4.6 million in FY2023, 4.7 million in FY2024, and 4.8 million in FY2025. The same report shows the downstream value: in FY25, 11.7% of all investigations opened by IRS Criminal Investigation originated from a BSA filing, and 89.3% of those originated from a SAR (FinCEN Year in Review FY2025).

You cannot see inside law enforcement's use of your filings, but you can track proxies: keep-open letters, grand jury subpoenas that follow your filings, 314(a) matches on subjects you filed on, and whether continuing-activity reviews escalate or peter out. A program whose SARs never generate any downstream signal should ask whether it is filing on the right activity — or just filing.

A practitioner's warning: precision improvements are easy to fake

Here is the workflow observation that a metrics glossary won't give you. When a team is drowning in alerts, the fastest way to make the precision number improve is to raise thresholds — and conversion climbs while recall quietly collapses. The tuning deck looks great; the exam does not.

The discipline that separates strong programs is procedural: no threshold change gets approved on conversion-rate evidence alone. Every tuning proposal must ship with below-the-line sampling from the same cycle showing what the new threshold would have suppressed. Internally, we've found the most predictive single indicator of how a validation will go is the share of tuning changes in the last year that carried paired BTL evidence. Programs where that share is high rarely get findings on tuning governance.

This is also where investigation capacity distorts everything. Teams under alert pressure close borderline alerts faster and file fewer SARs, which silently deflates precision measurements. Taking investigation throughput off the critical path — which is exactly what an AI investigator like Rose, Axle's transaction monitoring agent, is built to do — removes the incentive to buy relief through thresholds, so tuning decisions can be made on detection evidence instead of capacity constraints. You can estimate what that capacity shift is worth for your alert volumes with the ROI calculator.

How to stand up this reporting in one quarter

Instrument dispositions. Make investigators record a structured outcome (true positive / false positive / data-quality) at alert close, not just a narrative.
Build the precision view. SAR conversion and true-positive rate by scenario, segment, and month. Flag scenarios with sustained near-zero precision.
Stand up below-the-line testing. Each cycle, sample activity just under current thresholds and disposition it like a live alert. This is your recall estimate.
Tag SAR provenance. Record whether each SAR originated from a TM alert, a referral, a subpoena, or negative news. Non-TM-originated SARs are your measured recall misses.
Close the yield loop. Log keep-open requests, subpoenas, and 314(a) matches against prior filings, and report the trend to the BSA committee quarterly.

FAQ

What is a good SAR conversion rate for transaction monitoring?

There is no regulatory benchmark, and chasing a universal number is a mistake — conversion varies legitimately by scenario type, customer base, and risk appetite. Examiners care whether you know your rates by scenario, can explain outliers, and act on scenarios that produce almost nothing over multiple cycles.

What is below-the-line testing in AML?

Below-the-line testing samples activity that scored just under your alerting thresholds and investigates it as if it had alerted. If sampled below-the-line activity keeps producing SAR-worthy cases, your thresholds are suppressing true positives and recall is worse than your dashboard implies.

How often should transaction monitoring metrics be reviewed?

Precision and alert-volume trends belong in monthly operational reporting; below-the-line testing and threshold revalidation typically run on a periodic tuning cycle aligned with your model risk management schedule. SR 11-7's expectation is that outcomes analysis is ongoing — an annual tuning exercise alone does not meet it.

Measure the pipeline, then fix its bottleneck. Most programs discover their metrics are distorted by investigation capacity, not detection logic. See how Rose, Axle's AI transaction monitoring investigator, clears that bottleneck so your precision and recall numbers reflect reality.

TM Alert Quality Metrics: Precision, Recall, and SAR Yield

More from the blog