AML model validation is the independent review that confirms a financial institution’s automated transaction monitoring system actually detects the money laundering and terrorist financing activity it’s designed to catch. Federal regulators expect banks and credit unions to prove these systems work, and the interagency Supervisory Guidance on Model Risk Management, updated in April 2026, sets the framework. Validation is more than checking that the software runs. It tests whether the system’s logic is sound, whether its outputs match real-world outcomes, and whether the data flowing into it is clean and complete.
The Guidance That Sets the Bar
The relevant framework was reissued in April 2026 by the OCC, Federal Reserve, and FDIC. The OCC published it as Bulletin 2026-13, which rescinds the older OCC Bulletin 2011-12.1Office of the Comptroller of the Currency. OCC Bulletin 2026-13 – Model Risk Management: Revised Guidance The Federal Reserve issued the same guidance as SR 26-2, replacing SR 11-7.2Federal Reserve. Revised Guidance on Model Risk Management References to “SR 11-7” or “OCC 2011-12” in older vendor documentation are outdated.
One caveat matters for how you read the rules below. The revised guidance explicitly states that it “does not set forth enforceable standards or prescriptive requirements” and that “non-compliance with this guidance will not result in supervisory criticism.”1Office of the Comptroller of the Currency. OCC Bulletin 2026-13 – Model Risk Management: Revised Guidance In practice, examiners use it as a benchmark. Falling short won’t automatically trigger a formal finding, but it draws scrutiny and pointed questions during an exam. The Bank Secrecy Act itself, under 31 U.S.C. § 5318(h), is what actually compels the underlying AML program.3Office of the Law Revision Counsel. 31 USC 5318 – Compliance, Exemptions, and Summons Authority
Does Your System Even Count as a Model
Not every piece of compliance software falls under the guidance. The revised guidance defines a model as “a complex quantitative method, system, or approach that applies statistical, economic, or financial theories to process input data into quantitative estimates.” Simple arithmetic calculations, basic spreadsheets, and deterministic rule-based processes without a statistical or economic foundation are explicitly excluded.4Office of the Comptroller of the Currency. Supervisory Guidance on Model Risk Management
A rule that flags every wire over a fixed dollar amount might not qualify. A system that scores transactions using algorithms trained on historical suspicious-activity data almost certainly does. Most commercial AML transaction monitoring platforms fall within the definition because they rely on statistical methods to assign risk scores, segment customers, or calibrate detection thresholds.
The Three Components of a Valid Validation
The revised guidance organizes validation around three components. Skip one and you leave a gap examiners will notice.
Conceptual Soundness
Conceptual soundness asks whether the model’s design makes sense. Validators assess the key modeling choices, assumptions, qualitative judgments, and data selection behind the system.5Federal Reserve. Supervisory Guidance on Model Risk Management For an AML transaction monitoring system, that means checking whether the detection rules and risk-scoring algorithms match the institution’s actual products, customer base, and geographic risk profile. A model built for a retail bank with domestic consumer accounts may be poorly suited to a correspondent banking operation with heavy cross-border wire volume. The validator looks for evidence that the model’s designers thought those distinctions through.
Outcomes Analysis
Outcomes analysis compares what the model predicted or flagged against what actually happened. The guidance describes this as comparing “model outputs to corresponding real-world outcomes to assess model performance relative to model objectives and business use.”5Federal Reserve. Supervisory Guidance on Model Risk Management In an AML context, that means examining whether alerts led to actual Suspicious Activity Report filings, and whether known suspicious activity was missed. Persistent deviations outside expected ranges signal that recalibration or redevelopment may be warranted.
Ongoing Monitoring
Ongoing monitoring evaluates whether the model continues to perform given changes in products, customer types, transaction volumes, or market conditions.5Federal Reserve. Supervisory Guidance on Model Risk Management A model that worked well two years ago can degrade as customer behavior shifts or new payment channels emerge. This component also covers process verification, confirming the model is still being used within the boundaries its developers intended.
Who Can Actually Do the Validation
The people reviewing the model cannot be the same people who built or operate it. The guidance frames this through “effective challenge,” which requires critical analysis by individuals with three qualities: expertise to evaluate model risk, sufficient independence to remain objective, and enough organizational standing to force changes when problems surface.4Office of the Comptroller of the Currency. Supervisory Guidance on Model Risk Management
That third element is the one institutions most often underestimate. Hiring an independent validator who identifies serious flaws accomplishes nothing if the findings get buried because the validator lacks authority to push remediation. The guidance also flags reporting-line conflicts, such as development and validation teams reporting to the same executive. Examiners look at reporting structure closely when assessing whether independence is genuine or cosmetic.
Vendor and Third-Party Models
Most institutions buy their AML monitoring software rather than build it. Purchasing does not shift the validation obligation to the vendor. The institution remains fully responsible for validating the model as used in its own environment.
Vendors often treat their detection algorithms as proprietary and may limit access to underlying code or detailed methodology documentation. Institutions should negotiate access to developmental evidence during procurement, including documentation of the product’s design, intended use, assumptions, and limitations. Any customization the institution applies to the vendor product, such as modified thresholds or custom risk-scoring rules, must be documented and justified as part of the validation. You should also maintain documentation of the model’s logic independent of the vendor’s ongoing involvement, in case the vendor discontinues support or goes out of business.
Documentation to Assemble Before Testing
Assembling documentation is often the most time-consuming phase. At a minimum, prepare:
- Data dictionaries defining every field in the transaction monitoring database, including account types, transaction codes, and beneficiary identifiers.
- Current system configuration settings, including all detection scenarios and their associated thresholds.
- Historical alert and filing data showing which alerts the model generated, how they were dispositioned, and which resulted in SAR filings.
- Model documentation from the vendor or internal teams explaining the algorithms, formulas, and assumptions behind the risk-scoring methodology.
- Change logs providing an audit trail of every modification to the system’s logic, thresholds, or data inputs, with dates and justifications.
Validators trace transactions from entry point to final alert disposition, checking data integrity at each stage. Gaps in data, unexplained field mappings, or missing change documentation are common findings that delay validation and weaken its conclusions.
The Testing Itself
Once documentation is in place, validation moves into hands-on testing. Two testing methods do most of the work, followed by tuning based on what they reveal.
Above-the-Line Testing
Above-the-line testing examines transactions the model flagged. Validators manually recalculate results to confirm the software followed its own internal logic when generating each alert. If a rule says a wire over a certain dollar amount to a high-risk jurisdiction should trigger review, above-the-line testing checks that the math worked and the alert fired for the right reasons. Discrepancies between model output and manual recalculation point to coding errors or data integrity problems.
Below-the-Line Testing
Below-the-line testing is where most compliance teams get nervous, with good reason. This test pulls a sample of transactions that fell just short of triggering an alert and asks whether any should have been flagged. If the threshold for structuring detection is set at a certain level and below-the-line testing reveals clearly suspicious patterns just underneath that level, the threshold is too high. This analysis is critical for assessing whether the model is missing real threats due to overly narrow settings.
Threshold Tuning
Threshold tuning adjusts the model’s sensitivity based on validation findings. The goal is to reduce false positives that waste analyst time without letting genuinely suspicious transactions slip through. Statistical metrics like the F1 score help quantify the trade-off between catching true positives and minimizing false alarms. Tuning is not a one-time exercise. It should happen in response to validation findings, changes in the institution’s risk profile, or shifts in transaction patterns.
After testing, the validator drafts a formal report documenting findings, weaknesses, and recommendations. That report should be presented to senior management and the board, and must be available for regulators on request.
How Often to Validate
The revised guidance does not mandate a fixed schedule. The OCC has specifically noted that its guidance “does not, and should not be interpreted to, require community banks to perform annual model validation.”1Office of the Comptroller of the Currency. OCC Bulletin 2026-13 – Model Risk Management: Revised Guidance Frequency should be risk-based, driven by the model’s complexity, the institution’s risk profile, and the pace of business change.
Certain events warrant fresh validation regardless of the calendar: a merger or acquisition that changes the customer base, new products or services, expansion into new geographic markets, or material changes to the model’s logic. Many larger institutions settle on a biennial cycle with interim monitoring. There is no one-size-fits-all answer, but the institution needs to justify its chosen frequency to examiners with a rationale tied to its specific risk profile.
Machine Learning and AI Models
A growing number of institutions incorporate machine learning into AML monitoring, using algorithms that learn from historical data rather than relying solely on fixed rules. These models present distinct challenges because their internal decision-making is often opaque. A traditional rule-based system can be traced step by step; a neural network that assigns risk scores cannot be reverse-engineered as easily.
The revised 2026 guidance explicitly states that “generative AI and agentic AI models are novel and rapidly evolving” and places them outside its scope.1Office of the Comptroller of the Currency. OCC Bulletin 2026-13 – Model Risk Management: Revised Guidance That doesn’t mean institutions using generative AI for compliance workflows get a free pass. Expectations for those specific technologies are still developing, and institutions should expect heightened scrutiny in the interim.
For non-generative machine learning models used in transaction monitoring, validation emphasizes conceptual soundness more heavily than re-performance testing. Replicating the exact output of a complex algorithm is often impractical, so validators focus on confirming the algorithms are appropriate for the task, that the input variables make statistical and business sense, and that parameter and hyperparameter tuning was thorough. Explainability is a growing regulatory concern. An inability to explain why the model flagged a particular transaction can be treated as an inadequate governance finding.
Fixing What Validation Finds
A validation report that identifies deficiencies is only useful if the institution acts on it. Remediation should address the root cause of each finding rather than applying surface-level fixes. If a tuning issue stems from stale customer segmentation data, adjusting the threshold without fixing the data problem means the issue will resurface.
Sound remediation practice includes documenting each finding, assigning ownership to a specific individual, setting a realistic deadline, and retesting after the fix to confirm the deficiency is actually resolved. The retesting step is often skipped, which is a mistake. Auditable evidence that fixes were applied and verified, including logs, before-and-after screenshots, and updated configuration records, should be maintained for regulatory review.
Why Validation Gaps Matter
The model risk management guidance itself is not enforceable, but the underlying BSA requirements are. On the civil side, FinCEN can impose penalties under 31 U.S.C. § 5321. Willful violations carry inflation-adjusted penalties from $71,545 to $286,184 per violation. Negligent violations carry up to $1,430, and a pattern of negligent activity can reach $111,308. Certain due diligence violations can reach $1,776,364 per violation, assessable for each day the violation continues.6eCFR. 31 CFR 1010.821 – Penalty Adjustment and Table
Criminal penalties apply when violations are willful. Under 31 U.S.C. § 5322, a willful BSA violation carries up to $250,000 in fines and five years in federal prison. If it occurs as part of a pattern of illegal activity involving more than $100,000 over 12 months, the maximums rise to $500,000 and ten years.7Office of the Law Revision Counsel. 31 USC 5322 – Criminal Penalties Courts can also order forfeiture of profits from the violation and repayment of any bonuses received during the year of the offense. A faulty monitoring system that regulators have previously flagged, without a credible validation behind it, is the kind of evidence that turns a compliance gap into a willfulness finding, with personal liability for officers who knowingly allow the deficiency to continue.