Join our Newsletter — 33% off our NHI Course
Home› Guides› AI Model Risk Management Guide: Validating and Monitoring…
Guide AI Governance & Risk

AI Model Risk Management Guide: Validating and Monitoring Models

PDF for newsletter subscribers · free to read online

← All guides
By Lalit Choda, NHI Mgmt Group Updated 24 October 2026 14 min read
On this page

Machine learning and AI models now make or shape decisions on credit, fraud, pricing, hiring and security, yet many organisations validate them less rigorously than the spreadsheets they replaced. Model risk management (MRM) is the discipline banks built, under the US Federal Reserve's SR 11-7, to stop flawed or misused models causing losses. SR 11-7 was superseded on 17 April 2026 by SR 26-2, which explicitly leaves generative and agentic AI out of scope, so organisations now have to decide how far the classic MRM lifecycle extends to AI. This guide explains the current regulatory position in the US, UK, Canada and EU, how to adapt inventory, tiering, validation and monitoring to AI models, and where MRM stops and wider AI governance takes over. It is for model risk, validation, data science, security and audit teams, inside and outside financial services.

Key takeaways

  • SR 11-7 is no longer current. The Federal Reserve, OCC and FDIC replaced it on 17 April 2026 with revised, non-enforceable guidance (SR 26-2, OCC Bulletin 2026-13) aimed mainly at banks over $30 billion in assets, though SR 11-7's lifecycle remains the common vocabulary.
  • Generative and agentic AI fall outside the new US guidance. SR 26-2 footnote 3 excludes them as "novel and rapidly evolving"; their risk must be managed through broader AI governance, which is where most organisations need to do new work.
  • Other regulators go the other way. UK PRA SS1/23 has a sub-principle on AI and machine learning, and Canada's OSFI Guideline E-23, effective 1 May 2027, puts AI/ML squarely in scope.
  • Validation has three core parts that still apply. Conceptual soundness, outcomes analysis and ongoing monitoring, adapted for AI with bias, robustness, drift and explainability testing.
  • Effective challenge is the control that matters most. Reviewers need the expertise, independence and standing to change or stop a model, whatever the organisation chart says.

Current regulatory status

Regulatory status was checked on official sources on 24 October 2026. Several of these texts changed in 2026, so check the issuing body's page before relying on any of them.

InstrumentIssuerStatusScopeAI position
SR 11-7 Guidance on Model Risk Management (4 April 2011), with OCC Bulletin 2011-12Federal Reserve, OCCSuperseded and replaced by SR 26-2 on 17 April 2026; OCC Bulletin 2011-12 rescindedWas: supervised banking organisationsPredates modern AI; widely applied to ML by analogy
SR 26-2 Revised Guidance on Model Risk Management (17 April 2026), OCC Bulletin 2026-13Federal Reserve, OCC, FDICCurrent. States it does not set enforceable standards and non-compliance will not result in supervisory criticism, though unsafe or unsound practices remain actionableExpected to be most relevant to banking organisations over $30 billion total assetsNon-generative, non-agentic AI in scope; generative and agentic AI excluded (footnote 3)
SR 21-8 / OCC Bulletin 2021-19, BSA/AML model risk statementInteragencySuperseded April 2026, no replacementBSA/AML systemsn/a
Interagency AI request for informationOCC, Federal Reserve, FDICAnnounced 17 April 2026 as planned "in the near future"; we could not confirm on 24 October 2026 whether it has been issuedBank use of AI including generative and agenticLikely route for future US AI expectations
PRA SS1/23 Model risk management principles for banksBank of England PRAIn force since 17 May 2024; current version effective 23 April 2026 after low impact amendmentsUK banks, building societies and PRA-designated investment firms with internal model approval (IRB, IMA, IMM)Sub-principle on risks from AI in modelling techniques such as machine learning
OSFI Guideline E-23 Model Risk ManagementOSFI (Canada)Published 11 September 2025, effective 1 May 2027All federally regulated financial institutionsAI/ML explicitly covered; more AI/ML use calls for more mature governance
EU AI Act, Regulation (EU) 2024/1689 as amended by Regulation (EU) 2026/1744EUIn force; Annex III high-risk obligations apply from 2 December 2027Providers and deployers in the EU marketCredit scoring and life and health insurance pricing are Annex III high-risk uses

The EU obligations for high-risk systems (risk management, data governance, logging, human oversight, accuracy and robustness) overlap heavily with MRM and are set out in the EU AI Act Compliance Guide.

What changed from SR 11-7 to SR 26-2

TopicSR 11-7 (2011)SR 26-2 (2026)
Definition of modelQuantitative method applying statistical, economic, financial or mathematical theories; qualitative inputs covered if output is quantitativeA complex quantitative method applying statistical, economic or financial theories; excludes simple arithmetic, spreadsheets and deterministic rule-based processes
ProportionalityScaled to size, complexity and extent of model use; materiality a key considerationOverall model risk = inherent risk (assumptions, complexity, input quality, data constraints) in context of materiality (exposure and purpose); immaterial models may only need identification and monitoring
Effective challengeObjective, informed parties; depends on incentives, competence and influenceCritical analysis by objective experts with expertise, independence and standing to effect change
ValidationConceptual soundness, ongoing monitoring, outcomes analysis including back-testing; review at least annuallySame three elements in much less detail; emphasis on rigour and effectiveness over organisational structure
Third-party modelsVendor and consultant models validated like internal onesPrinciples remain applicable when vendors keep code or methods proprietary; customisations documented and justified
InventoryModels in use, under development and recently retiredSufficient information to understand individual and aggregate model risk
Generative and agentic AINot addressedOut of scope

For practitioners the lesson is that the regulatory floor in the US has dropped, while model complexity has risen. Organisations that dismantle SR 11-7 era controls because the guidance is now non-enforceable will be badly placed when the promised AI expectations arrive, and the PRA and OSFI texts already expect more.

Which AI systems belong in the MRM programme

Decide scope in writing. A workable rule: anything that produces a quantitative or classification output that drives a business decision is a model; generative and agentic systems are governed under the AI governance framework but borrow MRM validation and monitoring where they make or inform material decisions.

  • Classic MRM scope: credit scoring, pricing, capital and loss forecasting, fraud and AML transaction monitoring, gradient-boosted and neural network classifiers, anomaly detection used for security alerting.
  • Extended AI scope: large language model applications that summarise, classify or recommend in decision workflows, retrieval-augmented systems, and agents. Use the risk tiering approach in the AI Use Case Risk Tiering Guide to decide how much MRM rigour applies.
  • Vendor models: embedded scoring in SaaS platforms, third-party fraud scores and hosted foundation models. All are in scope; the evidence you can obtain differs. See the Third-Party AI Risk Guide.

The model risk lifecycle for AI

OSFI E-23 describes the lifecycle as design, review, deployment, monitoring and decommissioning, and notes the stages are not strictly sequential. The table shows what each stage needs for AI models in addition to classic MRM.

StageClassic MRM requirementAdditional for AI and MLEvidence
Identification and inventoryRegister every model with owner and useCapture foundation model and version, prompts or system instructions, retrieval sources, tools, fine-tuning dataInventory record
Risk ratingInherent risk and materialityAutonomy, customer impact, opacity, regulatory classification (EU AI Act Annex III)Rating rationale
Design and developmentPurpose, theory, data quality, testingTraining data lineage and consent, feature leakage tests, bias testing across protected groups, hyperparameter rationaleDevelopment documentation, model card
Independent validationConceptual soundness, outcomes analysis, ongoing monitoring planRobustness and adversarial testing, explainability review, red teaming for generative systems, reproducibility checkValidation report with findings and severity
Approval and deploymentApproval by authority matched to ratingChange control on model, prompt and retrieval corpus; deployment gates in MLOps pipelineApproval record, release tag
Ongoing monitoringPerformance against expectations, benchmarkingData and concept drift, fairness metrics over time, hallucination and refusal rates, input attack signalsMonitoring dashboard, breach log
DecommissioningRetire and keep in inventory for a periodRevoke model endpoints, API keys and service accounts; archive weights and data for auditRetirement record

OSFI E-23 also lists inventory fields worth adopting anywhere: model ID, name and description, risk rating, owner, developer and origin for every identified model, and for inventoried models version, deployment date, reviewer, approver, dependencies, data sources, approved uses, limitations, last and next review dates and monitoring status. The AI Register and AI Inventory Template extends these for generative AI.

Validating AI models

Conceptual soundness

  • Purpose fit: is the model type appropriate to the decision? A black-box deep model for an adverse credit decision that must be explained to the applicant is a design issue, not just a documentation issue.
  • Data: representativeness of training data for the population it will score, label quality, leakage of future information, and lawful basis for personal data use.
  • Explainability: global and local explanation methods appropriate to the audience; check explanations are stable. See the Explainable and Fair AI Guide.

Outcomes analysis

  • Back-testing on out-of-time samples not used in development, as SR 11-7 required and SR 26-2 retains.
  • Benchmarking against a simpler challenger model: if a logistic regression performs almost as well, the extra opacity may not be justified.
  • Segment analysis by customer group, channel and geography to find where performance or fairness breaks down.
  • Generative systems: task-specific evaluation sets with defined acceptance thresholds for accuracy, groundedness and harmful output, covering the NIST AI 600-1 risks relevant to the use, such as confabulation, data privacy, harmful bias and information security.

Ongoing monitoring

SR 26-2 notes that a model that no longer performs as expected may warrant overlays, adjustment or redevelopment. For AI, monitoring must catch change you did not make: shifts in input data, user behaviour, or a provider silently updating a hosted model. NIST AI RMF MEASURE 2.4 expects functionality and behaviour to be monitored in production, and MANAGE 4.1 expects post-deployment monitoring plans that cover user input, appeal and override, decommissioning, incident response, recovery and change management.

Security risks inside model risk

Classic MRM treats model failure as error or misuse. AI adds deliberate attack, and the MRM programme should require security testing for higher-rated models.

ThreatReferenceModel risk effectControl
Training data poisoningMITRE ATLAS AML.T0020 Poison Training DataModel learns attacker-chosen behaviour; validation passes on clean test dataData provenance, integrity checks on training pipelines. See the AI Data Poisoning and Backdoors Guide
Prompt injection in LLM applicationsOWASP Top 10 for LLM Applications 2025, LLM01Outputs and actions controlled by untrusted inputInput isolation, output validation, least-privilege tools
Supply chain compromise of models or librariesOWASP LLM03 Supply ChainUnvalidated third-party weights or packages in productionModel provenance, AI-BOM. See the AI Supply Chain and AI-BOM Guide
Credential exposure for model endpointsNIST SP 800-53 SA-9 External System ServicesUnauthorised use or tampering with hosted modelsScoped, rotated keys; workload identity

Model endpoints, feature stores and training pipelines run on service accounts and API keys. Recording those identities in the model inventory, and revoking them at decommissioning, is part of model risk control. See the AI Infrastructure Workload Identity Guide and the AI Model Security Guide.

Governance and effective challenge

  • Accountability: PRA SS1/23 expects responsibility for the overall MRM framework to sit with the most appropriate Senior Management Function holder. Name an equivalent executive owner outside the UK.
  • Three lines: model owners and developers (first line), independent validation and model risk (second line), internal audit (third line). SR 26-2 is less prescriptive about audit, but the separation still matters.
  • Standing: validators must be able to block deployment or impose use restrictions. If findings are routinely overridden by the business, challenge is not effective.
  • Committee link: high-rated AI models should go to the AI review committee as well as the model risk committee, with one decision record. See the AI Ethics Board and AI Review Committee Guide.
  • Aggregate risk: SR 26-2 and SR 11-7 both expect model risk managed in aggregate. Report concentration on a single foundation model provider or shared feature pipeline.

Implementing AI model risk management

  1. Write the scope decision: what counts as a model, and how generative and agentic AI are covered (MRM, AI governance, or both).
  2. Run an identification sweep using surveys, code repository and cloud ML service scans, and SaaS discovery. Triage findings into model, non-model and out of scope.
  3. Rate every model on inherent risk and materiality, and set review frequency and depth per rating. Start validation with the highest-rated unvalidated models.
  4. Pilot the AI validation standard on three models of different types (tabular ML, vendor score, LLM application) and refine test requirements before scaling.
  5. Automate monitoring in the MLOps pipeline: drift and performance thresholds with named owners and escalation.
  6. Set the rollback path for each high-rated model: previous approved version, champion model, rules-based fallback or manual process, with a tested switch-over and a documented authority to invoke it (NIST AI RMF MANAGE 2.4).
  7. Review annually and on regulatory change, which in 2026 means tracking the interagency AI RFI and the OSFI E-23 effective date.

Detection and measurement

  • Inventory completeness: models found by discovery sweeps that were not in the inventory (target falling to zero).
  • Validation coverage: share of high-rated models with a current independent validation (target 100 per cent).
  • Overdue findings: high-severity validation findings past their remediation date.
  • Monitoring breaches: number of drift or performance threshold breaches, and median time to decision (accept, overlay, retrain, withdraw).
  • Use outside approval: instances of a model used for a purpose not in its approved uses.
  • What to log: model version and prompt version per prediction or response for high-rated models, input data statistics, overrides by humans, and any provider model change notices.

Framework mapping

FrameworkReferenceMRM activity
NIST AI RMF 1.0GOVERN 1.3, GOVERN 1.6, MEASURE 2.4, MANAGE 1.3, MANAGE 2.4, MANAGE 4.1Proportionality, inventory, production monitoring, risk response, override and deactivation, post-deployment plans
NIST AI 600-112 generative AI risks; GV-1.3-001, GV-1.6-001Validation scope for generative systems, risk tiers, inventory
NIST CSF 2.0ID.AM-02, ID.AM-08, ID.RA-05, GV.SC-07, DE.CM-09Inventory, lifecycle management, inherent risk, supplier models, runtime monitoring
NIST SP 800-53 Rev. 5RA-3, CA-7, CM-3, SA-11, SR-6, PM-5Risk assessment, continuous monitoring, change control, developer testing, supplier review, system inventory
PRA SS1/23Principles 1 to 5Identification and classification, governance, development, independent validation, mitigants
OSFI E-23Model risk rating, lifecycle, inventoryRating-driven governance including AI/ML

For the surrounding programme see the AI Governance Guide, the NIST AI RMF Guide and the AI Risk Assessment Guide.

Common failure patterns

  • Reading non-enforceable as optional: cutting validation because SR 26-2 removed supervisory criticism, while unsafe or unsound practice remains actionable and other regulators expect more.
  • Generative AI in a gap: excluded from MRM because SR 26-2 excludes it, and not picked up by AI governance either.
  • Validating once: a strong pre-deployment validation with no drift monitoring, so a model degrades for a year unnoticed.
  • Vendor opacity accepted: third-party scores used without performance data or customisation records, despite guidance that MRM principles still apply.
  • Unmonitored customer-facing outputs: the Air Canada chatbot ruling made the airline liable for a bot's wrong policy answer, the kind of error outcome monitoring and accuracy testing exist to catch.
  • Orphaned endpoints: retired models whose API keys and service accounts stay live.

Practitioner checklist

  • Update policies that cite SR 11-7 to reflect SR 26-2, and record your own position on generative and agentic AI.
  • Define "model" in writing, including how LLM applications and agents are treated.
  • Maintain one inventory with owner, rating, version, data sources, approved uses, limitations and identities used.
  • Rate models on inherent risk and materiality and set validation depth and frequency per rating.
  • Validate conceptual soundness, outcomes and monitoring plans, adding bias, robustness and security testing for AI.
  • Give validators documented authority to block deployment or restrict use.
  • Monitor drift, performance and fairness in production with thresholds, owners and escalation.
  • Require vendor performance evidence and change notices, including for hosted foundation models.
  • Test a rollback or fallback for each high-rated model.
  • Track the interagency AI RFI, PRA SS1/23 amendments, OSFI E-23 and EU AI Act dates.

Standards and references

Related NHI Mgmt Group resources: AI Governance Guide · AI Use Case Risk Tiering Guide · Explainable and Fair AI Guide · EU AI Act Compliance Guide. This guide summarises model risk management guidance and related AI regulation as of 24 October 2026 and is not legal advice.

PDF for subscribers

Take this guide with you

Every section, table and checklist in one PDF, with links to the sources and related guides. Subscribe to our weekly NHI & AI Identity Journal and we’ll email you a download link.

Free weekly newsletter

Subscribe to the NHI & AI Identity Journal

The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.

Bonus 33% off our NHI Course when you subscribe.

NHIMG Editorial Note
Written and reviewed by Lalit Choda, NHI Mgmt Group. Last updated 24 October 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org

PDF Downloads for Newsletter Subscribers

Get the guide

Enter your details and we’ll email you a download link.

The full guide is free to read online. The PDF is a thank you for subscribing.

Fill in all fields marked * and tick the PDF box to continue.