Summary
Monitoring is where model risk management earns its keep. A validated model that is not monitored is a snapshot; a monitored model is a control.
Population stability
Population Stability Index (PSI) and comparable measures detect shifts in the underlying data. Thresholds should be set on the basis of historical stability, not by convention.
Discrimination and calibration
Discrimination metrics (KS, AUC, Gini) and calibration analyses answer complementary questions. Both should be tracked with defined thresholds.
Segment-level monitoring
Aggregate monitoring can mask segment-level deterioration. Where segments drive materially different decisions, they should be monitored separately.
Triggers
Monitoring is only useful if breaches trigger action: enhanced review, recalibration, revalidation or use restriction. Trigger paths should be documented.
Change management
Model change following monitoring findings is itself governed: impact assessed, targeted validation performed, approval sought.
Limitations
Monitoring detects deterioration; it does not diagnose cause. Diagnostic investigation is a distinct activity that draws on monitoring signals.
Related expertise
See Model Risk and Validation.
Current compliance reading
About the author
Jonas Adam Mohamed Osman Abdelghafour is a Chief Risk Officer, Risk & Compliance Director and governance expert advising banks, insurers and regulated firms. See the professional profile, about and qualifications.
Frequently asked questions
What should risk leaders know about population stability?
Population Stability Index (PSI) and comparable measures detect shifts in the underlying data. Thresholds should be set on the basis of historical stability, not by convention.
What should risk leaders know about discrimination and calibration?
Discrimination metrics (KS, AUC, Gini) and calibration analyses answer complementary questions. Both should be tracked with defined thresholds.
What should risk leaders know about segment-level monitoring?
Aggregate monitoring can mask segment-level deterioration. Where segments drive materially different decisions, they should be monitored separately.
What should risk leaders know about triggers?
Monitoring is only useful if breaches trigger action: enhanced review, recalibration, revalidation or use restriction. Trigger paths should be documented.
What should risk leaders know about change management?
Model change following monitoring findings is itself governed: impact assessed, targeted validation performed, approval sought.