As banks continue to adopt ML and AI to augment or replace legacy Fraud and AML transaction monitoring solutions, network analysis and case management processes, their use of such models will only continue to increase in complexity, with the potential for data and PII misuse or loss, bias, model drift and tagged data becoming stale. The need to adapt these models to rapid and emerging threats further burdens operations teams through high false positives, data quality degrading, documentation becoming outdated, and models losing their original risk intent and rationale.
A new approach
Perhaps recognizing the prevalence of these risks and the current pace of technological change, the Prudential Regulation Authority released new model risk management guidance on how banks should properly assess the efficacy of second-generation solutions.
This begs the question: with the Prudential Regulation Authority releasing its Supervisory statement on Model Risk Management (MRM) and it coming into force on May 17, 2024 – can UK Financial Crime fighters look to PRA SS1 /23 to support a more agile approach to its model governance?
The answer to this question hinges on whether institutions successfully develop new MRM frameworks and model testing approaches that align with the following SS1/23’s key principles:
- Model Identification and Risk Classification
- Governance
- Model Development, Implementation and Use
- Independent Model Validation
- Model Risk Mitigants
We know from experience that MRM teams are predominantly focused on Financial (Capital adequacy, Liquidity, VaR…) models, and there tends to be a distinct lack of domain knowledge to constitute a clear rationale to support oversight, guidance and governance of Financial Crime risks these models attempt to mitigate. The challenge for MRM teams is therefore a lack of talent both from an AI/ML perspective and Financial Crime domain knowledge. This generally necessitates that independent validation and oversight be delegated to external providers such as Davies, or else internal operational demands fester to create backlogs and slow down periodic assessments.
Evaluating models
To avoid these pitfalls, it is critical that banks adopt a cross-functional approach when evaluating their Financial Crime processes and models. Operational and technical SMEs need to work with MRM to ensure they are aligned with their organisation on the newly promulgated SS1/23 key principles and supporting process. This in turn enables a standardised approach and taxonomy of models that helps rationalise and support the speed of adoption and consistency of change; whilst providing an enhanced knowledge base.
As FI’s operate more and more in Agile Value Streams, embedding the five SS1/23 principles into, for instance an MLOps type process could help instil governance oversight – risk classification and reviews by the Business, QA and automated validation, with documented changes captured in knowledge bases – thereby empowering organizations to proactively manage models more efficiently and with greater input from domain experts, whilst validation is automated (measured against metrics and input from MRM) as a part of continuous improvement providing oversight on the life-cycle of each model.
I’m sure in the short term, this idea presents a burden and increased cost to resources; however, long-term benefits will ensure a more seamless and unified approach to ongoing MRM and a reduction in the backlog.
Why Davies
Davies regularly supports clients through periodic review (e.g. DFS part 504), expansion of model scenarios, data completeness and validation of current models across their Market Abuse, Fraud and AML operations, where we may or may not be engaged by an MRM team.
