best practices in model risk management - sas · best practices in model risk management. meeting...
TRANSCRIPT
![Page 1: Best Practices in Model Risk Management - SAS · Best Practices in Model Risk Management. MEETING REGULATORY GUIDELINES GLOBALLY AND LOCALLY ... Bank Risk Conference presentation](https://reader031.vdocuments.mx/reader031/viewer/2022011815/5e61524ffc560375f27abe2a/html5/thumbnails/1.jpg)
Sridhar Sourirajan, FRM
Director, Model Risk Management
SAS Institute, North Carolina, USA
Best Practices in Model Risk Management
![Page 2: Best Practices in Model Risk Management - SAS · Best Practices in Model Risk Management. MEETING REGULATORY GUIDELINES GLOBALLY AND LOCALLY ... Bank Risk Conference presentation](https://reader031.vdocuments.mx/reader031/viewer/2022011815/5e61524ffc560375f27abe2a/html5/thumbnails/2.jpg)
MEETING REGULATORY GUIDELINES GLOBALLY AND LOCALLY
Model risk management Regulation tightened first in US
EMEA is now following
EBA SREP
Guidelines
(Dec 2014)
Local Regulator
Implementation
(Jan 2016)
Approximately 3
pages focus on
model risk
management
Polish Komisja Nadzoru
Finansowego
(Jun 2016)
SAS Model Risk Management
![Page 3: Best Practices in Model Risk Management - SAS · Best Practices in Model Risk Management. MEETING REGULATORY GUIDELINES GLOBALLY AND LOCALLY ... Bank Risk Conference presentation](https://reader031.vdocuments.mx/reader031/viewer/2022011815/5e61524ffc560375f27abe2a/html5/thumbnails/3.jpg)
PRACTICAL CUSTOMER ISSUES
MODEL RISK MANAGEMENT
![Page 4: Best Practices in Model Risk Management - SAS · Best Practices in Model Risk Management. MEETING REGULATORY GUIDELINES GLOBALLY AND LOCALLY ... Bank Risk Conference presentation](https://reader031.vdocuments.mx/reader031/viewer/2022011815/5e61524ffc560375f27abe2a/html5/thumbnails/4.jpg)
CUSTOMER ISSUES: GAP ANALYSIS VS CURRENT APPROACH
Model Inventory - No consolidated record
of potentially 1,000+ material models
MRM Procedures- Multiple model
validation and review processes across the
Group (and of variable quality)
Compliance Documentation - Lack of
documentary evidence of MRM processes
across group & model lifecycle
Integrated Groupwide
Target Approach
Model
1
Max
Model 4Model
2
Model
3
divsum
Current, Siloed
Manual Approach
Vs
![Page 5: Best Practices in Model Risk Management - SAS · Best Practices in Model Risk Management. MEETING REGULATORY GUIDELINES GLOBALLY AND LOCALLY ... Bank Risk Conference presentation](https://reader031.vdocuments.mx/reader031/viewer/2022011815/5e61524ffc560375f27abe2a/html5/thumbnails/5.jpg)
SAS SUPPORTS ALL FOUR LINES OF MRM DEFENSE
Court
esy:
Chart
is R
esearc
h L
td 2
014.
![Page 6: Best Practices in Model Risk Management - SAS · Best Practices in Model Risk Management. MEETING REGULATORY GUIDELINES GLOBALLY AND LOCALLY ... Bank Risk Conference presentation](https://reader031.vdocuments.mx/reader031/viewer/2022011815/5e61524ffc560375f27abe2a/html5/thumbnails/6.jpg)
Related
models
Model Stakeholders: owner,
developer, validator, MRM
Oversight, Senior Mgt, Internal
Audit, Regulator
Related
Findings
Related Action
Plans
Dashboard
Related
UsagesRelated Policy
Exceptions
Related
Correspondence
Model
![Page 7: Best Practices in Model Risk Management - SAS · Best Practices in Model Risk Management. MEETING REGULATORY GUIDELINES GLOBALLY AND LOCALLY ... Bank Risk Conference presentation](https://reader031.vdocuments.mx/reader031/viewer/2022011815/5e61524ffc560375f27abe2a/html5/thumbnails/7.jpg)
PROCESS FRAMEWORK*
Governance
& Control
Operational
Model
Lifecycle
Independent
(periodic)
Audit
Process Design
Organisation Structure
IT Systems
* Source: Bank Risk Conference presentation in April 2015 by Konstantina Armata (Head of Global Model Validation & Governance at Deutsche Bank)
= Process area
![Page 8: Best Practices in Model Risk Management - SAS · Best Practices in Model Risk Management. MEETING REGULATORY GUIDELINES GLOBALLY AND LOCALLY ... Bank Risk Conference presentation](https://reader031.vdocuments.mx/reader031/viewer/2022011815/5e61524ffc560375f27abe2a/html5/thumbnails/8.jpg)
Process Design
Organisation Structure
IT Systems
ORGANIZATION*
Functional teams structured in 3 lines of defence
= Functional team
2nd Line:
1st Line:
3rd Line:
Model Owners &
Developers
Model Reviewers
& Validators
Model Risk
Management Internal
Audit
Risk Senior
Executives
* Source: Bank Risk Conference presentation in April 2015 by Konstantina Armata (Head of Global Model Validation & Governance at Deutsche Bank)
![Page 9: Best Practices in Model Risk Management - SAS · Best Practices in Model Risk Management. MEETING REGULATORY GUIDELINES GLOBALLY AND LOCALLY ... Bank Risk Conference presentation](https://reader031.vdocuments.mx/reader031/viewer/2022011815/5e61524ffc560375f27abe2a/html5/thumbnails/9.jpg)
Process Design
Organisation Structure
IT Systems
SYSTEMS DESIGN FOR MRM – SCREEN VIEWS= Process area
Systems
Processes
Governance &
Control Processes
Operational
processes
WORKFLOW SCREEN DESIGNER DATA LOADER
MODEL INVENTORY MODEL DETAILS MODEL REVIEWS
MODEL PORTFOLIO FINDINGS ANALYSIS
![Page 10: Best Practices in Model Risk Management - SAS · Best Practices in Model Risk Management. MEETING REGULATORY GUIDELINES GLOBALLY AND LOCALLY ... Bank Risk Conference presentation](https://reader031.vdocuments.mx/reader031/viewer/2022011815/5e61524ffc560375f27abe2a/html5/thumbnails/10.jpg)
MODERN MRM SYSTEM
KEY CAPABILTIES
![Page 11: Best Practices in Model Risk Management - SAS · Best Practices in Model Risk Management. MEETING REGULATORY GUIDELINES GLOBALLY AND LOCALLY ... Bank Risk Conference presentation](https://reader031.vdocuments.mx/reader031/viewer/2022011815/5e61524ffc560375f27abe2a/html5/thumbnails/11.jpg)
KEY FUNCTIONAL REQUIREMENTS
Model Candidate Assessment
Model InventoryModel Review &
ValidationModel Usage
Model Change Request
MRM Correspondence MRM Policy Mgt.
Model Retirement
Model Risk Assessment
Document Management MRM Dashboard
Issues & Action Plan
Model Performance Tracking
Workflow Management
Orchestration of working
documents among many
players in a workflow
Import metadata from any type of
model developed in any technology
Flexible mechanism to
assess whether a
candidate meets the
qualifying criteria for a
model
Manage
documents in a
secure fashion
Capability for
confirming and
validating the accuracy
of a model inventory
Fact based conversations with regulators to
facilitate effective defense of the model quality
Flexible workflow at
every level – matching
the bank’s methodology
Ability to partition model validation
responsibilities at any level: model
family, LOB, model use etc.
Ability to link to Bank’s MRM
Policies and Standards
![Page 12: Best Practices in Model Risk Management - SAS · Best Practices in Model Risk Management. MEETING REGULATORY GUIDELINES GLOBALLY AND LOCALLY ... Bank Risk Conference presentation](https://reader031.vdocuments.mx/reader031/viewer/2022011815/5e61524ffc560375f27abe2a/html5/thumbnails/12.jpg)
MODEL LIFECYCLE WORKFLOW
![Page 13: Best Practices in Model Risk Management - SAS · Best Practices in Model Risk Management. MEETING REGULATORY GUIDELINES GLOBALLY AND LOCALLY ... Bank Risk Conference presentation](https://reader031.vdocuments.mx/reader031/viewer/2022011815/5e61524ffc560375f27abe2a/html5/thumbnails/13.jpg)
EXAMPLE WORKFLOW
![Page 14: Best Practices in Model Risk Management - SAS · Best Practices in Model Risk Management. MEETING REGULATORY GUIDELINES GLOBALLY AND LOCALLY ... Bank Risk Conference presentation](https://reader031.vdocuments.mx/reader031/viewer/2022011815/5e61524ffc560375f27abe2a/html5/thumbnails/14.jpg)
AGGREGATE VIEW OF MODEL LIFECYCLE
DASHBOARDING
![Page 15: Best Practices in Model Risk Management - SAS · Best Practices in Model Risk Management. MEETING REGULATORY GUIDELINES GLOBALLY AND LOCALLY ... Bank Risk Conference presentation](https://reader031.vdocuments.mx/reader031/viewer/2022011815/5e61524ffc560375f27abe2a/html5/thumbnails/15.jpg)
ENTERPRISE DASHBOARD REPORTING
![Page 16: Best Practices in Model Risk Management - SAS · Best Practices in Model Risk Management. MEETING REGULATORY GUIDELINES GLOBALLY AND LOCALLY ... Bank Risk Conference presentation](https://reader031.vdocuments.mx/reader031/viewer/2022011815/5e61524ffc560375f27abe2a/html5/thumbnails/16.jpg)
EXAMPLE REPORT
![Page 17: Best Practices in Model Risk Management - SAS · Best Practices in Model Risk Management. MEETING REGULATORY GUIDELINES GLOBALLY AND LOCALLY ... Bank Risk Conference presentation](https://reader031.vdocuments.mx/reader031/viewer/2022011815/5e61524ffc560375f27abe2a/html5/thumbnails/17.jpg)
EXAMPLE REPORT: FINDINGS SUMMARY
![Page 18: Best Practices in Model Risk Management - SAS · Best Practices in Model Risk Management. MEETING REGULATORY GUIDELINES GLOBALLY AND LOCALLY ... Bank Risk Conference presentation](https://reader031.vdocuments.mx/reader031/viewer/2022011815/5e61524ffc560375f27abe2a/html5/thumbnails/18.jpg)
AGGREGATION OF MODEL RISK
![Page 19: Best Practices in Model Risk Management - SAS · Best Practices in Model Risk Management. MEETING REGULATORY GUIDELINES GLOBALLY AND LOCALLY ... Bank Risk Conference presentation](https://reader031.vdocuments.mx/reader031/viewer/2022011815/5e61524ffc560375f27abe2a/html5/thumbnails/19.jpg)
EXAMPLE MODEL RISK ASSESSMENT FRAMEWORK
Courtesy: Badrish Prakash, Discover Financial Services, American Bankers Association, Oct 2014
![Page 20: Best Practices in Model Risk Management - SAS · Best Practices in Model Risk Management. MEETING REGULATORY GUIDELINES GLOBALLY AND LOCALLY ... Bank Risk Conference presentation](https://reader031.vdocuments.mx/reader031/viewer/2022011815/5e61524ffc560375f27abe2a/html5/thumbnails/20.jpg)
IS YOUR MRM PROGRAM EFFECTIVE?
360 degree view of model lifecycle (design, test, independent validation, implementation, monitoring, retirement)
Any policy exception, escalations
Misapplication of models
Vendor models and third party oversight
![Page 21: Best Practices in Model Risk Management - SAS · Best Practices in Model Risk Management. MEETING REGULATORY GUIDELINES GLOBALLY AND LOCALLY ... Bank Risk Conference presentation](https://reader031.vdocuments.mx/reader031/viewer/2022011815/5e61524ffc560375f27abe2a/html5/thumbnails/21.jpg)
MODEL RISK ASSESSMENT QUESTIONS
![Page 22: Best Practices in Model Risk Management - SAS · Best Practices in Model Risk Management. MEETING REGULATORY GUIDELINES GLOBALLY AND LOCALLY ... Bank Risk Conference presentation](https://reader031.vdocuments.mx/reader031/viewer/2022011815/5e61524ffc560375f27abe2a/html5/thumbnails/22.jpg)
Individual Model's Risk Score
combined assessment of the Model's Materiality and Model's Overall Risk on individual model level
5-Very high
4-High
3-Medium
2-Low
1-Very Low
1-No problem 2-Low 3-Medium 4-High 5-Very high
> Model's Overall Risk Rating>
> M
od
el'
s M
ate
rial
ity
Rat
ing
>
MODEL RISK MATRIX EXAMPLE – CORPORATE PORTFOLIO
PD IRB model PD IFRS 9 model
PD Stress Testing
ALM prepayment
model
Pricing model
Marketing model
Model's Risk Score = 25
Model's Materiality Rating= 5
Model's Overall Risk Rating = 5
Model's Risk Score = 16
Model's Materiality Rating= 4
Model's Overall Risk Rating = 4
Model's Risk Score = 8
Model's Materiality Rating= 2
Model's Overall Risk Rating = 4
Model's Risk Score = 3
Model's Materiality Rating= 1
Model's Overall Risk Rating = 3
Model's Risk Score = 5
Model's Materiality Rating= 5
Model's Overall Risk Rating = 1
Model's Risk Score = 3
Model's Materiality Rating= 3
Model's Overall Risk Rating = 1
2015 Aggregated Model Risk Score = 60
2015 Average Model's Risk Score = 12
![Page 23: Best Practices in Model Risk Management - SAS · Best Practices in Model Risk Management. MEETING REGULATORY GUIDELINES GLOBALLY AND LOCALLY ... Bank Risk Conference presentation](https://reader031.vdocuments.mx/reader031/viewer/2022011815/5e61524ffc560375f27abe2a/html5/thumbnails/23.jpg)
THANK YOU