rmpi’s accru validation methodrmpiconsulting.com/wp-content/uploads/2014/11/rmpi... · throughout...

RMPI’s ACCRU Validation Method for ALLL & Stress Testing

3rd Annual Risk Management Summit Nashville, TN

September 26, 2014

“Risk Management for Community Banks”

© 2014 RMPI Consulting LLC - confidential

About RMPI Consulting

2

RMPI Consulting §  RMPI Consulting provides process improvement, enterprise

risk and credit management services to Community Banks throughout the credit life cycle.

§  Its thought leadership has defined industry “best practices” on a broad range of risk management issues.

§  We are Bank Consultants specializing in: Credit Services; Loan Review; Enterprise Risk Management; Process Improvement; Regulatory Order Management; Strategic Planning; Policies and Procedures; Loan Workouts; Expert Witness; and M&A

§  More on articles and information at www.rmpiconsulting.com


About the Presenter

David E. Brown is the founder of RMPI Consulting which he started in 2005. He has more than 25 years of commercial banking experience specializing in risk management including commercial loan review, workout, acquisition, due diligence, compliance, OREO and commercial underwriting. Prior to starting his own firm, David worked at CCR LLP as the Executive Director of its Risk Management Consulting practice. Citizens Bank as a Senior Vice President, Team Leader and Credit Officer

3


Discussion Topics

§  What Is Model Validation?

§  Is Validation Necessary?

§  What Is the Purpose of Validation?

§  Examiners Review Validation …

§  … But Examiners Do Not Validate!

§  Which Models Need Validation?

§  What Is “ACCRU?”

§  Overview of RMPI’s ACCRU Validation Method

§  Reverse Stress Testing

§  Continued Validation

4


What Is Model Validation?

Model validation is the set of processes and activities intended to:

§  Verify that models are performing as expected, in line with their design objectives and business uses.

§  Help ensure that models are sound.

§  Identify potential limitations and assumptions, and assesses their possible impact.

§  Be performed by staff with appropriate incentives, competence, and influence.

5


Is Validation Necessary?

The Interagency Policy Statement on Allowance for Loan and Lease Losses Methodologies and Documentation for Banks and Savings Institutions requires institutions to verify that the ALLL modeling is valid and conforms to GAAP and supervisory guidance.

•  It also requires that the institution's directors establish internal control policies that include procedures for a review, by a party who is independent of the ALLL estimation process, of the ALLL methodology and its application in order to confirm its effectiveness.

6


What Is the Purpose of Validation?Validation is an important check used during periods of benign economic and financial conditions, when estimates of risk and potential loss can become overly optimistic, and when the data at hand may not reflect impact to capital under more stressed conditions.

An effective validation framework includes these main elements:

§  Evaluation of conceptual soundness

§  Ongoing monitoring

§  Process verification and benchmarking

§  Outcomes analysis, including back-testing

7


Examiners Review Validation …

When reviewing validation, examiners:

§  Evaluate the scope of validation work performed

§  Review the report summarizing validation findings and any additional work papers needed to understand findings

§  Evaluate management's response to the report summarizing the findings, including remediation plans and time frame

§  Assess the qualifications of staff or vendors performing the validation

8


… But Examiners Do Not Validate!Examiners only test the effectiveness of the institution's validation function by selectively reviewing various aspects of validation work performed on individual models.

Validation remains responsibility of the institution. It is also required directors establish internal control policies for a review by a qualified party that is independent of the ALLL estimation process.

Therefore a third party, independent of the estimation process, should conduct the validation not only for compliance but safety and soundness practices as well.

9


Which Models Need Validation?

Given the latest pronouncements on Basel III, capital adequacy will continue to dominate the examiners’ feedback (and criticism) for the foreseeable future.

§  Heightened regulatory scrutiny over the Allowance for Loan and Lease Losses (ALLL) in the past several years, examiners, auditors, and senior management are all looking for additional ways to measure the effectiveness of an institution’s ALLL methodology.

§  Regulators increasingly expect institutions of all sizes to perform some type of Stress Testing and use those results to gauge current capital levels and plan ahead for adverse economic environments.

10


What Is “ACCRU?”

RMPI Consulting uses its ACCRU methodology of validation on both ALLL and Stress Testing validation engagements.

The root of ACCRU is as follows:

§ Accurate

§ Compliant

§ Controlled

§ Reasonable

§ Useful

11


ACCRU - Accurate

Accuracy is the nuts and bolts of validation. Conduct quality assurance reviews of the process itself.

§  Whether completed in-house or by a vendor, incorrect data, math errors or even corrupt spreadsheet cells can create very big mistakes.

§  Logic errors can occur in the data gathering and analysis process itself, so that too needs testing for accuracy.

Example: 1) corrupted cells; 2) old valuations

12


ACCRU - Compliant

In this regulatory dynamic time, it is critical to keep up with the most current regulator “Hot Buttons.”

§  What was good last year may not pass this year.

§  Even if the regulator gives an institution the required changes to be compliant, that does not necessarily make the institution compliant.

Example: 1) 4 new systems in 4 consecutive exams

13


ACCRU - Controlled

Does the Board of Directors and Executive Management consistently review the data and are there active early preventive measures to mitigate the risk?

§  Validation of timeliness and quality of reporting is crucial.

§  Also important is whether the Board or Executive Management has feedback mechanisms to change behavior or processes when the data indicates there is an issue.

Example: 1) board looked at stress the one time; 2) qualitative factor changed with no backup.

14


ACCRU - Reasonable

An institution may have accurate ALLL or Stress Test plans, but are they reasonable? Testing the underlying assumptions for reasonableness is critical.

§  The level of movement for given conditions by the chosen stress factors need to be justifiable.

§  The movement of qualitative factors in an institution’s ALLL and Stress Test need to have written quantitative factors to support the recommended changes (or even the lack of change).

Example: 1) 1-year, 3-year, 5-year histories; 2) 5% and 10% moderate and worse cases.

15


ACCRU - Useful

ALLL and Stress Testing need to be permanently integrated into the strategic planning of the institution.

§  Sporadic Stress Testing has little long-term value.

§  Stress testing should be included within all pro forma financial projections when forecasting desired product growth, especially for either riskier or new products.

Example: 1) not part of strategic planning or acquisition planning; 2) don’t take stress to failure.

16


Reverse Stress Testing

Reverse Stress Testing should be used when forward Stress Testing indicates loss in a loan category under stressed conditions.

§  The losses are traced back up the stress trail to determine the specific root causes of the loss.

§  Through early detection, future behavior under actual stressed conditions might be better controlled to minimize potential losses due to the identified risks.

17


Continued Validation

Validation activities should continue on an ongoing basis after a model goes into use, to track known model limitations and to identify any new ones.

§  Banks should conduct a periodic review (at least annually) of each model to determine whether it is working as intended and if the existing validation activities are sufficient, including updating documentation of all activities.

§  Ongoing validation activities help to ensure that changes in markets, products, exposures, activities, clients, or business practices do not create new model limitations.

18


Thank You

60 State Street Suite 700 Boston, MA 02109 (617) 973-5742 - Telephone (978) 844-4192 - Cell Phone (617) 963-7119 - Fax [email protected] www.rmpiconsulting.com

3001 N. Rocky Point Dr. East, Suite 200 Tampa, FL 33607 (813) 367-3510 - Telephone (978) 844-4192 - Cell Phone (813) 982-3310 – Fax [email protected] www.rmpiconsulting.com

David Brown, PartnerRMPI Consulting LLC

rmpi’s accru validation methodrmpiconsulting.com/wp-content/uploads/2014/11/rmpi... · throughout...

Documents