internal risk ratings effective - rma u presents a detailed case study of the design stagein...

T he first part of this article(September 2001) des-cribed how an effective

credit risk system should bedesigned and what issues shouldbe resolved for successful imple-mentation. This concluding sec-tion presents a detailed casestudy of the design stage in imple-menting such a system atSunTrust Banks, Inc.

In September 2000, SunTrustbegan a thorough and methodicalredesign of its entire credit risk-rating system. Despite its strongreputation as one of the industryleaders in credit quality, the bankdecided that the substantial costsassociated with initiating an evenmore effective credit risk systemwere justified. The improvementswould facilitate the implementa-

tion of Basel II in 2005 and wouldstrengthen the bank’s position atthe forefront of the lending indus-try. Oliver, Wyman & Company(OWC) was asked to assistSunTrust in accelerating theimplementation of the new systemwith minimal disruption to thebank’s daily business and ensurethat, once in place, the systemwould be consistent with industrybest practices. Though projects ofthis type can vary in scope andtiming, SunTrust’s experiencesshould be helpful to other institu-tions involved in similar projects.

Priorities and SpecificationsSunTrust’s first step was to

identify design specifications thatwould become the risk-rating sys-tem’s foundation. Because the

redesign of a bank-wide creditrating system is always a majorundertaking, demanding manyinternal resources, consensusamong the key players was a pre-requisite. After discussionsthroughout the bank that includ-ed line managers, credit officers,and senior management, fourstrategic imperatives were identi-fied for the new ratings system tosupport:

1. Loan approval. The cor-rect assessment of the level ofcredit risk is a key component toa successful loan approval system.

2. Loan pricing. A proper dif-ferentiation of credit risk is nec-essary for a competitive loan pric-ing model, both to fully exploitpotential opportunities and to

44 The RMA Journal December 2001 - January 2002

Effective Credit Risk-Rating Systems

by Jim Stoker, Tom Garside, and Tom Yu

© 2001 by RMA. Yu is a senior manager specializing in North American work at Oliver, Wyman & Company (OWC) in NewYork; Garside is director and head of the firm’s Risk Management Practice in London; and Stoker is a senior manager special-izing in risk, based in New York. The authors acknowledge the assistance of George Morris, Jim Wiener, and John Stroughair,directors, and John Stewart, senior manager, all at OWC. Oliver, Wyman & Company is a strategy consulting firm dedicatedexclusively to the financial services industry. Contact Stoker at [email protected]; visit OWC’s Web site at www.owc.com

T his conclusion of a two-part article presents a case study of

a regional bank’s initiative to upgrade its credit risk

management process.

INTERNAL RISK RATINGS

avoid adverse selection problemsin client relationships.

3. Risk-Adjusted Return onCapital (RAROC). Accurate capi-tal management and performancemeasurement require consistentrisk measures across all lines ofbusiness (LoB).

4. Portfolio monitoring andcontrol. Effective monitoring andcontrol rely on timely, accuratemeasurement of credit risk.

The challenge was to supportthese critical applications whilemoving the bank closer to com-pliance with anticipated changesin regulatory guidelines. Based onthe priorities of SunTrust man-agement and expectations regard-ing the Basel II regulatory guide-lines, OWC identified five keyimprovements to the current sys-tem:

1. Two-dimensional credit rat-ings—independent obligorand facility ratings to increaseaccuracy and be consistentwith Basel II requirements.

2. Additional granularity in the“pass” obligor ratings—a nec-essary component of certainsystems, such as a RAROCmodel currently in develop-

ment, that use credit risk rat-ings as an input.

3. Explicit separation of theassignment of risk ratings andthe use of the given ratings.In many credit risk-rating sys-tems, such management deci-sions as “reduce risk” caninfluence credit risk ratingsby leading people to assignharsh ratings. The new sys-tem should assign gradesbased solely on the risk of thecredits.

4. Bankwide consistency in rat-ings, both within and acrossLoBs. In the old system,each LoB had its own ratingsscale, and it was difficult tocompare these ratings.

5. Increased resources and capa-bilities for data capture andanalysis. The proposed re-quirements of Basel II makethis particularly important.

A Master Credit Risk Scale forAll Credit Exposures

Risk-rating systems shouldmeasure the credit risk consis-tently throughout the entireorganization. Within LoBs, guide-lines should be set to minimizevariation across raters. Risk raters

should assess objective factorsregarding an obligor or facilitysimilarly, and discrepancies in rat-ings for the same credit shouldoccur only when raters differ inassessing the qualitative, or judg-mental, factors. A line officeroften assigns better ratings thando credit officers; this may beconsidered acceptable because ofthe conservative “watchdog”nature of Credit versus theaggressive “sales-oriented” natureof the line. A good risk-rating sys-tem, however, should lead to oneanswer, regardless of who is doingthe rating; deviations from thiswill ultimately lead to problemswithin the organization.

Consistency of ratings acrossthe LoBs (or within the same LoBbut across geographies) should alsobe a goal. A robust risk culture dic-tates that all bank staff understandcredit risk as a common language.Currently, many institutions’ ratingsystems fail to support this. It isnot uncommon for ratings acrossLoBs or geographies to be on dif-fering scales, for example, “Our 3rating is more like the 5 rating inthat group.” This approach dra-matically complicates effective riskand capital management. Instead,SunTrust chose to define a com-mon “master” rating scale to beused across the entire bank.

Move to a two-dimensionalsystem. A two-dimensional creditrating system calculates ExpectedLoss as the product of obligor andfacility risk characteristics. Theobligor risk is described by theprobability that it will default(PD) and the facility risk by theamount of loss given default(LGD). These two elements are

45

E f f e c t i v e C r e d i t R i s kR a t i n g S y s t e m s

Design• Set objectives• Establish master

scale• Change rating

assignment• Align policies

SWITCHOVER REFINEMENT

Figure 1

THREE STAGES OF IMPLEMENTATION

separately recorded and used joint-ly to compute an expected loss(EL) for a credit exposure. A one-dimensional credit rating systemuses a single rating scale that isbased on EL. The single rating isusually assignedtaking into con-sideration char-acteristics of theobligor, facility,and collateral,but the factorsare never explic-itly separated.

Over andabove the needto comply withBasel II, thebusiness bene-fits of using atwo-dimensionalsystem far out-weigh its addi-

tional complexity. First, a two-dimensional system improves con-sistency among raters. Factors thatdrive the PD and the LGD can beevaluated more objectively andtested for relevance. Without sep-

arate rating scales forPD and LGD, theimportance of these twocomponents will differfrom rater to rater.Second, the improvedunderstanding of whatdrives the EL helpsfeed tactical applica-tions. It can provide theline with guidance onhow to change the facil-ity details to get thedeal done—for exam-ple, “If I require morecollateral, the LGD willbe reduced, driving ELdown, and I can pricethe deal where Iwant….” Third, mod-ern credit portfoliomodel and economiccapital applicationsrequire PD and LGD tobe quantified inde-

pendently.

Increase and optimize thegranularity of the ratings scales.There is an optimal number ofpass ratings. If there are too many



Figure 2

EXPECTEDLOSS(EL)

A TWO-DIMENSIONAL SYSTEM ALLOWS MORE ACCURACY IN

COMPUTING EXPECTED LOSSES

LOANEQUIVALENTEXPOSURE

LOSS IN THE EVENTOF DEFAULT (LIED)

Loss in Event ofDefault as %

Facility OutstandingRating Principal

A 5%. .. .. .E 50%. .. .. .. .J 90%

EXPECTED DEFAULTFREQUENCY (EDF)

Obligor Expected AnnualRating Default Rate (%)

1 .05%, ,, ,, ,

5 1.50%. .. .. .. .

10 100%

The two-dimensional ratinghelps answer thequestion: What isthe average lossfor this credit?

Obligor rating scaleanswers the

question: Whatis the likelihood

of default?

Facility ratings scaleanswers the

question: How muchof what I am owedwill I lose if there

is a default?

= x x

Figure 3

DISTRIBUTION OF RISK RATINGS

40%

35%

30%

25%

20%

15%

10%

5%

0%

40%

35%

30%

25%

20%

15%

10%

5%

0%1 2 3 4 5 6 7 8 9 10 1112 13 14 15 16 17

Risk Rating1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Risk Rating

Non-Optimal Distribution Optimal Distribution

pass grades, assigning an obligorrating becomes too time consum-ing or subjective and cumber-some. If there are too few passgrades, it’s not possible to differ-entiate risk between the obligorssatisfactorily. As a rule of thumb,there should be enough risk rat-ings to reflect instances in whichthere is a true difference indefault risk. In systems that con-tain a large portion of credits (over30%) in one rating, there is likelyto be either insufficient granulari-ty or an inappropriate segmenta-tion of the obligor scale.

Optimizing the granularity inthe obligor scale needs to be donewith the risk profile of the bank’sportfolio in mind. If a bank has alarge subprime mortgage portfolio,it will need more granularity inthe lower range of the PD scale;otherwise, it will not be able todistinguish the default risks foreach segment within the portfolio.

A bank withprimarilylarge corpo-rate creditswill not needa high levelof granularityin the upperrange of thePD scale.Thus, whiletwo banks

might have the same number ofrisk ratings, they may have vastlydifferent “bucketing” of theobligor scale.

Increase resources andcapabilities for data capture andanalysis. A risk-rating system canbe undermined by insufficientdata capture and analysis. Analysisis necessary to design and para-meterize the models and, of equalimportance, to validate their out-put. A successful ratings systemrequires that all users—line, cred-it, and external parties—trust itsoutput. As all approaches toassigning ratings are prone toerror, it is necessary to continuallycheck and update the models. Forsuch validation, data capture is ofcentral importance. In the bestcase, the institution maps andretains all the flows of data usedin the ratings processes, includingratings migrations. For example, acommercial LoB should retain allthe spreading data used duringthe initial ratings assignment andthen record the subsequentlyassigned ratings as the credit is re-rated over its life. At a minimum,all ratings systems require data torecalibrate the systems to the PDsand LGDs experienced for eachasset class. Establishing a modelmanagement unit is one approach

to providing such resources.

Changes to How Ratings AreAssigned in the New System

A risk rating should be a purereflection of the risk associatedwith a particular credit. The ratingshould be applied without regardto the decisions that will subse-quently be taken based on thisrating. Two errors are common inthis process. The first is theassignment of whatever grade isrequired to “get the deal done,”that is, to get approval. This canlead to hidden risk and under-priced loans. The other error is amove by the institution toward“conservatism,” by grading newdeals too harshly. This practicewill result in less risk in the port-folio, but lower profitability aswell. Though SunTrust showedno consistent bias in either direc-tion, it was agreed that an explicitgoal of the system redesign wouldbe to ensure that no such prob-lems could occur in the future.

Model development. Giventhe variety of different businesslines at SunTrust, it was impossi-ble to create one standard modelto evaluate credit risk. It was nec-essary to take two broadapproaches, differentiatingbetween commercial/corporatecredits and consumer/residentialmortgage credits.

1. Commercial and corpo-rate. The commercial and corpo-rate LoBs benefited by the exis-tence of broadly successful exter-nal models (Moody’s RiskCalcTM

and KMV’s CreditMonitorTM) usedto measure the probability ofdefault. Though the ultimate goalof the project was to develop

47


Master Obligor Scale Master Facility ScalePD Range (Basis Points) LGD Range (%)

Risk RiskGrade Min Max Grade Min Max

1 0 5 A 0 102 5 7 B 10 303 7 9 C 30 504 9 15 D 50 705 15 27 E 70 906 27 48 F 90 1007 48 768 76 1099 109 151

10 151 21211 212 27212 272 33313 333 42314 423 54415 544 70016 700 80017 800 90018 900 1,00019 1,000 10,000

Figure 4

models tailored to the SunTrustexperience, current data limita-tions made this difficult. As aresult, it was decided that externalmodels would be used, if possible,as a point of departure. Theprocess of testing and ultimatelyimproving these models was donein a two-stage exercise.

First stage. The first stagewas to select a sample of creditsfrom the relevant SunTrust port-folio (including credits that haddefaulted) and run them throughthe external models to get theassociated PDs. Concurrently, arepresentative group of internalcredit experts was asked to ratethese same credit packages usingthe old SunTrust rating scale,ignoring all facility-related factorsand listing what they consideredthe main drivers for the rating thatthey assigned. The purpose of theexercise was to: • Get a sense of whether the

external models were doing areasonable job of getting theordinal risk ranking of thepackages correct.

• Try to identify a few obligorswhere the model and creditexperts differed in their riskassessments. These would bethe packages used in the sec-ond stage of the credit experi-ment. The first stage of the experi-

ment indicated that the vendedmodels performed adequately as astarting point for assigning a rat-ing; however, the vended modelwas not usable in any of the port-folios for the assignment of aninternal rating without some formof adjustment. In all cases, therewere obligors where the ratingassigned by the model and the

credit experts diverged; these dif-ferences could have been dueeither to an omission of a relevantfactor by a lending expert or tothe weighting assigned to a factorused by the model. These oblig-ors formed the basis of thesecond-stage experiment.

Second stage. In the secondstage, the selected obligors wereredistributed to a broader group ofcredit experts. Included with thepackages was the list of factorsidentified in the first stage of theexperiment that were consideredpossible reasons for the differ-ences. The credit experts thenregraded the packages under dif-ferent assumptions regardingthese factors. For example, had apackage been downgraded due toproblems related to the obligor’sindustry, the experts were askedto regrade the package under theassumption that the obligor was ina different, more standard indus-try. By compiling the responses, itwas possible to estimate both thefactor’s potential significance andthe consistency, or lack thereof,between the experts when judg-ing the various factors. Severalfactors were subsequently exclud-ed, because of either low averageestimated importance or dramaticvariance in the experts’ measuresof the factors’ importance. Theremaining factors formed the basisfor developing templates for judg-mental overrides. These struc-tured overrides allow a loan officerto upgrade or downgrade a givencredit by a limited amount for avariety of well-defined criteria,such as management quality andindustry type, ensuring that theexperience and expertise of thecredit raters were not lost. The

last step, currently under way atSunTrust, is the writing of policygoverning ratings assignment,with particular attention to therules and procedures governingjudgmental override authority.

Due to the lack of reliableexternal data, the templates usedto determine LGD had to bedeveloped wholly in-house.Template development involvedin-depth discussions withSunTrust’s staff across all lendingoperations, including line, credit,and recovery. The systems arenow being put in place that willcapture the data needed to vali-date and fine-tune the templates.

2. Consumer and residen-tial mortgage. For the obligormodels of the consumer LoB, theultimate goal was to create score-cards that can use relevant cus-tomer data (for example, FICOscores, payment history, season-ing, and vintage) to predictdefaults. In fact, throughoutSunTrust, a variety of differentcustomized scorecards have andare being used. In the interest ofstandardizing the credit-ratingprocess, however, it was decidedto create an initial scorecard thatwas simple enough to be usedthroughout SunTrust’s retail oper-ations. For the first time, thisscorecard, in effect, standardizedthe credit process enterprise-wide. Such scorecards rely heavilyon bureau scores and collateraltypes and are being implementedso that they can be readilyupgraded as new data becomesavailable. The initial estimates forthese scorecards were developedusing a combination of internalSunTrust data and industry



benchmarks.A similar process was used to

develop the LGD estimates.Restricted data availability madeit impossible to rely solely oninternal collections data. Conse-quently, for each of the consumerproducts, internal data was thebasis for one part of the process;as necessary, the data was supple-mented by the judgment of inter-nal credit experts and industrybenchmarks.

Policy AlignmentTo enable the new ratings to

be used effectively and to providea strong governance framework,SunTrust must align its risk man-agement policies and practices.This requires several areas ofchange. First, it will be necessaryto write a set of procedures toguarantee effective use of the newcredit risk ratings. Second, it willbe necessary to ensure that themany people who participate inthe risk ratings process, includingline, credit, and higher levels ofmanagement, are fully aware ofthe ratings process and confidentin its use. A final area of concernis in IT systems. To ensure com-patibility, SunTrust simultaneous-ly upgraded its IT systems withits credit process.

Policies and procedures. Toget maximum value out of animproved credit risk-rating system,policies and procedures needed tobe put in place to ensure that theratings system is applied correctlyand that the results are used appro-priately. Performing this task suc-cessfully was central to theSunTrust initiative. By havingSunTrust personnel in key roles,

the process of understanding boththe capabilities and limitations ofthe new system, as well as how thesystem can be best applied, wassignificantly accelerated.

Two primary types of policiesand procedures were considered:

1. Detail-oriented procedures.The “SunTrust experience”was absolutely vital for estab-lishing these procedures. Forexample, a common questionin this type of redesign washow to treat loan guarantors.There is no broadly acceptedapproach to take. In thisinstance, the SunTrust “teamleaders” took on the chore ofsimultaneously developingthe strategy that was to beused for guarantors, oftenthrough extensive polling anddiscussion groups with theother team members.

2. Best use of the models. Thisis, of course, a much broaderissue, and one that cannot belimited to the time used todevelop the models. SunTrustput in place a monthly workinggroup that both oversaw thedevelopment of the modelsand was responsible for theirappropriate use. This group isexpected to continue meetingfor the foreseeable future.

Organizational alignment.A critical issue in successfullyredesigning a bank-wide risk rat-ings system is to ensure institu-tional acceptance. The impor-tance of buy-in cannot be empha-sized enough—the best system inthe world is worthless if it is nottrusted. To ensure that all usersof the ratings system would be

comfortable with it, OWCemphasized early and frequentcommunication.

People from line, credit, andsenior management were consult-ed in the development stage. Therisk-ratings initiative involvedmore than a hundred people, onlysix of whom were not SunTruststaff. The key participants fromSunTrust were the project spon-sors, the chief credit policy officer,the senior VP and managingdirector, the corporate and com-mercial credit policy manager,who served as project owner, andthe credit risk manager, whoserved as project manager. Tominimize coordination problemsfrom the range of participants, keystaff from each of the bank’s mainoperating regions were includedin the decision-making teamswhenever possible. Although thisdecision slowed down the initia-tive’s pace, it was considered areasonable investment. Theincreased time spent at the frontend (development) significantlydecreased time needed at theback end (rollout and implemen-tation). In addition, a great deal ofeffort went into ensuring that theopinions of people who would beaffected by the models wereheard and their opinions built intothe system. Given the data con-straints, this was doubly impor-tant. Beyond facilitating buy-in, itwas also OWC’s main window intothe SunTrust experience andmade tailoring the models toSunTrust much easier.

Installation of model man-agement. A process was requiredto test credit risk models for accu-racy and improve them over time.

49


Two options were identified:1. Allow all lines of business to

verify and adapt their modelsas data became available.

2. Standardize this process inone Model Management unit,whose members would beresponsible for validating, cali-brating, and broadly improv-ing all of the models devel-oped for credit rating. By electing to establish a cen-

tralized Model Management unit,SunTrust also ensured that thebank was kept up-to-date on allnew approaches to credit riskmeasurement. Consistencythroughout the various lines fur-ther ensured accuracy and mini-mized the chance that specificLoBs would lose faith in theprocess. In addition, the central-ized Model Management unitwould make it easier to keep cur-rent with the rapidly changingworld of credit risk management.

IT alignment. Concurrentwith improving its credit risk-rat-ings system, SunTrust is involvedin updating its IT systems. Thisadded to the complexity of theproject, particularly with respectto data analysis and capture. Tomeet management objectives, ITsystems were tailored to work asseamlessly as possible with therating models.

Data limitations. At the heart

of any credit risk-ratings systemdesigned to comply with futureBasel II requirements is a stronghistory of data collection. Rapidorganizational change, however,oftens means that data either is notavailable or is considered flawed. Itwas necessary to identify what datawas to be tracked without theadvantage of basing the decisionon extensive data analysis, and thisdata had to be collected immedi-ately, even if not currently usable.

As discussed above, thechoice was made to use externalmodels in the commercial and cor-porate LoBs. The IT update wasput in place with this in mind.The models directed the choice ofdata to be collected, and the sub-jective override templates wouldalso be tracked. In addition, theLoBs were directed to create listsof additional data that they feltmight be included in futureupgrades. For the consumer andmortgage LoBs, “target” modelswere developed to direct data cap-ture and will be tested when suffi-cient data is available.

Of critical importance wasmaking sure that data collectionbegan as soon as possible, to allowcustomization to SunTrust’s expe-rience as quickly as possible. Thiseffort required compatibility withthe bank’s concurrent initiative inupdating its IT systems. SunTrustwanted to ensure that the systems

ultimately put in place wouldremain sufficient for future needs.By interviewing individuals in thevarious LoBs, expectations wereset regarding model evolution,model updates, and the requisitesupporting data needed to meetthose expectations.

ConclusionThe new credit risk-grading

system at SunTrust is very mucha work in progress. For each ofthe main LoBs, models are inplace that will rate credits basedon the SunTrust master obligorand facility scales. However, thekey to these models is not thatthey be perfect (an impossiblegoal) but that they be sufficientlyrobust to add value to credit riskmanagement decisions and bedesigned to readily incorporatethe results of enhancementsderived from future data capture.It is highly unlikely that the mod-els will not materially changeover the next two to four years.This is likely to be true in mostLoBs, and a measure of success ofthe initiative will be how easilythis change takes place. Throughits initiative, SunTrust is in anexcellent position to enhance itsposition in commercial and corpo-rate lending and to continueadopting “best practice” advancesas they occur. ❐



Keep’em Covered!Does your Journal circulate? Keep it fresh with a made-to-fit cover.

Store a full year of the Journal in a binder that stores easily on a bookshelf.

Call 1-800-677-7621 to order a single-month cover (#616215) or full-year binder (#616209B).

internal risk ratings effective - rma u presents a detailed case study of the design stagein...

Documents