\G-7

A publication of the World Bank's

Quality Assurance Group

ARPP REPoRTsPORTFOLIO PERFORMANCE

ON THE UPSWINGp ~~rtfolio health improved during Portfolio Priorities in FY98the past few years, reports the AnnualReview of Portfolio Performance for CountriesFY97. The proportion of projects at Algeria Nigeriarisk of not meeting their development Angola Pakistan

objectives fell from 40 to 30% between FY94 Burundi PNGand FY97. This reduced riskiness reflects more Cameroon Russiaproactive portfolio management, greater selec- CAR Rwandativity in lending decisions, and improved eco- Caribbean CMU Sierra Leonenomic management in a number of borrowing Guinad Tanzni

- - ~~~countries. Girean TurkePortfolio riskiness continued to decline in Keany Tuganda

FY98. March data show the proactivity index, Madagascar Ukrainewhich reflects timely action by managers to re- Morocco Venezuela

~~~ ~~structure or cancel problem projects, has Niger Yemenclimbed from 67 to 72% over the last 6 months,although it still has not reached the FY98 target Agecutors xeso Hdooeof 80%. The proportion of projects in problem ArclueEtninHdooeOil and Gas Urban Water Supplystatus for more than 24 months and projects Urban Managementwithout a supervision mission in the last 1 2months also declined sharply. Instruments

Still, the news is not entirely positive. The Technical Assistancerealism index, which measures the candor and Financial Intermediary Lendingaccuracy of Bank reporting on portfolio prob- Large Projectslems, registered a worrisome decline over the 39 Operations Each With Net Commitments ofpast year, reversing an earlier positive trend. $200M or moreReinforcing this concern, only about 12%l ofprojects scheduled to be closed in FY98 are________________________

rated unsatisfactory in meeting developmentobjectives. This compares with OED findings Inside...that over one quarter of projects in recent yearshave had unsatisfactory outcomes. Either the Staff honored for excellence in supervision 2Bank is achieving a major improvement in port- Quagging the QAG- Resuks of the 3folio outcomes or, more likely, the problem of QAG interntal audit"disconnect" (between supervision and OED Findings from the Rapd Supervision Assessment 3ratings) is worsening. Rapid Stupervtslon Asssmemnt-A Primer 4

(continiued on page 2)

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

(Continued from page 1)Other problems: over one-third of projects are

receiving less than satisfactory supervision, and about Projects at Risk20% of new projects dont fully meet our quality at Actual problem projeccts and potential problem projects.

2ucmsfr7-0% of nwprojects do'tfllp eetor quaiable.2ity8Sa alt

entry standards. Project monitoring and evaluation, 50essential to quality assurance, are especially weak. ,

Notwithstanding the above problems, the Portfo-lio Improvement Program is working. No major initia- 30

tives are needed, perseverance is. Over the medium 2 term, with continued sustained attention, satisfactory0outcomes for 75-80% of projects appear attainable. M~ Beyond that a steady state of 80-85% is a reasonable 1 14~~

goal. Further improvements beyond that level may re- fgquire even greater selectivity in terms of who we lend 0to, what we lend for and how we lend. AFR EAP ECA LCR MNA SAR Bank

Realism IndexPercentage of projects at risk that are rated problem projects.

100

40-

§}tq AFR EAP ECA LCR MNA SAR Ban k

i.<. W2 ~~~~~~~~~~~~~~~~Proactivity Index,tie SS1Wt: ,g, Sgl, H>Sa e arPerc of p rhich rated as po,roblem projects twelve months

I 0 h a * 30

pt~~~~~~~~~~~~ssed ~ ~ ~ ~ ~ ~ ~ ~~~~~~a *6~ (. 2 Xg @ggSga

4 0 *-gg@G,_@ 20- > a0

Vation ~ ~ ~ ~ ~ ~ ~ ~ ~~~~O SS SrTe~p~dh lti1pvtirUi(@ ----i E0-Sgft.BBa,

STAFF HONORED FOR AFR EAP ECA LCR MNA SAR Bank

EXCELLENCE IN PROJECT SUPERVISION a *Mar-97|Twenty-five project teams wo recognition last Mar-98

January for excellence in supervision of projects.These projects were selected from a total of 250projects reviewed by the recent Rapid Supervision As- QAG once again congratulates them for a job wellsessment. Those honored were identified by their done. Across the Bank, there are many more examplesmanagers as having made a major contribution to of excellence-QAG hopes to highlight them in futureproject success. reviews,

Caio Koch-Weser and Sven Sandstrom hosted areception on January 24 to honor the staff selected. See insert for the list of honorees.

~~~~~~~~~~~~~~~~~~~~~*5 _**

L24 - ; - w n

STAFF HONORED FOR EXCELLENCE IN SUPERVISIONEXCELLENT PERFORMANCE IN SUPERVISION

QUALITYASSURANCE GROUP

-"'' :. V'..*.-

Albania G04

Albania r$t r T

Argentina fini ion n J t ' ..;

Bolivia 'i1; ' t;t'Jv't:..>...: . .

Boliva - .~id~e .d S0ttmPc . Osa ;faao - -i;. 4-

China Mtrkad hlfeat rjv ads p4y TM

-t! .~ ~ ~ ~ ~~I Z"' .i . j . i;: .; ;,

China t kPcfrTA .... ....

C&te d'lvo ireSA ~ tt h 'JaCwdJ3ie,M

Honduras Pubik a~~dz* ~~AJ~mn mri"Mti c .i~N

Hungary HclhSrksli tWe c

- . . .NkJan~~~~~~~~~~~~~~~~~~~~11,4- M . i

India Uttar1rdh nali_ Ulmn '0jc N l4ailuT

-j Nj i l( ij y iadbyi E~4deepSA4 W.

India 's~~~~~~~~~~~~~ SSa Rod

-~~~~~~~~~ S -~

Reform of Secondary Education Project Kin Bing Wit, TMRobin HughesAbigail HairrisRosalia Sanchez-Rushtoii

Environmental Techliology Development Project Robert McGough, TMDaniel PincockCarol Hau-Lai BallSing-Zak Sung

Highway Improvement Project Denis Robitaille, TMEva NMonteblancoGaye LindseyMostafa El-Erian

Financial Sector and Private Enterprise Development Project Simuon Gray, TMFrancoise SchattenHerminia Martinez

Technical Assistance to Enhance Competitiveness Project Simon Gray, TMFrancoise SchattenRobert Schware

Domestic Energy Resources Development Project Rashid Aziz, TMMikael MengeshaJulia FraserHannachi Morsli

Small Coconut Farms Development Project Syed Husain, TMt

j _ * * Power Transmission Project Rachid Benmessaoud, TMKarl Enar WennerstromAngelica Fernandes

_ 2 * Emergency Oil Spill Recovery and Mitigation Project Peter Pease, TM_Vadim Voronin

Telecommunications Project Alberto Cruzat, TMSvetoslav TintchevBarbara SantosHarbaksh Sethi

Northwest Mountainous Areas Development Project Laurent Msellati, TMIdah Pswarayi-RiddihoughAline Cabal

* WVater Supply and Sewerage Project Mohammed Benouahi, TMDominiqtue Dietrit hClaudia Pardinas O anaJohan Van Beuzekom

First Urban Project Gautam Sengupta, TMISabine CorneliusSudeshna RoychoudhuryVictoria Fofanah

AI-Mukalla Water Supply Project Uruj Kirmani, TMAli HusniJosephine MasanqueNaji Abu-Hatim

"QUAGGING" THE QAG: OFF TO A GOOD START, BUT SCOPE FORFURTHER IMPROVEMENT

From a recent Internal Audit Department report: We found, however, that QAG's impact and im-

"QAG has achieved impressive results in a short age could be enhanced by:period of time. We found that QAG has completed .Allowing for greater task manager participation inthe development of a significant number of the tools

and products it needs to fulfill its mandate. QAG hasbeen successful in identifying and selecting qualified V Identifying accountability more proactively;panels to conduct the two types of assessmentsreviewed under this audit (Quality at Entry and V Developing and formalizing its conflict of interestQuality of Supervision). QAG has assessed the quality policy; andof an impressive number of projects in a relativelyshort period of time. We noted that QAG achievedthese results in a difficult environment where most ofits activities were new to the Bank and required a QAG agrees with these recommendations and has begunsignificant degree of experimentation. Finally, wenoted that QAG's work has been well received 'ymost Regional Managers and Senior Advisers.

Rapid Supervision AssessmentFINDINGS FROM THE RAPID Results by RegionSUPERVISION ASSESSMENT % Satisfactory or Better

I100Bank supervision was deficient in more than one-thirdof projects reviewed, according to the Rapid Supervision 80Assessment for FY97. The survey looked at a randomsample of projects representing about 20% of the port- 60 -. ! __

folio. Performance varied across regions and sectors.40t

Other findings: 20 jV The quality of task managers and teams was a key 0

determinant of supervision quality. AFR EAP ECA LCR MNA SAR Total

V Poor supervision typically reflected management Rapid Supervision Assessmentinattention and lack of focus on the projecfs devel- Results by Networkopment objectives. % Satisfactory or Better

V Excellent supervision was characterized especiallyby prompt attention to problems as they arose. 80 -

V The fiduciary aspects of projects (procurement, fi- 60nancial reporting, etc.) were better supervised on 40balance than other elements.

20

(Cotiue o- pag 4)m''(continued or. page 4) ESSD HDD FPSI PREM

Rage~~~~~~~~~~~~~~~~

(continued from page 3)

V Problem projects and older projects were gener-ally less well-supervised. Rapid Supervision Assessment

-A PrimerSupervision quality across the Bank can be en-

hanced quickly and at minimum cost. The keys are What is it?greater management attention and quick and aggres- A fast, limite sc9pgogsk review of a random sample ofsive adaptation of projects to changing circumstances projecta'd'provide annUedback on supervision qual-to permit achievement of development objectives. ity idEnable management tomonitor trends.Some recommendations: U .Z

Who does the Assessmerit"Task Teams-Be realistic and act promptly: 4nel of 10-12 well-respected M staff or consultants.

y Identify problems quickly and report them hat criteria do they us@:candidly. Use the Form 590s to grab manage- panel assesses four princispects: attention toment attention. Don't wait for bad situations d ment objectives, a i to fiduciary responsi-to improve on their own. cesseswp ridnessro ision inputs and pro-

V Expect change and be flexible. Don't hesitate What is the p ss?to modify the project design or scope, espe- The Panel works in gAz alg

cially to simplify. sectoral lines. For each project, a praassessor ratesthe Bank's supervision effort on a4gpointale using a

V Work with the Borrower actively and continu- standardized guidance questionre. The fiis areously in the process of adaptation. vetted by a second assessor. Aoderator adjtdicates

where necessary and ensures c6rgsfstency

V Maintain perspective. Don't lose sight of theultimate project objective. What does QAG do witlh the findings

Individual assessments go to the task teams Thm-

Managers-Stay Involved: mary assessment report is shared with rgbnaln net-work managers as well as Bank senior managent. Linemanagers and sector boards decide onsfllo-u V Select the right task manager and teams and m gs and ct u.

keep them together. What if task teams dis agre

assessment?V Keep on top of major project developments Panel members make every frt to take staff viw

and take decisions promptly. into account in preparing tWli assessment. Nev -

less, it is possible thatM ikopanel memberssmdis-sV Encourage constant project "rework". agree. When this occurs, staff are encouraged to let

QAG know of their views.V Recognize and reward good supervision!

Bank staff can obtain a copy of this report by callingLeila Cruz x34484. S

9 n

S~~~~~~~~~~~~~~~~~~~~ ~~~ D ~