Manoj

Mohanan

(Duke /

Sanford Public

Policy )

NIPFP

Conference

“New Thinking

in Health

Policy”

Oct 26-27

2017

DIFFERENT STROKES FOR

DIFFERENT FOLKS: EXPERIMENTAL EVIDENCE ON

PERFORMANCE INCENTIVE CONTRACTS

* with

Katherine Donato (Harvard),

Grant Miller (Stanford & NBER),

Yulya Truskinovsky (Harvard SPH),

Marcos Vera Hernández (UCL & IFS)

This research was made possible by funding and support from:

3ie &DFID-India (Grant #OW2:205 co-PIs: Mohanan & Miller),

World Bank HRITF (Grant number TF099435: PI Mohanan),

Government of Karnataka.

Usual disclaimers about funders apply.

FUNDING

Performance incentives to correct principal -agent problems ( H a l l & L i e b m a n 1 9 9 8 , H o l m s t r o m & M i l g r o m 1 9 9 1 , J e n s e n & M u r ph y 1 9 9 0 , K h a l i l & L a w a r r e e 1 9 9 5 ,

L a z e a r 2 0 0 0 , P r e n d e r g a s t 2 0 0 2 , R o l a n d 2 0 0 4 , R o s e n t h a l e t a l . 2 0 0 4 ) .

Reward inputs or outputs?

Output contracts assume that agents know the production function

and can find optimal combination of inputs in their own contexts

With low skilled worker, if inputs are observable & verifiable,

rewarding inputs is first best (Khali l and Lawarree 1995; Prendergast 2002).

Do agents perform better when they are able to innovate or do they do

better when they „follow orders‟?

P4P in health globally … ( F i n a n , O l k e n , a n d P a n d e , 2 0 1 5 )

Most P4P programs in health reward measures of service delivery (Ashraf, Bandiera, and Jack 2014, Basinga et al . 2011, Celhay et al . 2015, Dupas and Miguel 2016, Gertler

and Vermeersch 2013, Mil ler and Babiarz 2014)

Evidence of P4P‟s impact on health (outcomes) is mixed (Mil ler and Babiarz 2014; Sherry, Bauhoff , and Mohanan 2017)

V low quality / provider effort in LMIC … large „Know -do‟ gaps

MOTIVATION

Wide theoretical literature on input / output contracts, but

empirical evidence is relatively scarce

2 Key Contributions that we make:

Test effectiveness of input and output incentive contracts

First to empirically compare performance of agents when contracted on

inputs or outputs (esp in health)

Study differential effectiveness of input and output contracts among

providers with varying levels of human capital

Extends literature on optimal contracts and performance incentives

2 Concerns:

Selecting low risk patients

Multitasking (Holmstrom and Milgrom 1991)

OBJECTIVES & CONTRIBUTION

Agents produce health outcomes

Agents with high and low skills; beliefs about θ

Input contract,

Principals can reward specific inputs directly regardless of agents‟

beliefs on productivity shifters θ

But –because θ are local (principals do not know), this could lead

providers in input contracts to pick inefficient levels of effort .

Output contract,

Risk: outcome is not fully under agents‟ control. Premiums need to

compensate agents for this risk

But if providers have correct beliefs about productivity shifters and

hence can choose inputs optimally.

Testable implication: performance will depend on provider

skills in output contracts; but independent under input

contracts.

BRIEF CONCEPTUAL FRAMEWORK

Field experiment: randomize three types of performance

contracts to maternal care providers:

(a) Rewards based on outputs (PPH, Sepsis, Pre-eclampsia and

Neonatal Mortality)

(b) Rewards based on adherence to inputs (WHO / G.o.I. Guidelines)

(c) Control

All providers are given identical WHO / G.o.I . Guidelines as

information material

All providers sign an agreement - A & B sign performance

contracts, C sign an agreement to participate in study on MCH

All providers receive identical participation payments (~$45

at each visit) as compensation for time to answer surveys etc.

THE EXPERIMENT

Outputs ( in PPH, Sepsis, Pre-eclampsia)

Positive payments for reductions below a pre -intervention

level of outcomes.

INR 15000 for avoiding neonatal deaths

α is set based on allocating available balance across range of

improvements in 3 outputs (Exp. 5% min incidence)

Example: pre-intervention rate of PPH = 35% (x -bar). α = INR

850 (~$17); if x i = 25% then the provider gets $170.

THE CONTRACTS

Similar for inputs, except payments for improvements in

adherence to input guidelines above a min. level of

performance on 5 domains of care:

ANC,

Childbirth,

Post Natal Maternal Counseling,

Newborn care,

Post Natal Newborn Counseling.

Providers do not know what inputs are measureable (from

validation) and what the survey questions would look like.

CONTRACTS… CONTD.

135 providers

Eligibility: Pvt. rural practice in areas not served by large

public facilities.

120 from govt. survey data, 15 additional found during our

field visits

56% female, 59% have advanced OBGYN training, 47 yrs, 20

yrs experience, 17 years clinic. (Table 2; for balance see

Appendix Table A1)

PROVIDERS AND RANDOMIZATION

Timeline:

Analysis sample: 25 mothers who delivered at study facility

Additional community sample

DATA COLLECTION

Household surveys

Interview Mothers / attending family member within 2-3 weeks after

each delivery.

Not a cross section, to avoid recall problems (Das et al 2012)

Questions on health history, symptoms of outputs, and recall of

inputs provided (survey included validated and non-validated

questions).

Provider surveys

Expectations, demographics, capacity – before contract

Strategies – 2 months after contract

Follow up surveys – after contract including qualitative.

DATA - 2

Pre-analysis plan on AEA registry

https://www.socialscienceregistry.org/trials/179/history/728

OLS to estimate effect of treatment, controlling for household

and provider characteristics, district and enumerator FE

Clustered at level of provider

Multiple hypothesis testing:

4 outputs, 5 input indices, 2 treatment arms

Familywise error rate (Westfall & Young 1993)

ANALYSIS

(Table 3)

Both groups reduce PPH by about 21% ( rel to 36.5% in C)

PPH most amenable to improvement?

RESULTS - 1

Inputs (Table 4)

No significant improvements in any of the indices – especially

those activities pertaining to PPH

Improvement in postnatal maternal counseling (6) has

unadjusted p = 0.033, but after multiple outcomes correction

it is not significant.

PNCC (10) is important – we will revisit in a few slides.

RESULTS - 2

Potentially dif ficult to see improvements in inputs due to aggregation of many items into index ( Anderson 2008 )

E.g. Active Management of Third Stage of Labor (AMTSL)

Early Cord Clamping, Controlled Cord Traction, Abdominal Massage note

2 specific actions that are most closely related to PPH:

Parenteral Oxytocic Drugs and

Manual Removal of Placenta (potentially reflects complications)

Table 5:

Providers in both arms ~ 7pp more likely to stock drugs (and use them)

Manual removal of placenta is conducted less often (7/27 = 26%)

RESULTS - 3

Ex-post, we see that average payments to outputs was much higher than to input contracts ($1033 v/s $252)

Potentially reflects the risk premium for output contracts

In our setting, input contract was more efficient

Unable to make generalizable inference about efficiency.

COST OF CONTRACTS

Actual Payments to Input Actual Payments to Output

Counterfactual Payments

Our conceptual framework suggests that provider skills would

determine effectiveness of output v/s input contracts

Innovate to meet target in former, v/s follow order in the latter

We look at those with advanced OBGYN training (MBBS+) v/s others

RESULTS ON TYPE OF AGENTS

Do MBBS+ providers Innovate more?

RESULTS ON TYPE OF AGENTS

Table 6:

OUTPUT contract: MBBS+ providers reduced PPH 9pp more than other

providers

INPUT contract: No better or worse

Exploring whether the MBBS+ folks innovated more (Table 7):

OUTPUT contract: increased Pr(new strategy) for MBBS+ providers

Lincom coeff = 0.36 (se = 0.14)

INPUT contract: No increase; lincom coeff = 0.14 (se =0.17)

RESULTS ON TYPE OF AGENTS

Also relates to reduction in PNCC (col 10 in T4) in output arm

75% providers thought PPH was most important to improve

among their patients

On 9% thought NM

Context of NM in

OBGYN care in

India

Col 10 – refers to

counseling about

postnatal care

No change in input

“followed orders”

v/s

Reduction in output

WHY PPH? MULTITASKING?

Output and Input contracts can achieve comparable gains –

and also reduce PPH significantly (major health issue)

Heterogeneity based on skills:

With high skilled workers, output contracts might induce better

performance

In contrast, output contracts with low skilled workers might not be as

effective

Current focus of ongoing incentive programs globally to

reward inputs might in fact be appropriate despite lack of

previous empirical evidence on the rationale for this choice.

CONCLUSIONS & POLICY IMPLICATIONS

Acknowledgements

T h is r esea r c h wa s m a d e p o ss ib le by f u n d in g a n d su p p o r t f ro m 3 ie a n d

DF I D - I n d ia ( G r a n t n u m b er OW 2 : 2 0 5 c o - P I s : M o h a n a n a n d M i l l e r ) , Wo r ld

B a n k H R I T F ( G r a n t n u m b er T F 0 99435: P I M o h a n a n ) a n d G over n m en t o f

Ka r n a t a ka .

We a r e g r a te f u l , fo r c o m m en t s a n d su g g es t io n s , to A lessa n d r a Vo en a ,

A lessa n d ro Ta ro z z i , A m a r H a m o u d i , Du n c a n T h o m a s , E r i c a F ie ld , J e r r y L a

Fo r g ia , J i sh n u Da s , I m r a n R a su l , M er ed i t h Ro sen t h a l , M ic h a e l C a l len ,

N ava A sh r a f , N eer a j S o o d , O r ia n a B a n d ie r a , Pa u l G er t le r , Ro h in i Pa n d e ,

Ro b G a r l i c k , V i c to r ia B a r a n ov, X ia o Yu Wa n g , a n d to a u d ien c es a t

A E A / A S S A 2 017 , A S H E c o n 2 016, B a r c e lon a G S E 2 016 , B R E A D/ C E P R

2 016, Du ke , E r a sm u s , H a r va r d , iH E A C o n g r ess M i la n , T i l b u r g Un i ve r s i t y,

a n d Un i ve r s i t y o f S o u t h er n C a l i fo r n ia .

M a nveen Ko h l i p rov id ed exc e l len t p ro jec t m a n a g em en t . We a r e t h a n k f u l

to Ku l t a r S in g h , S wa p n i l S h ek h a r, a n d A n i l L o b o f ro m S a m b o d h i , a s we l l

a s t h e f i e ld tea m fo r p ro jec t im p lem en t a t io n a n d d a t a c o l l ec t ion .

We g r a te fu l l y a c k n ow ledg e t h e su p p o r t we r ec e i ved f ro m Wo r ld B a n k

( Pa o lo B e l l i , Pa t r i c k M u l len , a n d V ik r a m R a ja n ) a n d t h e G over n m en t o f

Ka r n a t a ka ( Va n d i t a S h a r m a , S e l va Ku m a r, S u r esh M o h a m m ed ,

R a g h aven d r a J a n n u , At u l T iwa r i , D r. N a g a r a j , D r. S r id h a r, D r. P r a ka sh

Ku m a r, D r. A m r u tesh wa r i , a n d seve r a l o t h e r s ) . We a r e esp ec ia l l y g r a te fu l

to t h e m a ny d o c to r s a n d c l in i c a l ex p er t s w h o p rov id ed va lu a b le g u id a n c e

a n d feed b a c k , in c lu d ing M a t t h ew s M a t h a i , D in esh A g a r wa l , Aya b a

Wo r jo la h , V in o d Pa u l , S h a r a d I yen g a r, K i r t i I yen g a r, A m a r j i t S in g h ,

S u n eet a M i t t a l , L a l i t B ave ja , a n d S u n esh Ku m a r

Growing literature on incentives, performance, and

personality traits

Focus on Conscientiousness and Neuroticism – correlated with

labor market outcomes (Borghans et a l . 2008, Heckman, St ixrud and Urzua

2006, Heckman and Rubinste in 2001)

Conscientiousness : dependability, organization skills , perseverance,

and achievement oriented thinking

Neuroticism: the converse of emotional stability – is associated with

anxiety, worry, anger, and insecurity

PERSONALITY TRAITS &

PERFORMANCE CONTRACTS

CONSCIENTIOUSNESS & NEUROTICISM

10/31/2017 AEA Conference - Chicago

Conscientious providers do relatively better absent incentives.

Beneficial effect of incentive is weaker among high Cons. providers

At 25th percentile of Conscientiousness (4.3/5), the incentive contracts reduce PPH risk by 13.3 %pts.

At 75th percentile of C. (5/5), no statistically significant effect

No evidence of association b/w neuroticism and performance in absence of incentives (could be due to selection into MD?)

Performance improvement from incentives is amplified among low Neuroticism (high emotional stabil ity):

13%point reduction at 25 th percentile (1.25/5),

No significant results at 75 th percentile.

“Choking under pressure” hypothesis, -- performance deteriorates due to over-arousal and distraction that accompany high stakes (Ariely et al. 2009,

Baumeister 1984, Yu 2015)

PERSONALITY TRAITS (AER P&P)

PERSONALITY TRAITS (AER P&P)

Output and Input contracts can achieve comparable gains – and also reduce PPH significantly (major health issue)

Heterogeneity based on skil ls:

With high skilled workers, output contracts might induce better performance

Output contracts with low skilled workers might not be as effective

Personality traits

Evidence of significant heterogeneity by personality traits – both high conscientiousness and high neuroticism providers don ‟t show improvements with incentives.

Among high conscientiousness providers, the dampened effect is suggestive of crowding out.

Among high neuroticism (the converse of emotional stability) these results are consistent with “choking”

Current focus of ongoing incentive programs globally to reward inputs might in fact be appropriate despite lack of previous empirical evidence on the rationale for this choice.

CONCLUSIONS & POLICY IMPLICATIONS