Can Measurement Increase Quality?Or

What if we Implemented RBF without the F?

Kenneth L. LeonardUniversity of Maryland, College Park

A project with repeated measurement of protocol adherence (based on patient exit interviews)

• Sample of 96 health workers providing outpatient care in urban and peri-Urban Arusha, Tanzania.• Assessed for protocol adherence for fever,

cough and diarrhea with a general assessment of other conditions. • Interviews with patients after they left the

consultation. • Enumerators were unannounced, although it

appears that, after some time at the facility, health workers figured out there was someone around. (more on this later).

• Brock, M, Lange, A and Leonard K.L. “Giving and Promising Gifts: Experimental Evidence on Reciprocity from the Field”, mimeo 2015

• Brock, M, Lange, A and Leonard K.L., “Generosity and Prosocial behavior in Health Care Provision: Evidence from the Laboratory and Field”, Journal of Human Resources, forthcoming.

• Leonard, K.L. and Masatu. M.C. “Changing Provider Performance Incentives without Payments: The Long Term Impacts of a Program to Encourage Quality in Outpatient Care”, mimeo 2015.

Baseline Adherence

baseline72

74

76

78

80

82

84

overall adherence

overall adherence

Now a peer sits in the room to watch

baseline scrutiny72

74

76

78

80

82

84

overall adherence

overall adherence

Then the peer leaves

baseline scrutiny post scrutiny72

74

76

78

80

82

84

overall adherence

overall adherence

A well known doctor visits each health worker with an “encouragement” script, priming key items of protocol (no data collected)

baseline scrutiny post scrutiny72

74

76

78

80

82

84

overall adherence

overall adherence

Encouragement Visit

Quality about 2 to 4 weeks later

baseline scrutiny post scrutiny post encouragement72

74

76

78

80

82

84

overall adherence primed not primed

Encouragement Visit

Randomly Assigned Feedback Visit, discussing results, implied end of study (no data collected)

baseline scrutiny post scrutiny post encouragement72

74

76

78

80

82

84

overall adherence primed not primed

Encouragement Visit Feedback Visit

Quality collected again, about 2 weeks later

baseline scrutiny post scrutiny post encouragement post study72

74

76

78

80

82

84

overall adherence primed not primed

Encouragement Visit Feedback Visit

Quality collected about two years later

baseline scrutiny post scrutiny post encouragement post study long term follow-up72

74

76

78

80

82

84

overall adherence primed not primed

Encouragement Visit Feedback Visit

What happened?

• Immediate scrutiny increases effort, but has no lasting impact• Encouragement, by itself has limited impact, but only on those items

specifically mentioned• The feedback visit looks like it improves effort, but half the health

workers received no feedback and their improvement is exactly the same, so feedback isn’t helping.• The gains are long-lasting.

What does it mean?

• The project causes health workers to increase their effort and quality without any increase in resources or training• And without any financial incentives:• This is RBF without F.

• This is not a program to be implemented at scale but a indicator that the necessary elements of expectations and accountability are missing in many health systems. • However, these necessary elements are not sufficient by themselves:

we need measurement to complete the picture.

Measurement doesn’t improve quality, but if health workers know that:

Then, we get increased effort

Effort is being measured

Measurable effort improves

Outcomes

Effort (quality) can be

Measured