data triangulation

Data Triangulation

Objectives:

At the end of the session, participants will be able to:

Describe the role of data triangulation in program evaluation

List data sources and approaches that can be used for triangulated analysis

Data Triangulation for M&E

Linking different information sources involving persons, place and time

Analysis of data from multiple sources can increase the validity and reliability of findings; it can corroborate findings and weakness of any one data source can be compensated for by the strengths of another

Analysis of program level data with outcome/impact level data can help substantiate the linkage between program interventions and population-level outcomes/impacts

Questions that data triangulation can help address:

Are interventions working and are they making a difference?

What changes in population-level outcome and impact indicators have been observed and what do they mean?

Can the observed changes in outcomes and impact indicators be attributed to program outputs?

Are the collective efforts being implemented on a large enough scale to impact the course of the epidemic?

Source: A framework for monitoring and evaluating HIV prevention programmes for most-at-risk populations. UNAIDS 2007

Source of data

Repeated HIV and/or STI prevalence surveys/surveillance

Repeated population surveys with behavioral data

Routine program or service delivery data

Quality of service assurance and quality improvement assessments

Qualitative studies (in-depth interviews, focus group discussions, key informant interviews, etc)


When to do Data Triangulation

When interventions have been in place for a sufficient duration of time to reasonably expect that changes at the population level may be attributable to program interventions

When interventions have been implemented with sufficient intensity and with high enough coverage to reasonably expect effects to be observed in the target population.

When good program-level data (i.e outputs, coverage, quality of implementation are available) are available

When process evaluation indicates that program activities are being realized as planned.


Considerations

Is there a culture of data sharing?

Are data from different sources representative of te population ofinterest?

Do the time frames of te data points match?

Example 1:

Female Sex Worker Program in Nepal

Family Health International

Logic of the program in Nepal Reach people with information to increase their

knowledge and awareness Provide services to allow them to act on that

knowledge Expand coverage of information and services so that

so that changes are observable at the population-level

Improve the quality of services while expanding coverage (e.g. STI treatment)

Information and good quality services for enough target group members will lead to service use, which will lead to safer behaviour and lower risk of exposure to HIV

Source: FHI Regional Office, Bangkok.

Number of sex workers reached and contacts by type of staff

02,0004,0006,0008,000

10,00012,00014,00016,000

2000 2001 2002 2003 2004

Year

Total Number Reached

Total Number of Contacts -Outreach WorkersTotal Number of Contacts -Peer Educators

FHI Nepal, 2005

Program records indicate the program is being delivered and reaching more of the target

population through more channels over time


This increasing coverage is confirmed by FSWs in target areas

(exposure to NGO-related activities among FSWs, 1998 – 2002)

29 31

61.868.3

32.3 30

47.8

64.8

38.5

48

63

74

52

44.3

68

21.5 19.3

39.344.3

56.4

0

10

20

30

40

50

60

70

80

90

100

1998 1999 2000 2001 2002

Per

cent

age

(%)

Received condoms

Received brochures/booklets/pamphlets

Received other information about HIV/AIDS

Received items/information from all three sources


There were signs of improved quality of services e.g. STI diagnosis and treatment

Quality Criteria Situation Prior to Dec. 2003 Current Situation

Treatment All STI patients receiving treatment as per the STI management guidelines. Drug quality needs to be standardized. Follow up visits are infrequent.

Compliance of the drug therapy cannot be assured with multi-dose drugs.

Drugs supply is of better quality. Drugs are mostly single dose and taken under direct observation, improving compliance

Clinic location & accessibility

Most static clinics placed centrally in the town and are easily accessible but mobile clinics are kept at DIC or government owned offices and changed from time to time

Static clinics which were not appropriately located have been changed and improved

Clinic setup Clinics have mostly 3 rooms one each for registration, consultation and laboratory test. Only some of the clinics have private rooms for consultation and examination

VCT now integrated into STI clinics, and a separate room for counseling has been added in most static clinics

Staffing Clinical teams consist of 3 persons: one medical officer, a staff nurse and a laboratory technician. Frequent turnover of the trained staff.

Addition of one trained VCT counselor in static clinics. Staff turnover rate has decreased.

Clinical Skill General history taking, general and genital examination are performed confidently, oral and anal examinations are not routinely conducted

Improved sexual history taking skills and clinical skills, but regular guidance still needed.


Trends in desirable outcomes are evident – Knowledge of condoms is high among FSWs and condom carrying

behavior is increasing over time, 1998 - 2002

94.5 98 98.3 99.392.8 95.8 97.9 96.9

92.285

28.334.7

39.3

54.858.4

0102030405060708090

100

1998 1999 2000 2001 2002

Per

cen

tag

e (%

)

Ever heard of condoms

Know using condom can prevent HIV transmission

Usually carry condoms with them


Trends in Desirable Outcomes are evident – Condom use with clients among FSWs is increasing steadily over

time, 1998 - 2002

77.582.3

94 95

61.867

87.3 90.3

3340.3

51 54.3

19.826.5

36.3 40.2

93.386

44.8

38.7

0102030405060708090

100

1998 1999 2000 2001 2002

Per

cen

tag

e (%

)

Ever use of condoms

Use of condoms with last client

Consistent use of condoms with clients in the past year

Gave condom to client at last sex


Those Reached by the Program Have Safer Behaviors: Exposure to various NGO-related activities in the past year impacts on

consistent condom use with clients among FSWs - 2002

64.8 65.7

36.7 37.3

61.5

38.4

0

10

20

30

40

50

60

70

80

90

100

Received condoms Received brochures &materials

Received IPC informationabout HIV/AIDS

Co

nsi

sten

t Co

nd

om

Use

(%)

Exposed Unexposed


There is evidence of a favourable Dose-Response relationship:

Greater exposure to various NGO-related activities results in better condom use behavior - 2002

24.6

33.3

75.8

28.2

39.7 42.1

6869.276.2

83.9

91.9

48.2

0

10

20

30

40

50

60

70

80

90

100

None Any one* Any two* All three*

Per

cen

tag

e (%

)

Carrying condoms Consistent condom use with clients

Consistent condom use with regular clients

*1) Received condoms, 2) received brochures/materials, 3) received IPC information about HIV/AIDS


Summary of Key Findings among FSWs

Program activities have expanded in scale and improved in quality

HIV and STI prevalence are decreasing over time

Knowledge of condoms is high and consistent condom use is increasing and high among clients and regular clients

Exposure to NGO-related information sources about HIV/AIDS and condoms is increasing and high

Exposure to various NGO-related activities is increasing and high

Exposure to NGO-related activities is strongly linked with condom carrying & consistent condom use in a dose-response manner


Example 2: Botswana

Antiretroviral scale-up in Botswana

National AIDS Committee Botswana,

WHO, UCSF and UNAIDS (2006)

Overview of Botswana Triangulation

Objective:

To develop a model to measure the impacts of ART and PMTCT programs on adult and child mortality

Application:

Provide timely information on the impact of national scale-up of ART for policy and programmatic decision making

Source: Case Study. Country-enhanced monitoring and evaluation for antiretroviral therapy scale-up: analysis and use of strategic information in Botswana. WHO 2006.

Approach Meetings with stakeholders to identify objectives of

analysis

Identification and compilation of data from many sources Vital statistics (morbidity and mortality)

Population survey data

Patient Management systems (HMIS)

Program data (i.e. HIV testing, ART)

Examination of trends in HIV prevalence and mortality in relation to ART and PMTCT availability and service uptake


Conclusions of Botswana Analysis Since the inception the ART programme,

Botswana has achieved reductions in mortality of adults aged 25–54 years

Reduced mortality is associated with early initiation of district ART programmes and with the overall rate of ART uptake in the district.



Strengths and Limitations of Triangulation

Strengths

Pre-existing data sources are used

Can provide relatively rapid results

Limitations

Existing data may be insufficient

Institutional barriers to data sharing

Triangulation Resources

HIV Triangulation Resource Guide: Synthesis of Results from Multiple Data Sources for Evaluation and Decision-making (WHO 2009)

Data Triangulation for HIV Prevention Program Evaluation in Low and Conncetrated Epidemics (FHI 2010)

MEASURE Evaluation is a MEASURE project funded by the

U.S. Agency for International Development and implemented by

the Carolina Population Center at the University of North Carolina

at Chapel Hill in partnership with Futures Group International,

ICF Macro, John Snow, Inc., Management Sciences for Health,

and Tulane University. Views expressed in this presentation do not

necessarily reflect the views of USAID or the U.S. Government.

MEASURE Evaluation is the USAID Global Health Bureau's

primary vehicle for supporting improvements in monitoring and

evaluation in population, health and nutrition worldwide.

data triangulation

Documents

program level data

program outputs

timeanalysis of data

data triangulationobjectives

good programlevel data

population level

program activities

program records