April 21, 2023April 21, 2023 11

DATA SOURCESDATA SOURCES

BYBY

DR. DC TSHIBANGUDR. DC TSHIBANGU

April 21, 2023April 21, 2023 22

SESSION OBJECTIVESSESSION OBJECTIVES

• Define what is a health information system (HIS) and understand its components

• Define routine health data/information• Discuss routine data collection methods • Define non-routine data• Discuss methods of collection for non-

routine data

April 21, 2023April 21, 2023 33

SESSION OBJECTIVESSESSION OBJECTIVESTo help M & E OFFICERS to:

• Appreciate the varied sources & forms of information on specific project/program/service

• Develop a toolkit for thinking about the complexity of information and its uses

• Assess the completeness, accuracy, relevance and timeliness of available information

• Decide which types of information are most appropriate for a particular activity within a project/program

• Make optimal use of information which is not ideal, and assess the effects of its departure from perfection

April 21, 2023April 21, 2023 44

FROM REALITY TO ACTIONFROM REALITY TO ACTION

Real worldReal world(Collection, coding)(Collection, coding)

DataData(Processing, interpretation, presentation)(Processing, interpretation, presentation)

InformationInformation (Politics, commitment)(Politics, commitment)

ActionAction

Source: Oxford Handbook of Public Health PracticeSource: Oxford Handbook of Public Health Practice

April 21, 2023April 21, 2023 55

USE OF WORDS ‘DATA’ & ‘INFORMATION’USE OF WORDS ‘DATA’ & ‘INFORMATION’

• DATUM (singular) or DATA (plural) refers to raw numbers or other measures, usually discrete and gives objective facts about events.

• INFORMATION refers to what emerges when data are processed, analyzed, interpreted and presented. Information is data transformed (contextualized, categorized, corrected, calculated, condensed) into a message

April 21, 2023April 21, 2023 66

WHEN TRANSFORMING DATAWHEN TRANSFORMING DATA

Always bear in mind the issues that affect the quality of the data:

• Validity - are the data capturing the concept or quantity you intended?

• Selection bias – where the data mislead because they are not representative of the population

• Classification bias – where there is a non-random effect on putting data into groupings (non-blind assessments of any outcome)

April 21, 2023April 21, 2023 77

KINDS OF DATA SOURCESKINDS OF DATA SOURCES

In most countries, there are many different sources of information on any

Specific project/program/service and different types of information vary in their

C.A.R.T:

• Completeness• Accuracy• Relevance and/or Representativeness• Timeliness

DATA SOURCES also vary in the ease with which a

base population can be identified, for use in the

denominator, for calculating rates.

April 21, 2023April 21, 2023 88

WHAT DOES THE DATA SOURCEWHAT DOES THE DATA SOURCE DESCRIBEDESCRIBE??

This depends on the goals/ objectives of program and may include information such as:

• Demographic & Socioeconomic features of the study-population: age, sex, education, occupation, mobility and geographical distribution.

• Health status: health service use data (diagnoses, interventions, procedures, health outcomes of interventions), morbidity, mortality (TB, Malnutrition, HIV/AIDS, co-infections and OIs)

• Programmatic: inputs, process, outputs, outcome & impact

April 21, 2023April 21, 2023 99

HOW IS THE INFORMATION COLLECTED?HOW IS THE INFORMATION COLLECTED?

Information can be Routine or Specially collected

• Routine refers to collected, assembled, and made available regularly, according to well-defined protocols and standards.

Such data are usually available at regular intervals

They intend to allow tracking over time

They are codified using national or international standards (ICD)

April 21, 2023April 21, 2023 1010

HOW IS THE INFORMATION COLLECTED?HOW IS THE INFORMATION COLLECTED?

• Specially collected refers to collection for a particular purpose, without the intention of regular repetition or adherence to standards (other than those needed for the

specific study or tasks); such data are usually:

- aimed at a specific , time-limited study or tasks;

- codified according to the goals in hand and the

wishes of the investigators.

April 21, 2023April 21, 2023 1111

CLASSIFICATION OF INTRINSIC TYPES CLASSIFICATION OF INTRINSIC TYPES OF DATAOF DATA

Sometimes data are categorized as hard or soft:

Hard data: are precise (or intend to be precise):

They are often numerical; if not, then coded according to

a protocol;

They are reproducible, and likely to be similar even if the

data collectors are varied.

April 21, 2023April 21, 2023 1212

CLASSIFICATION OF INTRINSIC TYPES CLASSIFICATION OF INTRINSIC TYPES OF DATAOF DATA

Soft data: tend to be:

- qualitative, attempting to capture some of the

subtlety of human experience;

- often narrative or textual form, at least as

they are collected;

- Imbued with some subjectivity, due to the

complexity of the personalities of the data

collectors and the individuals studied.

April 21, 2023April 21, 2023 1313

THE UTILITY OF THE INFORMATIONTHE UTILITY OF THE INFORMATION

Neither hard nor soft data are intrinsically better than the other.

The utility of the information (in terms of better decision making)

often comes from combining the two:

• Harder data usually allow more precise analysis and comparisons, but may fail to capture subtleties.

• Softer data usually capture more of the ‘truth’ about the world, but often at the expense of emphasizing the uniqueness of the circumstances, and are less likely to allow comparisons and conclusions.

April 21, 2023April 21, 2023 1414

DATAWISE WHAT DO YOU NEED TO DATAWISE WHAT DO YOU NEED TO ASSESS?ASSESS?

You need to assess ‘the fitness for purpose’

by asking the following question:

Are the existing or proposed sources of data

fit for the purpose for which they are intended,

the conclusion to be drawn or the decision to

be made?

April 21, 2023April 21, 2023 1515

KEY ISSUES FOR ASSESSING APPROPRIATENESSKEY ISSUES FOR ASSESSING APPROPRIATENESSAND USEFULNESS OF DATA & DATA SOURCES AND USEFULNESS OF DATA & DATA SOURCES

Here are some guiding issues but none is absolute, and the

balance of advantage & disadvantage must be assessed using

judgment.

• Technical issues

- Are the definitions clear and appropriate?

- Are the target and study population clear?

- Are the data collection methods clear and sound?

- How complete, accurate, relevant, and timely are the data?

How much does this matter?

April 21, 2023April 21, 2023 1616

KEY ISSUES FOR ASSESSING APPROPRIATENESSKEY ISSUES FOR ASSESSING APPROPRIATENESSAND USEFULNESS OF DATA & DATA SOURCES AND USEFULNESS OF DATA & DATA SOURCES

• Issues relating to outcome or decision involved

- Is the study population sufficiently representative of the target

population for the purpose of the decision?

- Do you need absolute or relative estimates, to make the best

decision ?

- Would existing data source suffice, by using comparative

data or by extrapolating with care?

- Would qualitative information suffice, when habit automatically

suggests quantitative data?

April 21, 2023April 21, 2023 1717

Health Information Systems (HIS)Health Information Systems (HIS)

• Health system – All resources, organizations and actors that are

involved in the regulation, financing, and provision of actions whose primary intent is to protect, promote or improve health.” (WHO, 2000)

• Health Information System (HIS):– A system that provides specific information support to

the decision-making process at each level of an organization (Hurtubise, 1984)

– Similar to a health management information system (HMIS)

April 21, 2023April 21, 2023 1818

What is the problem with many existing What is the problem with many existing routine health information systems routine health information systems

(RHIS)?(RHIS)?

• Irrelevance and poor quality of the data collected

• Fragmentation into “program- oriented” information systems: duplication and waste

• Centralization of information management without feedback to lower levels

• Poor and inadequately used health information system infrastructure

April 21, 2023April 21, 2023 1919

As a result…As a result…• Poor use of information by users at all

levels: care providers as well as managers

• “Block” between facility and community health information systems

• Reliance on more expensive survey data collection methods

April 21, 2023April 21, 2023 2121

What characterizes a good HIS?What characterizes a good HIS?

• Regular production of good quality data

• Continued use of health data for improving health system operations and health status.

April 21, 2023April 21, 2023 2222

·Standard indicators

·Data collection forms

·Appropriate IT

·Data presentation

·Trained people

Technicalfactors

What influences data quality and use?

April 21, 2023April 21, 2023 2323

·Resources

·Structure of the health system

·Roles, and responsibilities

·Organizational culture

System and environmen

tfactors

What influences data quality and use?

April 21, 2023April 21, 2023 2424

·Motivation

·Attitudes and values

·Confidence

·Sense of responsibility

Behavioral

factors

What influences data quality and use?

April 21, 2023April 21, 2023 2525

SYNOPSIS OF SOME HEALTH & SOCIAL SYNOPSIS OF SOME HEALTH & SOCIAL PROGRAMSPROGRAMS

• Malaria Program• TB Program• HIV Program• Nutrition Program• Family Planning Program• Immunization Program• Tobacco Prevention Program• Poverty Alleviation Program

April 21, 2023April 21, 2023 2626

M & E MANAGERSM & E MANAGERS

• Are likely to get involved in all or some of these programs

• The selection/choice of appropriate Data Sources depends upon the type of program one is involved in.

• Some selected examples are provided below:

April 21, 2023April 21, 2023 2727

M & E HIV/AIDS MANAGERSM & E HIV/AIDS MANAGERS

Are likely to get involved in

• Preventive Programs and/or• Care & ART Programs and/or• Support Programs

and

The selection/choice of appropriate Data Sources is

dictated by the type of HIV programs.

April 21, 2023April 21, 2023 2828

M & E POVERTY PROJECT M & E POVERTY PROJECT MANAGERSMANAGERS

Are likely to get involved in

• Designing and Implementing Poverty-targeted programs

and

The selection/choice of appropriate Data Sources

depends on whether one needs to determine who

should qualify for services and who should not.

April 21, 2023April 21, 2023 3333

FIVE MINUTE EXERCISEFIVE MINUTE EXERCISE

1. Choose any population/health/nutrition program

2. Define one objective of that program

3. List 3 data sources and 3 reasons why you have selected them

April 21, 2023April 21, 2023 3434

DATA SYSTEMS

• TWO TYPES OF DATA SYSTEMS:

ROUTINE: Health information systems

NON-ROUTINE:

- Surveys

- Research programs

April 21, 2023April 21, 2023 3535

ROUTINE DATA SOURCESROUTINE DATA SOURCES

• Such as HIS (Health Information System) and its subsystems that are collected as part of an ongoing system

April 21, 2023April 21, 2023 3636

CHARACTERISTICS OF HISCHARACTERISTICS OF HIS

• A health system is not a static phenomena. It is in a continuous process of change due to pressures from both outside and within the system

• HIS is an integral part of the health system

• HIS generates the data to measure the change of a health system

April 21, 2023April 21, 2023 3838

NON-ROUTINE DATA SOURCESNON-ROUTINE DATA SOURCES

Such as

• DHS

• Special Surveys

• Program or Project Evaluation

• Clinical trials

• Epidemiological Surveys (Descriptive/Analytical)

April 21, 2023April 21, 2023 3939

Non-Routine Data Sources by LevelsNon-Routine Data Sources by Levels

Policy or program level

Facility/Service delivery point level

Client level

Population level

April 21, 2023April 21, 2023 4040

LEVELS OF INFORMATION WITHIN THE LEVELS OF INFORMATION WITHIN THE IDENTIFIED DATA SOURCESIDENTIFIED DATA SOURCES

The next quest is to identified the level of information one is interested in within identified theData sources

• FIVE LEVELS OF DATA:1. Policy or Program level2. Population level3. Service Environment level4. Client level5. Spatial/Geographic level

April 21, 2023April 21, 2023 4141

POLICY/PROGRAM LEVELPOLICY/PROGRAM LEVEL

• This is policy/legislation formulation level, Sources of:

- Official legislative & administrative documents- National budgets or other related data- Policy inquiries- Reputational rankings (program efforts scores)

• Tools:- Indexing questionnaires (for country specialists

and rankings)- Special/contract studies

April 21, 2023April 21, 2023 4242

FACILITY LEVELFACILITY LEVEL

Facilities-services, infrastructure, etc. Audits/inventories Facility surveys

Health care providers, other staff Performance reviews, competency

measures Training records

April 21, 2023April 21, 2023 4343

POPULATION LEVELPOPULATION LEVEL

Where you need to know the size/composition of a population.

Sources such as: - Population census bureau; - Sentinel surveillance systems

- Vital statistics system (birth & death certificates)

- Sample households or individuals; - Special population samples

(demographic/occupational group, or geographic sector)

Tools:- Birth/Death certificates

- Census questionnaires

- Household/Individual Special Surveys

April 21, 2023April 21, 2023 4444

SERVICE ENVIRONMENT LEVELSERVICE ENVIRONMENT LEVEL

This is a complex level requiring different types of data from

Sources such as:- Administrative records (service stats, HMIS data, financial & transport

data)- Service delivery point information (audit information, inventories, facility survey data)- Staff information (performance assessments, training records, provider

data, quality of care data)- Client visit registers

Tools:- Health Service Information Systems; - Facility Sample Surveys; - Facility records; - Performance Monitoring Reports

April 21, 2023April 21, 2023 4545

INDIVIDUAL LEVELINDIVIDUAL LEVEL

“Individual” in this context refers to a client, participant,patient or documents related to a single person as can be obtained from • Sources such as:

- Medical records; - Interview data; - Case Surveillance (epidemiology of disease)- Provider-Client interactions

Tools:- Case reports; - Survey questionnaire; - Client register analysis- Patient flow analysis; - Direct observation

April 21, 2023April 21, 2023 4646

INDIVIDUAL LEVELINDIVIDUAL LEVEL

Can measure “program exposure” represented by utilization, as well as service experience, quality of care/service delivery, disease surveillance

– Is the volume increasing?– What is the service mix?– Who are the clients?

• How does it vary by public/private sector?

– What are their consultation experiences?• Would they return/recommend the service?

Other questions?

April 21, 2023April 21, 2023 4747

INDIVIDUAL LEVELINDIVIDUAL LEVEL

1. Client Exit Interviews

2. Case surveillance (epidemiology)

3. Provider-client observation

4. Service Delivery Point records and

registers

5. Patient-flow analysis

6. Others?

April 21, 2023April 21, 2023 4848

MEASUREMENTMEASUREMENT TOOLSTOOLS

• Facility audits, Inventories• Facility surveys • Provider interviews• Provider-client observation• Provider training records• Situation analysis• Others

April 21, 2023April 21, 2023 4949

Some Strengths and Weaknesses of Facility Some Strengths and Weaknesses of Facility Surveys as a Source of M&E DataSurveys as a Source of M&E Data

• Strengths– Can cover both public and private health facilities – Timing can coincide with program implementation– Can combine with population survey for outcome

monitoring and impact evaluation

• Limitations– Survey sampling design and analysis may be

complex– Expensive, time-consuming– Information rapidly outdated, unless repeated

• Others??

April 21, 2023April 21, 2023 5050

Surveys: When are they appropriateSurveys: When are they appropriate??

• Surveys especially useful-– when other data are not available or inadequate– when they can be tailored to fit specific measurement

objectives• Yield cost-efficient data on population and services• Good sampling techniques produce representative

results for facilities, providers and clients• Surveys are expensive, but versatile and widely used

April 21, 2023April 21, 2023 5151

Rapid Appraisal / Qualitative MethodsRapid Appraisal / Qualitative Methods

• Key Informant Interviews

• Focus Group Discussions

• Community Interviews

• Direct Observation

April 21, 2023April 21, 2023 5252

COMPONENTS OF DATA SYSTEMCOMPONENTS OF DATA SYSTEM

• A sound Data System is likely to have:

1. Multiple, operationally defined indicators2. A variety of Appropriate Data Sources3. Baseline and Target Values4. Feasible Data Collection Plan and Budget:

- Specified Frequency- Identified Responsibility

April 21, 2023April 21, 2023 5353

GEOGRAPHIC LEVELGEOGRAPHIC LEVEL

These are modern and specialized sources that include:

- Cadastral maps (land ownership)- Land Demarcation Department with: - Satellite Imagery and Area Photography - Digital Line Graphs and Elevation Models

Tools:

- Global Positioning System- Computer Software Programs (GIS)

April 21, 2023April 21, 2023 5555

CONCLUSION: DATA SOURCES and CONCLUSION: DATA SOURCES and YOUR M&E PLANYOUR M&E PLAN

• Assess the type of information your program/project needs

• Assess what information is already available and from what sources and levels

• Use those sources to help developing your M&E Plan

• Decide what gaps need to be filled and plan accordingly

• Diagram the flow of data through the M&E system (collection to analysis)

April 21, 2023April 21, 2023 5656

THANK YOUTHANK YOU