1

Udo Buchholz,

WHO/Stop TB/TME

Operational research: methods and examples

2

What is operational research? (OR)

• Definitions found on the internet:– "Mathematical common sense"– "Systematic study, by observation and

experiment, of the working of a system, e.g. health services, with a view to improvement"

– "Using scientific methods to attack a complex problem or system"

3

In the beginning there was ... a question

"Why in the world is it that 30% of our patients on treatment default?"

NTP manager in the morning

4

Description of defaulters in Russia1

• Profession: unemployed: 26%, labourers 21%, students of vocational schools 19%, disabled 7%

• Education: incomplete secondary education: 70%

• Residence: homeless 5%, >5km away from treatment site 26%

• Behavioural risk factors: alcoholism 44%

1Data are from W. Jakubowiak, Russia

5

Are these variables risk factors for default? – use of patient cohort for cohort study

0%

1%

2%

3%

4%

5%

6%

7%

8%

9%

Alcoholics No alcohol addiction

p-value<0.001

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Ex-prisoner No prison history

p-value<0.001

6

Social support system

• Examples from different oblasts:– Food incentives– Hygienic kits– Free transportation– Psychological support– ....

7

Adherence with social support

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Ex-prisoner No prison history

p-value=0.2

8

More examples

• "Defaulting from anti-tuberculous treatment in a teaching hospital in Rio de Janeiro, Brazil" (IJTLD 2004)

• "A concurrent comparison of home and sanatorium treatment of PTB in South India" (BWHO 1959)

• " 'Lost' smear positive PTB cases: where are they and why did we lose them?" (IJTLD 2005)

9

Determinants of a study

• Problem or question

• Data available

• Funding and staff available

• Political or hierarchical support Type of study

10

Which scientific methods can we use? - Type of studies

• Descriptive studies– Analysis of surveillance data– Ecological study (correlational)– Cross-sectional survey

• Analytical studies– Observational (case-control study, cohort

study)– Experimental

• Other– E.g. capture-recapture study

11

Smear-positive diagnosis by province, Syria

0.00.10.20.30.40.50.60.70.80.91.0

A Ar D H I K L M OS Q R RD S SW T Z

ss+

/all

pu

lmo

nar

yExample: Surveillance data reveal large provincial

differences of ss+ TB/all PTB

12

No. of slides/patient is correlated with proportion of ss+/PTB

Smear-positive diagnosis: Syria

20

30

40

50

60

70

80

90

100

1.6 1.8 2.0 2.2 2.4 2.6 2.8 3.0 3.2 3.4

slides/patient

ss

+/a

ll p

ulm

on

ary

13

Ecological comparison (correlational)

• Correlation of aggregated or group data

• Association on the individual level is unknown and may be different

• Many relationships on global level are strictly speaking of ecological nature

14

Example of an "ecological" comparison: The prevalence of HIV in TB patients (y-axis) against the prevalence of HIV in adults (x-

axis).

ZIM

TAN

SOA

RWA

NIE

MOZ

LES

KEN

GHA

ETH

DJI

COD

IVC

CNG

CAE

BUU

BFA

MAL

KEN

HAI

DJI

IVC

CAF

CAM

BFA

BOT

0

20

40

60

80

0 10 20 30 40Estimated prevalence of HIV in adults (%)

Mea

sure

d p

reva

len

ce o

f H

IV i

n T

B p

atie

nts

(%

)

15

Cross-sectional survey

• Collection of representative data

• Based on sampling size calculations, sampling frame and sampling scheme– Simple random sample– Systematic sampling– Cluster sample (design effect!)

16

Surveys are frequently used in TB epidemiology

• Sampling universe is the population:– Prevalence surveys– Tuberculin skin test surveys

• Sampling universe is "all TB patients"– Proportion of diagnosed new TB patients with

HIV test

• Sampling universe is the number of culture positive TB patients– Drug resistance surveys

17

Analytical studies

• Are used to identify risk factors or other forms of "exposure" and their association with an outcome, e.g. death, default, etc.

• Make use of a comparison group• Hypotheses are tested• Null hypothesis: "There is no association

of exposure and outcome" or: "Exposure and outcome are independent"

We then calculate the probability that this is true based on the data

18

Case control study

• Starts with a group of cases, i.e. with a certain outcome, that is consistent with a case definition

• The case definition must be specific in regards to time, place and person

• E.g. "a person with smear positive TB diagnosed in Geneva city in 2004"

• Then select a group of persons without the outcome from the same population, here for example the general population

• From the case definition it follows: "a person without TB living in Geneva in 2004"

19

Case control study: ascertainment of exposure status

• After identification of cases and controls the exposure status preceding the outcome is investigated

• E.g.: income (high versus low)

• Thus, the directionality is usually retrospective

20

Selection of controls

• Imagine the cohort from which the cases would have arisen

• Or: Would the control have been a case if he/she had had the outcome in question?

• Example: cases of rare kidney disease in the Mayo clinic

21

Typical control options

• Friend controls• Neighbourhood controls• Physician controls• Hospital controls• Population-based controls

• Consider: – Selection bias– Feasibility

22

2 x 2 table (CCS (1))

• 50/1000 ss+ TB cases (5%) were poor, but only 5 of 2000 (0.25%) among the non-TB persons

Ss+ TB patients were 20 times more likely than the general population to be poor, however ...

Ss+ TB No ss+ TB

Low income 50 5

High income 950 1995

1000 2000 3000

23

2 x 2 table (CCS (2))

• The chances of ss+TB patients to be poor is expressed as the odds = probability of poverty / prob of rich = 50/1000 / 950/1000 = 0.053

• The odds of non TB persons for poverty is therefore:5/2000 / 1995/2000 = 0.00251

• The ratio of the two odds (the odds ratio (OR)) is: 0.053/0.00251 = 21

Ss+ TB No ss+ TB

Low income 50 5

High income 950 1995

1000 2000 3000

24

Use of case control studies

• When type of outcome is rare• We can examine >1 exposure• Usually relatively quick and inexpensive• Disadvantages:

– Not useful for rare exposures– Because exposure is in the past: watch out for

recall bias– Selection of cases and controls often not

straightforward (selection bias)

25

Cohort study

• Starts with a group of people or a population that can be divided in two groups based on a defined exposure which some have and some don't

• The groups are then followed-up and an outcome is counted

• A case definition is still important• The directionality is usually forward, but

can also be backwards (retrospective cohort study)

26

2 x 2 table (cohort study)

• We follow 100 low income TB patients and 200 high income TB patients up for adverse outcomes

• It turns out that 20 of 100 (20%) poor have a bad outcome versus 10 of 200 (5%) of the rich.

• Thus, the poor are 4 times more likely to have an adverse treatment outcome.

• Measure of association is the risk ratio (RR) = 0.2/0.05 = 4

Adverse outcome

Treatment success

Low income TB patient

20 80 100

High income TB patient 10 190 200

300

27

Use of cohort studies

• When exposure is rare• We can examine >1 outcome• The outcome measure for the strata is an incidence

rate or (cumulative) risk and the overall point estimate the rate ratio or risk ratio (RR)

• Disadvantages:– Not suitable for rare outcomes– Not ideal for outcomes in the far future (unless you have

much time or lots of scientific altruism)– Watch out for loss to follow-up (they may represent a certain

category of patients)

28

The TB quarterly "cohort"

• Pro- or retrospective cohort study

• (Nested) case-control study

Alhocol addiction

No addiction

Default?/Cure?

Default?/Cure?

Cases ControlsDefault Cured

Alhocol addiction

No addictionInformation may be available from start of treatment

Pro- Retro-

29

2 x 2 table

exposed(unprotected)

not exposed(protected)

exposed(unprotected)

not exposed(protected)

HEALTHY

HEALTHY

DISEASED

DISEASED

ill Not ill

exposed

not exposed

30

Cohort study

ill healthy

exposed

not exposed

time

31

Case control study

ill healthy

exposed

not exposed

time

32

Analytical study: experimental / intervention study

• Prospective• Use of a cohort• Exposure is usually an intervention, a

drug or vaccine• Patients are ideally randomized which

guarantees minimisation of bias• Example: IPT intervention study in South

African gold miners; recruitment in random sequence; comparison before / after IPT phase

33

Steps for a OR protocol (1)

• Starts with a problem or question: e.g. "Why is there no decline in urban TB in Japan?"

• Gathering of information: – Analyse exhaustively routinely collected (surveillance)

data and disaggregate also by province etc – Talk with stakeholders – Investigation of the literature– Contact other countries

• Develop a hypothesis• Depending on money and staff available:

generate a protocol; but this can also be used to generate money and staff

34

Steps for a OR protocol (2)

• Writing of the protocol: – You can structure it similar to a scientific paper– Introduction/rationale– Objective– Methods (study type, sample size, case definitions used,

inclusion/exclusion criteria, training, data collection, data entry (double entry?, data validation), quality control, lab methods, method of analysis)

– Ethical considerations– Results: shell tables, expected figures– Timeline– Budget– Appendices (questionnaire, maps, consent form...)– Good idea to do a pilot: feasibility, cost, first crude data

verify sample size assumptions

35

Now it is up to you