march 20131 back to basics, 2013 population health (1): epidemiology methods, critical appraisal,...

94
March 2013 1 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology & Community Medicine Other resources available on Individual & Population Health web site

Upload: johnathan-hamilton

Post on 27-Dec-2015

216 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 1

Back to Basics, 2013POPULATION HEALTH (1):

Epidemiology Methods, Critical Appraisal,

Biostatistical Methods

N. Birkett, MDEpidemiology & Community Medicine

Other resources available on Individual & Population Health web site

Page 2: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 2

THE PLAN (1)

• Session 1 (March 18, 1300-1700)– Diagnostic tests

• Sensitivity, specificity, validity, PPV

– Critical Appraisal– Intro to Biostatistics– Brief overview of epidemiological research

methods

Page 3: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 3

THE PLAN (2)

• Aim to spend about 2.5-3 hours on lectures– Review MCQs in remaining time

• A 10 minute break about half-way through• You can interrupt for questions, etc. if

things aren’t clear.– Goal is to help you, not to cover a fixed

curriculum.

Page 4: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

4March 2013

INVESTIGATIONS (1)

• 78.2– Determine the reliability and predictive value

of common investigations– Applicable to both screening and diagnostic

tests.

Page 5: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 5

Reliability

• = reproducibility. Does it produce the same result every time?

• Related to chance error

• Averages out in the long run, but in patient care you hope to do a test only once; therefore, you need a reliable test

Page 6: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 6

Validity

• Whether it measures what it purports to measure in long run– is a disease present (or absent)

• Normally use criterion validity, comparing test results to a gold standard

• Link to SIM web on validity

Page 7: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 7

Reliability and Validity: the metaphor of target shooting. Here, reliability is represented by consistency, and validity by aim

Reliability Low High

Low

Validity

High

••

• •

••

•••

•••

•• ••••

Page 8: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 8

Test Properties (1)Diseased Not diseased

Test +ve 90 5 95

Test -ve 10 95 105

100 100 200

True positives False positives

False negatives True negatives

Page 9: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 9

Test Properties (2)

Diseased Not diseased

Test +ve 90 5 95

Test -ve 10 95 105

100 100 200

Sensitivity = 0.90 Specificity = 0.95

Page 10: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 10

2x2 Table for Testing a Test

Gold standard

Disease Disease

Present Absent

Test Positive a (TP) b (FP)

Test Negative c (FN) d (TN)

SensitivitySpecificity

= a/(a+c) = d/(b+d)

Page 11: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 11

Test Properties (6)• Sensitivity =Pr(test positive in a person

with disease)• Specificity = Pr(test negative in a person

without disease)• Range: 0 to 1

– > 0.9: Excellent– 0.8-0.9: Not bad– 0.7-0.8: So-so– < 0.7: Poor

Page 12: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 12

Test Properties (7)

• Sensitivity and Specificity

– Values depend on cutoff point between normal/abnormal

– Generally, high sensitivity is associated with low specificity and vice-versa.

– Not affected by prevalence, if ‘case-mix’ is constant

• Do you want a test to have high sensitivity or high specificity?

– Depends on cost of ‘false positive’ and ‘false negative’ cases

– PKU – one false negative is a disaster

– Ottawa Ankle Rules: insisted on sensitivity of 1.00

Page 13: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 13

Test Properties (8)

• Sens/Spec not directly useful to clinician, who knows only the test result

• Patients don’t ask:– “If I’ve got the disease, how likely is a positive

test?”

• They ask:– “My test is positive. Does that mean I have the

disease?”

• → Predictive values.

Page 14: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 14

Predictive Values

• Based on rows, not columns– PPV = a/(a+b); interprets positive test

– NPV = d/(c+d); interprets negative test

• Depend upon prevalence of disease, so must be determined for each clinical setting

• Immediately useful to clinician: they provide the probability that the patient has the disease

Page 15: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 15

Test Properties (9)Diseased Not diseased

Test +ve 90 5 95

Test -ve 10 95 105

100 100 200

PPV = 0.95

NPV = 0.90

Page 16: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 16

2x2 Table for Testing a Test

Gold standard

Disease Disease

Present Absent

Test + a (TP) b (FP) PPV = a/(a+b)

Test - c (FN) d (TN) NPV= d/(c+d)

a+c b+d N

Page 17: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 17

Prevalence of Disease

• Is your best guess about the probability that the patient has the disease, before you do the test

• Also known as Pretest Probability of Disease

• (a+c)/N in 2x2 table• Is closely related to Pre-test odds of disease:

(a+c)/(b+d)

Page 18: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 18

Test Properties (10)Diseased Not diseased

Test +ve a b a+b

Test -ve c d c+d

a+c b+d a+b+c+d =N

Prevalence odds

Prevalence proportion

Page 19: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 19

Prevalence and Predictive Values

• Predictive values of a test are dependent on the pre-test prevalence of the disease– Tertiary hospitals see more pathology then FP’s

• Their positive tests are more often true positives.

• How do you determine how useful a test is in a different patient setting?

Page 20: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 20

Prevalence and Predictive Values

• Often called ‘calibrating’a test for use– Relies on the stability of sensitivity &

specificity across populations.

– Allows us to estimate what the PPV and NPOV would be in a new population.

Page 21: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 21

Methods for Calibrating a Test

Four methods can be used:– Apply definitive test to a consecutive series of patients

• rarely feasible, especially during the LMCCs

– Hypothetical table• Assume the new population has 10,000 people• Fill in the cells based on the prevalence, sensitivity and specificity

– Bayes’s Theorem (Likelihood Ratio)– Nomogram

• only useful if you have access to the nomogram

• You need to be able to do one of the last 3. • By far the easiest is using a hypothetical table.

Page 22: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 22

Calibration by hypothetical table

Fill cells in following order:

“Truth”

Disease Disease Total PV

Present Absent

Test Pos 4th 7th 8th 10th

Test Neg 5th 6th 9th 11th

Total 2nd 3rd 10,000 (1st)

Page 23: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 23

Test Properties (11)

Diseased Not diseased

Test +ve 450 25 475

Test -ve 50 475 525

500 500 1,000

Tertiary care: research study. Prev=0.5

PPV = 0.89

Sens = 0.90 Spec = 0.95

Page 24: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 24

Test Properties (12)

Diseased Not diseased

Test +ve

Test -ve

10,000

Primary care: Prev=0.01

PPV = 0.1538

9,900

90

10

100

495

9,405

585

9,415

Sens = 0.90 Spec = 0.95

Page 25: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 25

Calibration by Bayes’ Theorem

• You don’t need to learn Bayes’ theorem• Instead, work with the Likelihood Ratio (+ve)

– Equivalent process exists for Likelihood Ratio (–ve), but we shall not calculate it here

• Consider the following table (from a research study)– How do the ‘odds’ of having the disease change

once you get a positive test result?

Page 26: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 26

Test Properties (13)Diseased Not

diseased

Test +ve

90 5 95

Test -ve

10 95 105

100 100 200 Pre-test odds = 1.00

Post-test odds (+ve) = 18.0

Odds (after +ve test) are 18-times higher than the odds before you had the test. This is the LIKELIHOOD RATIO.

Page 27: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 27

Calibration by Bayes’s Theorem

• LR(+) is fixed across populations just like sensitivity & specificity.– Bigger is better.

• Likelihood ratios are related to sens & specLR(+) = sens/(1-spec)

• Sometime given as the definition or the LR(+)– obscures what is really going on

Page 28: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 28

Calibration by Bayes’s Theorem

• How does this help?• Remember:

– Post-test odds(+) = pretest odds * LR(+)– And, the LR(+) is ‘fixed’ across populations

• To ‘calibrate’ your test for a new population:– Use the LR(+) value from the reference source– Estimate the pre-test odds for your population– Compute the post-test odds– Convert to post-test probability to get PPV

Page 29: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 29

Converting between odds & probabilities

• if prevalence = 0.20, then • pre-test odds = .20/0.80 = 0.25

• if post-test odds = 0.25, then • PPV = .25/1.25 = 0.20

Page 30: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 30

Example of Bayes' Theorem(‘new’ prevalence 1%, sens 90%, spec 95%)

• Compare to the ‘hypothetical table’ method (PPV=15.38%)

Page 31: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 31

Calibration with Nomogram

• Graphical approach which avoids arithmetic• Scaled to work directly with probabilities

– no need to convert to odds• Draw line from pretest probability

(=prevalence) through likelihood ratio– extend to estimate posttest probabilities

• Only useful if someone gives you the nomogram!

Page 32: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

32April 2011 32

Example of Nomogram (pretest probability 1%, LR+ 18, LR– 0.105)

Pretest Prob. LR Posttest Prob.

1%

18

.105

15%

0.01%

March 2013

Page 33: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 33

Are sens & spec really constant?

• Generally, assumed to be constant. BUT…..• Sensitivity and specificity usually vary with case

mix (severity of disease)– May vary with age and sex

• Therefore, you can use sensitivity and specificity only if they were determined on patients similar to your own

• Risk of spectrum bias (populations may come from different points along the spectrum of disease)

Page 34: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

Cautionary Tale #1: Data Sources

March 2013 34

The Government is extremely fond of amassinggreat quantities of statistics. These are raised to the nth degree, the cube roots are extracted, and

the results are arranged into elaborate and impressive displays. What must be kept ever in

mind, however, is that in every case, the figures are first put down by a village watchman, and he puts

down anything he damn well pleases!Sir Josiah Stamp,

Her Majesty’s Collector of Internal Revenue.

Page 35: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 35

78.2: CRITICAL APPRAISAL (1)

• “Evaluate scientific literature in order to critically assess the benefits and risks of current and proposed methods of investigation, treatment and prevention of illness”

• UTMCCQE does not present hierarchy of evidence (e.g., as used by Task Force on Preventive Health Services)

Page 36: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 36

Hierarchy of evidence(lowest to highest quality, approximately)

• Systematic reviews• Experimental (Randomized)• Quasi-experimental• Prospective Cohort• Historical Cohort• Case-Control• Cross-sectional• Ecological (for individual-level exposures)• Case report/series• Expert opinion

} similar/identical

Page 37: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

Cautionary Tale #2: Analysis

March 2013 37

Consider a precise number: the normal body temperature of 98.6°F. Recent investigations involving millions of measurements have shown that this number is wrong: normal body temperature is actually 98.2°F. The fault lies not with the original measurements - they were averaged and sensibly rounded to the nearest degree: 37°C. When this was converted to Fahrenheit, however, the rounding was forgotten and 98.6 was taken as accurate to the nearest tenth of a degree.

Page 38: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 38

BIOSTATISTICS Core concepts (1)

• Sample: – A group of people, animals, etc. which is used to represent

a larger ‘target’ population.• Best is a random sample• Most common is a convenience sample.

– Subject to strong risk of bias.

• Sample size: – the number of units in the sample

• Much of statistics concerns how samples relate to the population or to each other.

Page 39: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 39

BIOSTATISTICS Core concepts (2)

• Mean: – average value. Measures the ‘centre’ of the data. Will be roughly

in the middle.

• Median: – The middle value: 50% above and 50% below. Used when data

is skewed.

• Variance: – A measure of how spread out the data are.

– Defined by subtracting the mean from each observation, squaring,

adding them all up and dividing by the number of observations.

Page 40: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

40March 2013

Page 41: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 41

BIOSTATISTICS Core concepts (2)

• Standard deviation: – square root of the variance.

Page 42: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 42

BIOSTATISTICS Core concepts (3)

• Standard error (of the mean):– Standard deviation looks at the variation of the data in individuals

– We usually study samples.• Select 10 people

• measure BMI

• take the group average

– Repeat many times.• Each time, we get a mean of the sample

– What is the distribution of these means?

– Will be ‘normal’, ‘Gaussian’ or ‘Bell curve’

– Mean of the means• same as population mean

– Variance of the means is• smaller.

Page 43: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 43

BIOSTATISTICS Core concepts (4)

• Standard error (of the mean):

• Confidence Interval:

– A range of numbers which tells us where we believe the correct answer lies. • For a 95% confidence interval, we are 95% sure that

the true value lies inside the interval.– Usually computed as: mean ± 2 SE

Page 44: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 44

Example of Confidence Interval

• If sample mean is 80, standard deviation is 20, and sample size is 25 then:

• We can be 95% confident that the true mean lies within the range:

80 ± (2*4) = (72, 88).

Page 45: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 45

Example of Confidence Interval

• If the sample size were 100, then

– 95% confidence interval is 80 ± (2*2) = (76, 84).

– More precise.

Page 46: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 46

Core concepts (4)

• Random Variation (chance): – every time we measure anything, errors will

occur. – Any sample will include people with values

different from the mean, just by chance.– These are random factors which affect the

precision (SD) of our data but not the validity.– Statistics and bigger sample sizes can help here.

Page 47: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 47

Core concepts (5)

• Bias: – A systematic factor which causes two groups to

differ. • A study uses a two section measuring scale for

height which was incorrectly assembled (with a 1” gap between the upper and lower section).

• Over-estimates height by 1” (a bias).

– Bigger numbers and statistics don’t help much; you need good design instead.

Page 48: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 48

BIOSTATISTICSInferential Statistics

• Draws inferences about populations, based on samples from those populations. – Inferences are valid only if samples are representative

(to avoid bias).• Polls, surveys, etc. use inferential statistics to infer

what the population thinks based on talking to a few people.

• RCTs use them to infer treatment effects, etc.• 95% confidence intervals are a very common way

to present these results.

Page 49: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

49

An experiment (1)

• Here is a ‘fair’ coin• I will toss it to generate some data (heads or

tails)– Write the sequence on the board

March 2013

Page 50: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

50

An experiment (2)

• At some point, you get suspicious– the number of ‘heads’ in a row exceeds what is

reasonable.

• This is the core of hypothesis testing

March 2013

Page 51: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

51

An experiment (3)• Start with a theory

– Null Hypothesis• My coin is ‘fair’

• Generate some data

• Check to see if the data is consistent with the theory.– if the data is ‘unlikely’, then reject the theory or null

hypothesis.

• Statistics just puts a mathematical overlay on top of

this intuitive approach

March 2013

Page 52: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 52

Hypothesis Testing (1)• Used to compare two or more groups.

– We first assume that the two groups have the same outcome results.• null hypothesis (H0)

– Generate some data– From the data, compute some number (a statistic)

• Under this null hypothesis (H0), this should be ‘0’.

– Compare the value I get to ‘0’.• If it is ‘too large’, we can conclude that our assumption (null

hypothesis) is unlikely to be true (reject the null hypothesis).

Page 53: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 53

Hypothesis Testing (2)

• Quantity the extent of our discomfort with the statistic through the p-value.– If the null hypothesis were true, how likely it

that our statistic would be as big as we saw (or bigger).

• Reject H0 if the p-value is ‘too small’

• What is ‘too small’?– arbitrary.– tradition sets it at < 0.05

Page 54: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 54

Example of significance test

• Is there an association between sex and smoking: – 35 of 100 men smoke but only 20 of 100 women smoke

• Calculate the chi-square (the statistic)– = 5.64.

– If there is no effect of sex on smoking (the null hypothesis), a chi-

square value as large as 5.64 would occur only 1.8% of the time.• P=0.018

– Instead of computing the p-value, could compare your statistic to

the ‘critical value’• The value of the Chi-square which gives p=0.05 is 3.84

• Since 5.64 > 3.84, we conclude that p<0.05

Page 55: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 55

Hypothesis Testing (3)• Common methods used are:

– T-test– Z-test– Chi-square test– ANOVA

• Approach can be extended through the use of regression models– Linear regression

• Toronto notes are wrong in saying this relates 2 variables. It can relate many independent variables to one dependent variable.

– Logistic regression– Cox models

Page 56: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 56

Hypothesis Testing (4)

• Once you select a method for hypothesis testing,

interpretation involves:– Type 1 error (alpha)

– Type 2 error (beta)

– P-value• Essentially the alpha value

– Power• Related to type 2 error (Beta)

Page 57: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 57

Hypothesis testing (5)

No effect Effect

No effect No error Type 2 error (β)

Effect Type 1 error (α)

No error

Actual Situation

Results of Stats Analysis

Page 58: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 58

Hypothesis Testing (7)• Statistical Power:

– ‘Easy’ to show that a drug increases survival by 10 times– ‘Hard’ to show that a drug increase survival by 1.2 times– More likely to ‘miss’ the small effect than the large effect– Statistical Power is:

• The chance you will find a difference between groups when there really is a difference (of a given amount).

• Basically, this is 1-β

– Power depends on how big a difference you consider to be important

Page 59: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 59

How to improve your power?

• Increase sample size• Improve precision of the measurement tools

used (reduces standard deviation)• Use better statistical methods• Use better designs• Reduce bias

Page 60: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

Cautionary Tale #3: Anecdotes

March 2013 60

Laboratory and anecdotal clinical evidence suggest that some common non-antineoplastic drugs may affect the course of cancer. The authors present two cases that appear to be consistent with such a possibility: that of a 63-year-old woman in whom a high-grade angiosarcoma of the forehead improved after discontinuation of lithium therapy and then progressed rapidly when treatment with carbamezepine was started, and that of a 74-year-old woman with metastatic adenocarcinoma of the colon which regressed when self-treatment with a non-prescription decongestant preparation containing antihistamine was discontinued. The authors suggest ...... ‘that consideration be given to discontinuing all nonessential medications for patients with cancer.’

Page 61: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 61

Epidemiology overview

• Key study designs to examine (SIM web link)

– Case-control– Cohort– Randomized Controlled Trial (RCT)

• Confounding• Relative Risks/odds ratios

– All ratio measures have the same interpretation• 1.0 = no effect• < 1.0 protective effect• > 1.0 increased risk

– Values over 2.0 are of strong interest

Page 62: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 62

The Epidemiological Triad

Host Agent

Environment

Page 63: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 63

Terminology

• Prevalence: – The probability that a person has the outcome of

interest today. Relates to existing cases of disease. Useful for measuring burden of illness.

• Incidence: – The probability (chance) that someone without

the outcome will develop it over a fixed period of time. Relates to new cases of disease. Useful for studying causes of illness.

Page 64: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 64

Prevalence

• On July 1, 2007, 140 graduates from the U. of O. medical school start working as interns.

• Of this group, 100 had insomnia the night before.

• Therefore, the prevalence of insomnia is:

100/140 = 0.72 = 72%

Page 65: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 65

Incidence Proportion (risk)• On July 1, 2007, 140 graduates from the U.

of O. medical school start working as interns.

• Over the next year, 30 develop a stomach ulcer.

• Therefore, the incidence proportion (risk) of an ulcer in the first year post-graduation is:

30/140 = 0.21 = 214/1,000 over 1 yr

Page 66: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 66

Incidence Rate (1)

• Incidence rate is the ‘speed’ with which people get ill.

• Everyone dies (eventually). It is better to die later

death rate is lower.• Compute with person-time denominator:

PT = # people * duration of follow-up

Page 67: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 67

Incidence rate (2)• 140 U. of O. medical students were

followed during their residency– 50 did 2 years of residency– 90 did 4 years of residency– Person-time = 50 * 2 + 90 * 4 = 460 PY’s

• During follow-up, 30 developed ‘stress’.• Incidence rate of stress is:

Page 68: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 68

Prevalence & incidence

• As long as conditions are ‘stable’ and disease is fairly rare, we have this relationship:

That is,

Prevalence ≈ Incidence rate * average disease duration

Page 69: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 69

Cohort study (1)

• Select non-diseased subjects based on their exposure status

• Main method used:

• Select a group of people with the exposure of interest

• Select a group of people without the exposure

• Can also simply select a group of people without the disease and study a

range of exposures.

• Follow the group to determine what happens to them.

• Compare the incidence of the disease in exposed and unexposed

people

• If exposure increases risk, there should be more cases in exposed subjects

than unexposed subjects

• Compute a relative risk.

• Framingham Study is standard example.

Page 70: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 70

Exposed group

Unexposedgroup

No disease

Disease

No disease

Disease

time

Study begins Outcomes

Page 71: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 71

Cohort study (2)

YES NO

YES a b a+b

NO c d c+d

a+c b+d N

Disease

Exp

RISK RATIO

Risk in exposed: =

Risk in Non-exposed =

If exposure increases risk, you would expect

to be larger than . How much

larger can be assessed by the ratio of one

to the other:

Page 72: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 72

Cohort study (3)

YES NO

Yes 42 80 122

No 43 302 345

85 382 467

Death

Exposure

Risk in exposed: = 42/122 = 0.344Risk in Non-exposed = 43/345 = 0.125

Page 73: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 73

Cohort study (4)

• Historical cohort study

• Recruit subjects sometime in the past

• Follow-up to the present

• Usually use administrative records

• Can continue to follow into the future

• Example: cancer in Gulf War Vets

• Identify soldiers deployed to Gulf in 1991

• Identify soldiers not deployed to Gulf in 1991

• Compare development of cancer from 1991 to 2010

Page 74: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 74

Case-control study (1)• Select subject based on their final outcome.

– Select a group of people with the outcome/disease (cases)

– Select a group of people without the outcome (controls)

– Ask them about past exposures– Compare the frequency of exposure in the two groups

• If exposure increases risk, there should be more exposed cases than exposed controls

– Compute an Odds Ratio– Under many conditions, OR ≈ RR

Page 75: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 75

Disease(cases)

No disease(controls)

Exposed

Unexposed

Exposed

Unexposed

The study begins by selecting

subjects based on

Reviewrecords

Reviewrecords

Page 76: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 76

Case-control study (2)

YES NO

YES a b a+b

NO c d c+d

a+c b+d N

Disease?

Exp?

ODDs RATIO

Odds of exposure in cases =

Odds of exposure in controls =

If exposure increases risk, you would to find more

exposed cases than exposed controls. That is, the

odds of exposure for cases would be higher

This can be assessed by the ratio of one to the

other:

Page 77: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 77

Yes No

Yes 42 18

No 43 67

85 85

Exposure

Odds of exp in cases: = 42/43 = 0.977Odds of exp in controls: = 18/67 = 0.269

Case-control study (3)Death

Page 78: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 78

Randomized Controlled Trials

• Basically a cohort study where the researcher decides which exposure (treatment) the subject get.– Recruit a group of people meeting pre-specified

eligibility criteria.– Randomly assign some subjects (usually 50% of them)

to get the control treatment and the rest to get the experimental treatment.

– Follow-up the subjects to determine the risk of the outcome in both groups.

– Compute a relative risk or otherwise compare the groups.

Page 79: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 79

Randomized Controlled Trials (2)

• Some key design features– Allocation concealment

– Blinding (masking)• Patient

• Treatment team

• Outcome assessor

• Statistician

– Monitoring committee

• Two key problems– Contamination

• Control group gets the new treatment

– Co-intervention• Some people get treatments other than those under study

Page 80: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

• Number needed to treat, NNT (to prevent one adverse event) =

March 2013 80

Randomized Controlled Trials: Analysis

• Outcome is often an adverse event– RR is expected to be <1

• Absolute risk reduction

• Relative risk reduction

Page 81: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 81

RCT – Example of Analysis

Asthma No Total Incid

attack attack

Treatment 15 35 50 .30

Control 25 25 50 .50

Relative Risk = 0.30/0.50 = 0.60

Absolute Risk Reduction = 0.50-0.30 = 0.20

Relative Risk Reduction = 0.20/0.50 = 40%

Number Needed to Treat = 1/0.20 = 5

Page 82: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 82

Confounding

• Interest is in the effect of an exposure on an outcome– Does alcohol drinking cause oral cancer?

• BUT, the effect of alcohol is ‘mixed up’ with the effect of smoking.

• The effect of this third factor ‘confounds’ the relationship we are

interested in.– Produces a biased results.

– Can make result more or less strong

• Confounder is an extraneous factor which is associated with both

exposure and outcome, and is not an intermediate step in causal

pathway

• Proper statistical analysis must adjust for the confounder.

Page 83: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 83

The Confounding Triangle

Exposure Outcome

Confounder

Causal

Association

Page 84: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 84

Confounding (example)• Does heavy alcohol drinking cause mouth cancer?

– Do a case-control study

– OR=3.4 (95% CI: 2.1-4.8).

• BUT– Smoking causes mouth cancer

– Heavy drinkers tend to be heavy smokers.

– Smoking is not part of causal pathway for alcohol.

• Therefore, we have confounding.

• We do a statistical adjustment (logistic regression is most common):

– OR=1.3 (95% CI: 0.92-1.83)

Page 85: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 85

Standardization• An method of adjusting for confounding (usually used for

differences in age between two populations)

• Refers observed events to a standard population, producing

hypothetical values

• Direct:– yields age-standardized rate (ASMR)

• Indirect:– yields standardized mortality ratio (SMR)

• You don’t need to know how to do this

• Nearly always used when presenting population rates and trends.

Page 86: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 86

Mortality dataThree ways to summarize them

• Mortality rates– crude

• Overall all-cause mortality rate

– specific• mortality rate for a specific group (men), disease (lung cancer), etc.

– standardized• Mortality rate adjustment to take account of the aging population

Page 87: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 87

Mortality dataThree ways to summarize them

• Life expectancy: – average age at death if current mortality rates

continue. Derived from a life table.

• Potential Years of Life Lost (PYLL): – subtract age at death from some “acceptable” age of

death.– Sum up over a group

• estimates ‘potential’ lost due to early death• Places more emphasis on causes that kill at younger ages.

Page 88: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 88

0 100 200 300 400 500

HIV/AIDS

Respiratory disese

Suicide and violence

Unintentional injury

Circulatory disease

Cancer

Mortality rate (per 100,000) PYLL (000)

Impact of different causes of death in Canada 2001: Mortality rates and PYLL

Source: Statistics Canada

Page 89: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 89

Summary measuresof population health

• Mortality is a ‘crude’ measure of population health

• Need to consider– morbidity

– quality of life

– disability

– and so on.

• Many other measures have been developed

• Quality Adjusted Life Years (QALYs)– Years lived are weighted according to quality of life, disability, etc.

• Two ‘classes’ of these types of measures:– Health expectancies point up from zero

– Health gaps point down from ideal

Page 90: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 90

Attributable Risks (1) (SIM web link)

• Would like to know the amount of a disease which might be prevented if we eliminate a risk– Gives an upper limit on amount of preventable disease.– Meaningful only if association is causal.

• Tricky area since there are several measures with similar names.– Attributable risk– Attributable fraction– Population Attributable Risk– and so on

Page 91: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 91

Attributable Risks (1) (SIM web link)

• Two main targets for these measures• The amount of disease due to exposure in

the exposed subjects. The same as the risk difference.

• The proportion of risk attributed to the exposure in the general population – depends on

• Risk due to exposure• How common the exposure is.

Page 92: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 92

Attributable risks (2)

ExpUnexp

Risk Difference or Attributable Risk

Iexp

Iunexp

RD = AR = Iexp - Iunexp

Page 93: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

March 2013 93

Attributable risks (2)

ExpUnexp

PopulationAttributable Risk

Iexp

Iunexp

Ipop

Population

Page 94: March 20131 Back to Basics, 2013 POPULATION HEALTH (1): Epidemiology Methods, Critical Appraisal, Biostatistical Methods N. Birkett, MD Epidemiology &

94March 2013