Session 6: Basic Statistics Part 1(and how not to be frightened by the word)

Prof Neville YeomansDirector of Research, Austin LifeSciences

So now we’ve got some results? How can we

make sense out of them?

What I will cover(over 2 sessions, August 28 and September 25)

• Sampling populations• Describing the data in the samples• How accurately do those data reflect the ‘real’

population the samples were taken from?• We’ve compared two groups. Are they really from

different populations, or are they samples from the same population and the measured differences are just due to chance?

What I will cover (contd.)• Tests to answer the question ‘Are the differences

likely to be just due to chance?’– Data consisting of values (e.g. hemoglobin

concentration)(‘continuous variables’)– Data consisting of whole numbers – frequencies,

proportions (percentages)– Tests for just two groups; tests for multiple groups

• Tests that examine relationships between two or more variables (correlation, regression analysis, life-table)

What I will cover (contd.)

• How many subjects should I study to find the answer to my question? (Power calculations)

• Statistical packages and other resources

We’ve got some numbers. How are we going to describe them to others?

Suppose we’ve measured heights of a number of females (‘a sample’) picked off the street in Heidelberg

Subject #

Height (cm)

1 179

2 162

3 150

4 155

5 168

6 175

7 159

8 152

Subject #

Height (cm)

9 156

10 157

11 161

12 164

13 165

14 159

15 161

16 163

Subject #

Height (cm)

17 167

18 173

19 159

20 170

21 168

22 162

23 171

24 164

Heights of (sample of) Heidelberg women

Person number

0 2 4 6 8 10 12 14 16 18 20 22 24 26

Hei

ght (

cm)

145

150

155

160

165

170

175

180

185

How could we more concisely describe these data – using just one or two numbers that would give us useful information about the sample as a whole?

1. A measure of ‘central tendency’2. A measure of how widely the values are spread

Frequency of women with each height in sample

Height

145 150 155 160 165 170 175 180 185

Freq

uenc

y

0.5

1.0

1.5

2.0

2.5

3.0

3.5The median (middle value) = 162.5

The range (150-179)a poor measure for describing the whole population because it depends on sample size – range is likely to be wider with larger samples

Interquartile range (25th percentile to 75th

percentile of values: 159-168)

what we should always use with the median – it’s largely independent of sample size

Frequency distribution of height of Heidelberg women

Height

130 140 150 160 170 180 190 200

Freq

uenc

y

0

1

2

3

4

5

6 the Mean (average)(Ʃx/N) = 163.3 cm

the Standard Deviation*± 7.2 cm

*In Excel, enter formula ‘=STDEV(range of cells)’

- doesn’t vary much with sample size (except very small samples)

- approx. 67% of values will lie within ± 1 SD either side of mean**

- approx 95% of values will lie within ± 2 SD either side of mean**

(amalgamated into 3cm ranges: e.g.141-143, 144-146 etc.)

** Provided the population is ‘normally distributed’

The ‘Normal distribution’Mean = Median in a ‘perfect’normal distribution

Standard deviations away from mean

We measured the mean height of our sample of 25 women ... (it was 163.3 cm)

• But what is the average height of the whole population – of ALL Heidelberg women?

• We didn’t have time or resources to track them all down – that’s why we just took what we hoped was a representative sample.

• What I’m asking is: how good an estimate of the true population mean is our sample mean?

• This is where the Standard Error of the Mean* (or just Standard Error, SE) comes in.

*It’s sometimes called the Standard ESTIMATE of the Error of the mean

The Standard Error (contd.)• The mean height of our sample of 25 women was 163.3 cm• We calculated the Standard Deviation (SD) of the sample to

be 7.3 cm (that value, on either side of the mean, that should contain about 2/3 of those measured)

• Standard Error of the mean = SD/√N , i.e. 7.3/ √25 = 7.3/5 = 1.46

• So now we can express our results for the height of our sample as 163.3 ± 1.5 (Mean ± SEM) ...... But what does this really tell us?

• The actual true mean height of the whole population of women has a 67% likelihood of lying within 1.5 cm (i.e. 1 SEM) either side of the mean we found in our sample; and a 95% likelihood of lying within 3 cm (i.e. 2 x SEM) either side of that sample mean. (It’s actually 1.96xSEM for a reasonably large sample - e.g. roughly 30 – and wider for small samples, but let’s keep it simple).

The concept of the standard error of the mean (SEM) – e.g. serum sodium values

130 134 138 142 146 150 154 mmol/L

True population mean142 mmol/L (SD=4.0)

Sample mean =142.8 mmol/L

(SD of sample = 3.4)

1 x SEM = (i.e. 3.4/√10) = 1.1 mmol/L

2 x SEM (~’95% confidence interval’) = 2.2 mmol/LRandom sample of 10

normal individuals

That means: ‘There is a 95% chance (19 chances out of 20) thatthe actual population mean, estimated from our random samplelies between 140.6 and 145.0 mmol/L)’

Why does Standard Error depend on population SD and sample size?

SE = SD/√N

A: Narrow populationspread (i.e. small SD)

B: Wide populationspread (i.e. large SD)

Increasing N decreases SE of mean.i.e. increases accuracy of our estimate of the population meanbased on results of our sample

Testing significance of differencesSamples of Heidelberg men and women

Height (cm)

130 140 150 160 170 180 190 200

Freq

uenc

y

0

1

2

3

4

5

6

7

Women:Mean = 163.3SE = 1.46

Men:Mean = 176.5SE = 1.43

On a quick, rough, check we can see that:(a) the 95% confidence interval for our estimate of the height of women is160.3-166.4 cm (approximately mean ± 2SE).(b) our estimate of the mean height of the men sampled is quite a lot outsidethe 95% confidence interval (range)for the women, so it looks improbable thatthey are from the same population

95% confidence intervals

Testing significance of differences ....How likely is it that the two random samples

came from the same population?Student’s t-test

Standard deviation either side of mean

Composite frequency distribution, created by pooling data from both samples

Mean: 170.7SD: 10.0 cm

0 321-1-2-3

Women

Men

How likely is it that these two samples (the pink and the blue) were taken from the SAME population? [this is called the NULL HYPOTHESIS]

Tested for statistical significance of difference: p<0.001i.e. there is less than 1 chance in 1000 that these two samplescame from the same population

In fact, though, running a Student’st-test on the two samples of height in the Heidelberg men and women slide, gives this error message:

Assumptions to be met before testing significance of differences with PARAMETRIC TESTS* – i.e. tests that use the mathematics of the normal curve distribution• The combined data should approximate a normal

curve distribution– in this instance the male data were skewed (not

evenly distributed around the mean) and spread a bit too far out into the tails of the frequency-distribution curve

• The variances (=SD2) of the groups should not differ significantly from each other

*Student’s t-test, Paired t-test, Analysis of Variance

An example of data where groups have different variance (spread), and one group is skewed

Means

Airway obstruction - controls vs smokers

Controls Smokers

FEV1

/FVC

30

40

50

60

70

80

90

100

Means

Medians

The equal variance test failed (P<0.05), and the normality test almostfailed (P=0.08) – so we should not use a parametric test such as t-test

Lower limit of ‘normal’ range

So what do we do if we can’t use a parametric test to check for significances of differences?

• Use a non-parametric test• These tests, instead of using the actual numerical

values of the data, put the data from each group into ascending order and assign a rank number for their place in the combined groups.

• The maths of the test is then done on these ranks• Examples: Rank sum test, Wilcoxon Rank Test, Mann

Whitney rank test, etc.– (the P value for our slide of heights of Heidelberg men and

women was calculated using Wilcoxon test)

Group 1 data

Rank when groups

combinedGroup 2

data

Rank when groups

combined12 1 16 513 2.5 19 6.513 2.5 26 815 4 40 1019 6.5 78 1127 9 101 12

Sum of ranks: 25.5 46.5

Mann-Whitney test: P = 0.026

Example of how a rank-sum (non-parametric) test is constructed manually*

*In reality, these days you’ll just feed the raw data into a program to do it for you

Tests to examine significance of differences between 3 or more groups

• Parametric tests (tests based on the mathematics of the ‘normal curve’)– Analysis of Variance (1-way, 2-way, factorial, etc.)

• Non-parametric tests (rank-sum tests)– Kruskal-Wallis test

[strictly this should read ... ‘tests to decide how likely are data from 3 or more samples to come from the same population’]

Parallel group trial of placebo versus Normopress treatmentfor hypertension

Normopress dose

0 mg 0.5 mg 2.0 mg

Mea

n 24

hou

r blo

od p

ress

ure

0

20

40

60

80

100

120

140Mean +/- SE

Parallel group trial of placebo versus Normopress treatmentfor hypertension

Normopress dose

0 mg 0.5 mg 2.0 mg

Mea

n 24

hou

r blo

od p

ress

ure

0

20

40

60

80

100

120

140Mean +/- SE

One variable (dose), comparedacross 3 groups .... So this gets tested with one-way ANOVA

This tells us that it is very unlikely the three groups belong to the same population..But which differ from which?

Parallel group trial of placebo versus Normopress treatmentfor hypertension

Normopress dose

0 mg 0.5 mg 2.0 mg

Mea

n 24

hou

r blo

od p

ress

ure

0

20

40

60

80

100

120

140Mean +/- SE

One Way Analysis of Variance Thursday, June 28, 2012, 3:39:25 PM

Normality Test: Passed (P = 0.786)Equal Variance Test: Passed (P = 0.694)

Group Name N Missing Mean Std Dev SEMControl 12 0 126.500 7.243 2.091Normopress 0.5 mg 12 0 125.083 5.551 1.602Normopress 2.0 mg 12 0 114.000 5.625 1.624

Source of Variation DF SS MS F P Between Groups 2 1124.389 562.194 14.679 <0.001Residual 33 1263.917 38.301Total 35 2388.306

The differences in the mean values among the treatment groups are greater than would be expected by chance; there is a statistically significant difference (P = <0.001).

All Pairwise Multiple Comparison Procedures (Holm-Sidak method):Overall significance level = 0.05Comparisons for factor: Comparison Diff of Means t Unadjusted P Critical Significant?

LevelControl vs. Normopress 2.0 mg 12.500 4.947 <0.001 0.017 YesNormopress 0.5 mg vs. Normopress 2.0 mg 11.083 4.387 <0.001 0.025 YesControl vs. Normopress 0.5 mg 1.417 0.561 0.579 0.050 No

Before and after data – paired tests

• Create a paired analysis of length of hair after going to hairdresser.

• Hypothesis: cutting hair makes it shorter

Two independent groupsDifference between means tested for significance with

Student’s t test

Group

1 2

Hai

r len

gth

(cm

)

0

5

10

15

20

9.7±1.9*8.2±1.7

*mean ± SE

P=0.58

Our actual data

Group

1 2

Hai

r len

gth

(cm

)

0

5

10

15

20

P=0.025

Difference between means tested for significancewith paired Student’s t test

Before After

The variation within each individual is much less than between individuals

The paired t-test examines the mean and standard error of the changesIn each individual, and tests how likely are the changes due to chance

So far we have been dealing with‘CONTINUOUS VARIABLES’– numbers such as heights, laboratory values, velocities, temperatures etc. that could have any value (e.g. many decimal points) if we could measure accurately enough.

– whole numbers, most often as proportions or percentages.

Now we’ll look at ...‘DISCONTINUOUS VARIABLES’

Rates and proportions

• In 1969, a home for retired pirates has 93 inmates, 42 of whom have only one leg.

• In 2004, a subsequent survey finds there are now 62 inmates, 6 of whom have only one leg.

Has there been a ‘real’ change (i.e. a change unlikely to be due to chance) in the proportion of one-legged pirates in the home between the two surveys?

Year Pirates with 1 leg (%)

Pirates with 2 legs (%)

Total pirates

1969 42 (45.2) 51 (54.8) 93

2004 6 (9.7) 56 (10.3) 62

Totals 48 107 155

Chi-square= 20.282 with 1 degrees of freedom. (P = <0.001)

i.e. The likelihood that the differencein proportions of 1-legged inmates between 1969 and 2004 is due to chance ... is less than 1:1000

Expected

2919

One trap with chi-square tests and small numbers ....

Treatment No. dead No. surviving

Placebo 9 5

Penicillin 1 8

Treatment

No. dead No. surviving

Totals

Placebo 9 5 14

Penicillin 1 8 9

Totals 10 13 23

Fisher’s exact test:

P = 10! x 13! x 14! x 9! = 0.029 9! x 5! x 1! x 8! x 23!

Penicillin treatment for pneumonia

Correlation• Fairly straightforward concept of how likely are two

variables to be related to each other• Examples:

– Do children’s heights vary with their age, and if so is the relation direct (i.e. get bigger as get older) or converse (get smaller as get older)?

– Does respiratory rate increase as pulse rate increases during exertion?

• The correlation coefficient, R, tells us how closely the two variables ‘travel together’

• P value is calculated to tell us how likely the relationship is to be ‘only’ by chance

Examples of regression (correlation) data

Hours of sunlight0 2 4 6 8 10 12

MW

gen

erat

ed

4

6

8

10

12

14

16

18

X Data

0 2 4 6 8 10 12

Y D

ata

4

6

8

10

12

14

16

18

20

22

R = 0.965P<0.0001

R = - 0.37P = 0.30

Some other common statistical analyses

• Life-table analyses – Observing and comparing events developing over

time; allows us to compensate for dropouts at varying times during the study

• Multiple linear and multivariate regression analyses– Looking for relationships between multiple

variables

Life table analyses

Scagliotti et al. J Clin Oncol 2012; 30: 2829

Advanced lung cancer. Trial compared motesanib + 2 conventionalchemo drugs ... with placebo plus the two other drugs

Multiple regression analysis• Examines the possible effect of more than one

variable on the thing we are measuring (the ‘dependent variable’)

Perret JL et al. The Interplay between the Effects of LifetimeAsthma, Smoking, and Atopy on Fixed Airflow Obstruction inMiddle Age. Am J Respir Crit Med 2013; 187: 42-8....from Institute of Breathing and Sleep (Austin),University of Melbourne, Monash University, Alfred Hospital, And others

Perret et al. 2013

Sample Size Calculations

How many patients, subjects, mice etc. do we need to study to reliably* find the answer to our research question?

*We can never be certain to do this, but should aim to be considerably more likely than not to find out the truth about the question

Sample size calculations (1)First we need to grapple with two types of ‘error’ in interpreting differences between means and/or medians of groups:• Type 1 (or α) error: ... that we think the difference is

‘real’ (data are from 2 or more different populations) when it is not– This is what we’ve dealt with so far, and the P-

values assess how likely the differences are due to chance

• Type 2 (or β) error: ... that our experiment, and the stats test we’ll apply to the results, will FAIL to show a significant difference when there REALLY IS ONE

Sample size calculations (2)

• If we end up with a Type 2 () error, it will be because our sample size(s) was too small to persuade us that the actual difference between means was unlikely due to chance (i.e. P<0.05)

• The smaller the real difference between population means, the larger the sample size needs to be to detect it as being statistically significant

Sample size calculations (3)How do we go about it?

Most of the good statistical packages have a function for calculating sample sizes

1. Decide what statistical test will be appropriate to apply to primary endpoint when study completes

2. Estimate the likely size of difference between groups, if the hypothesis is correct

3. Decide how confident you want to be that the difference(s) you observe is unlikely due to chance

4. Decide how much you want to risk missing a true difference (i.e. what power you want the study to have)

Note: We really should have done a sample size calculation before we started ourexperiments, but for this course we needed to deal with the basics of stats tests first

Sample size calculationsA worked example (i)

• We want to see whether drug X will reduce the incidence of peptic ulcer in patients taking aspirin for 6 months

1. Decide what statistical test: chi square, to compare differences in frequencies in 2 groups

Sample size calculationsA worked example (i)

• We want to see whether drug X will reduce the incidence of peptic ulcer in patients taking aspirin for 6 months

• We expect a 10% incidence of ulcers in the controls• We hypothesize that a 50% reduction (i.e. 5%) in

those treated with X would be clinically worthwhile

2. We’ve now decided the size of the difference between groups we are interested to look for

Sample size calculations (4)A worked example (i)

• We want to see whether drug X will reduce the incidence of peptic ulcer in patients taking aspirin for 6 months

• We expect a 10% incidence of ulcers in the controls• We hypothesize that a 50% reduction (i.e. 5%) in

those treated with X would be clinically worthwhile• We decide to be happy with a likelihood of only 1:20

that difference observed is due to chance3. That is to say, we want to set P0.05 as the level of α (alpha) risk (the risk of concluding the difference is real when it’s actually due to chance)

Sample size calculations (4)A worked example (i)

• We want to see whether drug X will reduce the incidence of peptic ulcer in patients taking aspirin for 6 months

• We expect a 10% incidence of ulcers in the controls• We hypothesize that a 50% reduction (i.e. 5%) in

those treated with X would be clinically worthwhile• We decide to be happy with a likelihood of only 1:20

that difference observed is due to chance• We would like to have at least an 80% chance of

finding that 50% reduction (20% of missing it, i.e. of β risk)

4. That is, set Power of the study ≥80% (1-β) to detect such a difference (if it exists)

Sample size calculationsA worked example (i)

Summary of sample size calculation setting:• Estimated ulcer incidence in controls = 10%• Estimated incidence in group receiving drug X

= 5%• For P(α) 0.05, and Power (1-β) ≥80%• Data tested by chi square ......................................................................Calculated required sample size = 449 in each

group

Sample size calculationsA worked example (ii)

• We hypothesize that removing the spleen in rats will result in an increase in haemoglobin (Hb) from the normal mean of 14.0 g/L to 15.0 g/L

• We already know that the SD (Standard deviation) of Hb values in normal rats is 1.2 g/L (if we don’t know we’ll have to guess!)

• Testing will be with Student’s t test• We’ll set α (likelihood observed difference due to

chance) at 0.05• We want a power (1-β) of at least 80% to minimize risk

of missing such a difference if its real*

*More correctly, we should say if the samples really are from different populations

Sample size calculationsA worked example (ii)

Summary of sample size calculation setting:• Control mean = 14.0 g/L; Operated mean =

15.0 g/L• Estimated SD in both groups = 1.2 g/L• For P(α) 0.05, and Power (1-β) ≥80%• Data tested by Student t......................................................................Calculated required sample size = 24 in each

group

Summary of the most common statistical tests in biomedicine (1. Parametric tests)

Test Purpose Comments

Student t-test Compare 2 groups of ‘continuous’ data*

Only use if data are ‘normally distributed’ and variances of groups similar

Paired Student t-test Compare before-after data on the same individuals

The differences (between before-after) need to be ‘normally distributed’More powerful than ‘unpaired’ t-test because less variability within individuals than between them

1- way analysis of variance

Compare 3 or more groups of continuous data

Same requirements as for Studentt-test

2-way analysis of variance

Compare 3 or more groups , stratified for at least 2 variables

As above

*For measured values, not numbers of events (frequencies)

Summary of the most common statistical tests in biomedicine (2. non-parametric)

Test Purpose Comments

Rank-sum or Mann-Whitney test

Compare 2 groups of ‘continuous’ data, using their ranks rather than actual values

Use if t-test invalid because data not ‘normally distributed’ and/or variances of groups significantly different

Signed rank test Rank test to use instead of paired t-test

Use instead of paired t-test if the differences (between before and after) are not ‘normally distributed’

Non-parametric analysis of variance

Compare 3 or more groups of continuous data

As above (it’s the generalised form of Mann-Whitney test when there are >2 groups)

Some tools for statistical analyses• Excel spreadsheets – e.g. If column A contains

data in the 8 cells, A3 through A10– Mean : =average(a3:a10)– SD: =stdev(a3:a10)– SEM: =(stdev(a3:a10))/sqrt(8)

• Common statistical packages for significance testing– Sigmaplot– SPPS (licence can be downloaded from Unimelb)– STATA

Other resources

• Armitage P, Berry G, Matthews JNS. Statistical methods in medical research. 4th edn. Oxford: Blackwell Science, 2002.

• Dawson B, Trapp R. Basic and clinical biostatistics. 4th edn. New York: McGraw Hill 2004. (Electronic book in Unimelb electronic collection)

• Rumsey DJ. Statistics for Dummies. 2nd edn. Oxford: Wiley & sons 2011.