tests of significance ssm

7/30/2019 Tests of Significance Ssm

1/103

Tests of Significance


2/103

Testing Hypotheses

The purpose of a statistical test is to

assess the evidence provided by the data

against some claim about a parameter.

We have two ways to test claims or

hypotheses we make about the

parameter:

1. Confidence Intervals

2. Significance Tests


3/103

Confidence Statements

A confidence statement:

is based on the distribution of values of the

sample statistic that would occur if we took

many SRS

attaches a Margin of Error to a statistic so

that we can make a claim about the

parameter at a certain confidence level


4/103

Test of Significance

Tests of Significance:

are based on the distribution of values of

the sample statistic that would occur if we

took many SRS

use the sampling distribution and standard

scores to tell us how much information we

have against our claim


5/103

We Will Discuss

How to write hypotheses for statistical

problems

How sampling distributions help you to test

the hypothesis

The comparison of Confidence Intervals

and Significance Tests


6/103

Hypothesis

A hypothesis is a statement or claimregarding a characteristic of one or more

populations (Parameter).

Hypotheses are only concerned with the

population.

It can be something we are assuming tobe true or something we are trying to

prove.


7/103

Steps in Hypothesis Testing

A claim is made.

Evidence (sample data) is collected inorder to test the claim.

The data is analyzed in order to support

or refute the claim.


8/103

In 1997, 43% of Americans 18 years or olderparticipated in some form of charity work. A researcher

believes that this percentage different today.

H0: p=43%H1: p 43%

In June, 2001 the mean length of a phone call on a

cellular telephone was 2.62 minutes. A researcherbelieves that the mean length of a call has increased

since then.

Examples of Claims Regarding a Characteristicof a Single Population


9/103

We test these types of claims using

sample data because it is usuallyimpossible or impractical to gain

access to the entire population. If

population data is available, theninferential statistics is not necessary.

CAUTION!


10/103

The null hypothesis, denoted Ho(read H-

naught), is a statement to be tested. The nullhypothesis is assumed true until evidence

indicates otherwise. It is a statement regarding

the value of a population parameter.

The alternative hypothesis, denoted, H1 (readH-one), is a claim to be tested. We are trying to

find evidence for the alternative hypothesis. It is aclaim regarding the value of a population

parameter.


11/103

Three ways to set up the null and alternative

hypothesis.

1. Equal versus not equal hypothesis (two-tailedtest)Ho: parameter = some valueH1: parameter some value


12/103


hypothesis.

1. Equal versus not equal hypothesis (two-tailedtest)

Ho: parameter = some valueH1: parametersome value

2. Equal versus less than (left-tailed test)Ho: parameter = some valueH1: parameter < some value


13/103


hypothesis.

1. Equal versus not equal hypothesis (two-tailedtest)

Ho: parameter = some valueH1: parameter some value

2. Equal versus less than (left-tailed test)Ho: parameter = some valueH1: parameter < some value

3. Equal versus greater than (right-tailed test)Ho: parameter = some valueH1: parameter > some value


14/103

Hypothesis Forms for Proportions

0

00

:

:

ppH

ppH

a

0

00

:

:

ppH

ppH

a

0

00

:

:

ppH

ppH

a


15/103

Hypothesis Forms for Means

0

00

:

:

aH

H

0

00

:

:

aH

H

0

00

:

:

aH

H


16/103

Trying to prove the parameteris less than


17/103

Trying to prove the parameter ismore than


18/103

Trying to prove the parameter isnot equal to or different than


19/103


20/103

In Your Own WordsThe null hypothesis is a statement ofstatus quo or no difference and always

contains a statement of equality. The nullhypothesis is assumed to be true until wehave evidence to the contrary. The claim

that we seek evidence for alwaysbecomes the alternative hypothesis.


21/103

Four Outcomes from Hypothesis Testing1. We could reject Ho when in fact H1 is true. This

would be a correct decision.


22/103


would be a correct decision.2. We could not reject Ho when in fact Ho is true.This would be a correct decision.


23/103



3. We could reject Ho when in fact Ho is true. This

would be an incorrect decision. This type of erroris called a Type I error.


24/103



3. We could reject Ho when in fact Ho is true. This

would be an incorrect decision. This type of erroris called a Type I error.4. We could not reject Ho when in fact H1 is true.This would be an incorrect decision. This type of

error is called a Type II error.


25/103


26/103


27/103

In Your Own Words

As the probability of a Type I errorincreases, the probability of a Type IIerror decreases, and vice-versa.


28/103

CAUTION!


29/103

EXAMPLE Wording the Conclusion

In June, 2001 the mean length of a phone call on acellular telephone was 2.62 minutes. A researcher

believes that the mean length of a call has

increased since then.

(a) Suppose the sample evidence indicates thatthe null hypothesis should be rejected. State the

wording of the conclusion.

(b) Suppose the sample evidence indicates that

the null hypothesis should not be rejected. Statethe wording of the conclusion.


30/103

P- Value

Probability that the test statistic takes a

value more extreme than the oneobserved

Far from what we expect

The smaller the P-value, the strongerthe evidence against H0 provided by the

data


31/103

How do we find the area

under a Normal Curve?


32/103

Standard Scores Recall the formula for standard scores for

observations that are normally distributed

The formula for standard scores is similar,but we are dealing with the distribution of

statistics. Sampling distributions differ from

distribution of data.

deviationstandard

meannobservatio z


33/103

Recall the Sampling Distributions

xofonDistributiSampling pofonDistributiSampling

n

n

pp )1(

S


34/103

Population Proportion Standard

Score

n

pp

pp

z )1(

00

0


35/103

Population Mean Standard

Score

n

x

z

0


36/103

DATA

Qualitative Quantitative

Nominal /Categorical

Ordinal Discrete Continuous

Weight

Blood sugar

Viral Load

Marital status

Occupation

HIV test result

Agreement scale

Disease severity

Educational status

No.of children

No.of eggs

No.of Positives

TYPES OF DATA

STATISTICAL ANALYSIS PLAN


37/103

POPULATION

Sample Size Sample Design

SAMPLE

ESTIMATION TESTING OF HYPOTHESIS MODELS

Qualitative Data Quantitative Data Qualitative Data Quantitative Data Qualitative Data Quantitative Data

Parametric Non parametric

Proportions Mean/Median/GM 2 x 2 Chi square t- test Mann Whitney U Correlation

Percentage Range Fisher exact test Modified t-test Liner Regression

Prevalence Quartiles/Quintiles McNemars Chi square Paired t-test Wilcoxon Signed Rank Logistic SE 95%CI

Incidence Deciles/Percentiles R x C Chi square ANOVA Kruskal-Wallis Regression Multiple Regression

Rates Standard Deviation Trend Chi square

Ratios SE & 95%CI

SE & 95%CI Coefficient of Variation




38/103

POPULATION

Sample Size Sample Design

SAMPLE

ESTIMATION TESTING OF HYPOTHESIS MODELS

Qualitative Data Quantitative Data Qualitative Data Quantitative Data Qualitative Data Quantitative Data

Parametric Non parametric

Proportions Mean/Median/GM 2 x 2 Chi square t- test Mann Whitney U Correlation

Percentage Range Fisher exact test Modified t-test Liner Regression

Prevalence Quartiles/Quintiles McNemars Chi square Paired t-test Wilcoxon Signed Rank Logistic SE 95%CI

Incidence Deciles/Percentiles R x C Chi square ANOVA Kruskal-Wallis Regression Multiple Regression

Rates Standard Deviation Trend Chi square

Ratios SE & 95%CI

SE & 95%CI Coefficient of Variation



39/103

A random sample of n = 25 measurements of chest

circumferences from a population of newborns having

= 0.7 inches provides a sample mean of = 12.6 in. Is

it likely that the population mean has the value = 13.0?

1. The hypothesis: H0: = 13.0 versus H1: 13.0

2. The assumptions: Random sample from a normal

distribution with = 0.7 inches

3. The -level: = 0.05

x

Hypothesis Testing: = 12.6x


40/103

n

xz

86.214.0

4.0

257.0

0.136.12

z

4. The test statistic:

5. The critical region: Reject H0: = 13.0 if the value

calculated for z is not between 1.96

6. The result:

7. The conclusion: Reject H0: = 13.0 since the value

calculated for z is not between 1.96


41/103

Z

0H : = 13.00

~ (13.00,0.14)x N


42/103

This test was performed under the assumption that = 13.0.

Our conclusion is that our sample mean = 12.6 is so faraway from = 13.0 that we find it hard to believe that

= 13.0.

That is, our observed value of = 12.6 for the samplemean is too rare for us to believe that = 13.0.

How rare is = 12.6 under the assumption that = 13.0?

x

x

x

T t f i ifi l k l ti


43/103

Test of significance: sample mean vs known population mean

Population of 10,000

A random sample

of size 100 is drawnMean height = 68

Question : Could the population mean be 65 ?

Question : What is the probability of obtaining a sample mean

of 68 from this population when sample size=100 ?

The hypothesis: H0: = 65.0 versus H1: 65.0

The assumptions: Random sample from a normal distributionwith = 10

The -level : = 0.05


44/103

The test statistic:

The critical region: Reject H0: = 65.0 if the valuecalculated for z is not between 1.96

The result:

The conclusion: Reject H0: = 65.0 since the valuecalculated for z is not between 1.96

Exact probability = 0.0027 , i.e., 0.27%

Test of significance: sample mean vs known

population mean

31

3

10

10

3

100

10

6568

Z


45/103


46/103

Example

As the serum creatinine (SCr) normal range depends on the

population studied. A fellow wanted to evaluate the mean SCramong adult males in Pondicherry.

From the literature he found that one well estabilished study

showed an average SCr of 1.18 mg/dL with a SD of 0.15

mg/dL. But based on his knowledge and experience he

believes that mean of SCr among local adult males should be

different.

So he decided to check this by measuring the SCr for 49 local

adult male volunteers. The mean SCr was found to be 1.22

mg/dL.

Test whether the mean SCr for the local population differ

significantly from that of the all adult males in Pondicherry ?


47/103


48/103


49/103


50/103

Two Independent samples t-test

Test statistic:

2

11

deg2

11

21

2

22

2

112

21

21

21

nn

snsns

freedomofreesnnwith

nns

xxt


51/103


52/103


53/103


54/103

NUMERICAL EXAMPLE OF UNPAIRED t - TEST


55/103

NUMERICAL EXAMPLE OF UNPAIRED t - TEST


56/103

tcal > t tab indicating that the mean energy expenditure in

obese group (10.3) is significantly (P


57/103

Practical: Systolic BP for a

random sample of 18 pregnantwomen during the second

trimester antenatal visits to

Hospital A and Hospital B. The

data are given in the Tabe.

Is there a difference in the

systolic BP of pregnant women

visiting Hospital A andHospital B ?

women

ID

Hospital A Hospital B

1 120 145

2 125 150

3 130 145

4 125 130

5 140 136

6 125 132

7 115 135

8 135 130

9 130 145


58/103


59/103


60/103


61/103


62/103


63/103


64/103

Paired Tests: Difference


65/103

Paired Tests: Difference

Two Continuous Outcomes

Known variance: Z test statistic

Unknown variance: t test statistic

H0

: d

= 0 vs. HA

: d

0

Paired Z-test or Paired t-test

/ /d dZ or T

n s n


66/103

Unpaired Tests: Common Variance

Same idea

Known variance: Z test statistic

Unknown variance: t test statistic

H0: 1 = 2 vs. HA: 12

Assume common variance

21

21

21

21

1111

nns

xxtor

nn

xxz

NUMERICAL EXAMPLE OF PAIRED t - TEST


67/103

NUMERICAL EXAMPLE OF PAIRED t - TEST

ESR - 1 hour ( mm)Pt.

No.

Before Rx

(a)After Rx

(b)Difference

( a - b = d )

Square ofdifference

(d2)

123

45678

910

254338

2041481528

3533

8106

710589

43

173332

1331437

19

3130

28910881024

169961

184949

361

961900

Total 326 70 256 7652

Inference


68/103

Calculated value oft = 7.33 with 9 df

Tabulated value oft(df=9)(5%) = 2.262

tcal > ttabindicating that the treatment had asignificant (P < 0.05) benefit in reducing the ESR

The mean ESR after treatment (7.0 mm) is

significantly less than the mean pre-treatment

ESR value (32.6 mm)

Inference


69/103

Binary Outcomes

Exact same idea

For large samples

Use Z test statistic

Now set up in terms of proportions, not means

0

0 0

(1 ) /

p pZ

p p n


70/103

Two Population Proportions

Exact same idea

For large samples use Z test statistic

21

)1()1(2211

21

n

pp

n

pp

ppz

Example 1: Difference in


71/103

Example 1: Difference in

proportions

Research Question: Are

antidepressants arisk factor for suicide

attempts in children and adolescents?

Example modified from: Antidepressant Drug Therapy and Suicide in SeverelyDepressed Children and Adults; Olfson et al.Arch Gen Psychiatry.2006;63:865-872.


72/103

Example 1

Design: Case-control study Methods: Researchers used Medicaid records to

compare prescription histories between 263children and teenagers (6-18 years) who had

attempted suicide and 1241 controls who hadnever attempted suicide (all subjects sufferedfrom depression).

Statistical question: Is a history of use ofantidepressants more common among casesthan controls?


73/103

Example 1

Statistical question: Is a history of use ofparticular antidepressants more commonamong heart disease cases than controls?

What will we actually compare?

Proportion of cases who used antidepressants in

the past vs. proportion of controls who did


74/103

No (%) of

cases

(n=263)

No (%) of

controls

(n=1241)

Any antidepressant

drug ever 120 (46%) 448 (36%)

46% 36%

Difference=10%

Results

Wh t d 10% diff


75/103

What does a 10% difference

mean? Before we perform any formal statisticalanalysis on these data, we already have a

lot of information.

Look at the basic numbers first; THENconsider statistical significance as a

secondary guide.

Is the association statistically


76/103

Is the association statistically

significant?

This 10% difference could reflect a trueassociation or it could be a fluke in this

particular sample.

The question: is 10% bigger or smallerthan the expected sampling variability?


77/103

What is hypothesis testing?

Statisticians try to answer this questionwith a formal hypothesis test


78/103

Hypothesis testing

Null hypothesis: There is no associationbetween antidepressant use and suicide

attempts in the target population (= the

difference is 0%)

Step 1: Assume the null hypothesis.


79/103

Hypothesis Testing

Step 2: Predict the sampling variability assuming the null hypothesis is truemath

theory (formula):

The standard error of the difference in two proportions

is:

Thus, we expect to see differences between the group as big as about 6.6% (2

standard errors) just by chance

033.1241

)1504

5681(

1504

568

263

)1504

5681(

1504

568

)-1()1(

21

n

pp

n

pp


80/103


81/103


82/103


83/103


84/103


85/103


86/103


87/103


88/103


89/103


90/103


91/103

Comparison of a observed proportion with ahypothesised one


92/103

hypothesised one

Hypothesis : A pharmaceutical company claimed that theirnew product can cure 80% of the patients

Data : 56 out of 80 with disease got cured (i.e. 70%)

= ---------(56 - 64)2

64+ ----------

(24 - 16)2

16

= ----- + ----- = ------ + ------ = 1 + 4 = 564 16 64 16

(-8)2

(8)2

64 64

The calculated value of2 (i.e., 5) with 1 degree of freedom exceeds the

table value (3.84) at 5% level

Hence, we reject the Null hypothesis that the efficacy of the new product is 80%

2

Cured Not Cured Total

56 (64) 24 (16) 80

Comparison of percentages from 2 samples


93/103

Cure rate: - Treatment A : 90% ; 90 out of 100

- Treatment B : 70% ; 105 out of 150

Treatment Cured Not

cured

Total

A

B

90 (78) 10 (22)

105 (117) 45 (33)

100

150

Total 195 55 250

2 = ------------ + ------------ + --------------- + ----------- = 13.99(90 - 78)2 (10 - 22)2 (105 - 117)2 (45 - 33)2

78 22 117 33 The calculated value of2 (i.e., 13.99 ) with 1 degree of freedom exceeds the

table value (3.84) at 5% level

Hence, we reject the Null hypothesis that the two treatments are equally effective

Chi square test on paired observations


94/103

Chi square test on paired observations

100 pts. received two drugs A & B in a random sequence

15 manifest toxicity to A

5 to B (including 4 to both A & B)

Compare toxicity 15 / 100 Vs 5 / 100

-Incorrect, as the same as 100 patients are tested twice

Chi-square test on paired observation


95/103

100 patients received two drugs A & B in a random sequence

Group Drug A Drug B Example General case

(1) Toxic Toxic 4 a

(2) Toxic Not toxic 11 b

(3) Not toxic Toxic 1 c

(4) Not toxic Not toxic 84 d

Groups (1) & (4) make no contribution

Considering groups (2) & (3), the expected number of patients

in each, under the Null hypothesis that the 2 drugs have the

same toxicity, is (11+1) / 2 = 6

Observed ExpectedGroup (2) 11 6G (3) 1 6


96/103

Group (3) 1 6

(11 6 )2 (1 6)2 25 25----------- ------------ ----- -----

This expression is same as (11- 1)2 /

Applying correction for continuity

(|11 1| - 1)2

-----------------------------11+1

As the calculated value of21 (6.75) exceeds the Table value (3.84) at

5% level, we reject the Null hypothesis

In general ,

2 =

6 6 6 6

+ =

= 8.33(11 + 1)

= 8.33 = 6.75

+

2 =

2 =

[|b-c| - 1]2-------------------

(b +c)

Review Question 1


97/103

Review Question 1

If we have a p-value of 0.03 and so decide that our

effect is statistically significant, what is the

probability that were wrong (i.e., that the

hypothesis test gave us a false positive)?

a. .03

b. .06

c. Cannot telld. 1.96

e. 95%

Review Question 2


98/103

Review Question 2

Standard error is:

a. For a given variable, its standard deviationdivided by the square root of n.

b. A measure of the variability of a samplestatistic.

c. The inverse of sample size.

d. A measure of the variability of a characteristic.e. All of the above.

Review Question 2


99/103

Review Question 2

Standard error is:

a. For a given variable, its standard deviationdivided by the square root of n.

b. A measure of the variability of a samplestatistic.

c. The inverse of sample size.

d. A measure of the variability of a characteristic.e. All of the above.

Review Question 3


100/103

Review Question 3

A randomized trial of two treatments for depressionfailed to show a statistically significant difference inimprovement from depressive symptoms (p-value

=.50). It follows that:

a. The treatments are equally effective.

b. Neither treatment is effective.

c. The study lacked sufficient power to detect adifference.

d. The null hypothesis should be rejected.

e. There is not enough evidence to reject the nullhypothesis.

Review Question 3


101/103

Review Question 3

A randomized trial of two treatments for depressionfailed to show a statistically significant difference inimprovement from depressive symptoms (p-value

=.50). It follows that:

a. The treatments are equally effective.

b. Neither treatment is effective.

c. The study lacked sufficient power to detect adifference.

d. The null hypothesis should be rejected.

e. There is not enough evidence to reject the nullhypothesis.

Review Question 4


102/103

Review Question 4

Following the introduction of a new treatment regime in arehab facility, alcoholism cure rates increased. Theproportion of successful outcomes in the two years following

the change was significantly higher than in the precedingtwo years (p-value:


103/103

Review Question 4

Following the introduction of a new treatment regime in arehab facility, alcoholism cure rates increased. Theproportion of successful outcomes in the two years following

the change was significantly higher than in the precedingtwo years (p-value:

tests of significance ssm

Documents