Lecture 9

Raul Cruz-CanoEPIB 698E Fall 2013

Change of Schedule

• 11/13/2013: Lecture 9-Hypothesis testing• 11/20/2013: Lecture 10-Regression• 11/27/2013: Review of Midterm• 12/4/2013: Lecture 11-Collinearity & Normality Tests• 12/11/2013: Lecture 12-Macros• 12/18/2013: Final Exam

No review before the Final Exam

One-Sample T-test1. A one-sample t-test is used to compare a

sample to an average or general population. 2. You may know the average height of men in

the U.S., and you could test whether a sample of professional basketball players differ significantly in height from the general U.S. population.

3. A significant difference would indicate that basketball players belong to a different distribution of heights than the general U.S. population.

4

Student's t-test

• Independent One-Sample t-test• This equation is used to compare one sample mean to a

specific value μ0.

• Where s is the grand standard deviation of the sample. N is the sample size. The degrees of freedom used in this test is N-1.

0

/

Xts N

T-Test using PROC Univariate

• We can also specify a null hypothesis value for the mean when using Proc Univariate by using the mu0 option.

proc univariate data=blood mu0=15; var WBC;run;

DATA blood;INFILE ‘C:\blood.txt';INPUT ID Sex $ BloodType $ AgeGroup $ RBC WBC cholesterol;run;

Or we can use the SAS Dataset

PROC TTEST

The following statements are available in PROC TTEST. PROC TTEST < options > ; CLASS variable ; PAIRED variables ; BY variables ; VAR variables ; RUN;

PROC TTEST

OPTIONS :ALPHA=pspecifies that confidence intervals are to be 100(1-p)% confidence intervals,

where 0<p<1. By default, PROC TTEST uses ALPHA=0.05. If p is 0 or less, or 1 or more, an error message is printed.

H0=mrequests tests against m instead of 0 in all three situations (one-sample, two-

sample, and paired observation t tests). By default, PROC TTEST uses H0=0. DATA=SAS-data-setnames the SAS data set for the procedure to use

*One sample ttest*;Proc ttest data =blood H0=200;var cholesterol;run;

One sample t test Output

The TTEST Procedure

Variable: cholesterol

N Mean Std Dev Std Err Minimum Maximum

795 201.4 49.8867 1.7693 17.0000 331.0

Mean 95% CL Mean Std Dev 95% CL Std Dev

201.4 198.0 204.9 49.8867 47.5493 52.4676

DF t Value Pr > |t|

794 0.81 0.4175

95%CL Mean is 95% confidence interval for the mean.

95%CL Std Dev is 95% confidence interval for the standard deviation.

One sample t test Output Variable: cholesterol N Mean Std Dev Std Err Minimum Maximum 795 201.4 49.8867 1.7693 17.0000 331.0

Mean 95% CL Mean Std Dev 95% CL Std Dev 201.4 198.0 204.9 49.8867 47.5493 52.4676

DF t Value Pr > |t| 794 0.81 0.4175

DF - The degrees of freedom for the t-test is simply the number of valid observations minus 1. We loose one degree of freedom because we have estimated the mean from the sample. We have used some of the information from the

data to estimate the mean; therefore, it is not available to use for the test and the degrees of freedom accounts for this

T value is the t-statistic. It is the ratio

of the difference between the sample mean and the given

number to the standard error of the

mean.

It is the probability of observing a greater absolute value of t under the null hypothesis.

Matched Pairs T-test

1. A matched pairs t-test usually involves the same subjects being measured on some factor at two points in time.

2. For example, subjects could be tested on short-term memory, receive a brief tutorial on memory aids, then have their short-term memory re-tested.

3. A significant difference in score (after-before) would indicate that the tutorial had an effect.

12

Student's t-test• Dependent t-test is used when the samples are dependent; that

is, when there is only one sample that has been tested twice (repeated measures) or when there are two samples that have been matched or "paired".

• For this equation, the differences between all pairs must be calculated. The pairs are either one person's pretest and posttest scores or one person in a group matched to another person in another group. The average (XD) and standard deviation (sD) of those differences are used in the equation. The constant μ0 is non-zero if you want to test whether the average of the difference is significantly different than μ0. The degree of freedom used is N-1.

0

/D

D

Xts N

Paired Statements• PAIRED: the PAIRED statement identifies the variables to be

compared in paired t test1. You can use one or more variables in the PairLists. 2. Variables or lists of variables are separated by an asterisk (*)

or a colon (:). 3. The asterisk (*) requests comparisons between each

variable on the left with each variable on the right. 4. Use the PAIRED statement only for paired comparisons. 5. The CLASS and VAR statements cannot be used with the

PAIRED statement.

title 'Paired Comparison';data pressure;input SBPbefore SBPafter @@;diff_BP=SBPafter-SBPbefore ;

datalines; 120 128 124 131 130 131 118 127 140 132 128 125 140 141 135 137 126 118 130 132 126 129 127 135 ; run;

proc ttest data=pressure; paired SBPbefore*SBPafter; run;

Paired t test Output

The TTEST Procedure

Difference: SBPbefore - SBPafter

N Mean Std Dev Std Err Minimum Maximum

12 -1.8333 5.8284 1.6825 -9.0000 8.0000

Mean 95% CL Mean Std Dev 95% CL Std Dev

-1.8333 -5.5365 1.8698 5.8284 4.1288 9.8958

DF t Value Pr > |t|

11 -1.09 0.2992

Mean of the differences

T statistics for testing if the mean of the difference is 0

P =0.3, suggest the mean of the difference is equal to 0

Paired T-test (Example 2)10 dieters following Atkin’s diet vs. 10 dieters following Jenny Craig

Hypothetical RESULTS:Atkin’s group loses an average of 34.5 lbs.

J. Craig group loses an average of 18.5 lbs.

Conclusion: Atkin’s is better?

What if data were paired?

e.g., one-to-one matching; find pairs of study participants who have same age, gender, socioeconomic status, degree of overweight, etc.

Atkin’s• +4, +3, 0, -3, -4, -5, -11, -14, -15, -300

J. Craig• -8, -10, -12, -16, -18, -20, -21, -24, -26, -30

Enter data differently in SAS…10 pairs, rather than 20 individual observations

data paired;input lossa lossj;diff=lossa-lossj;datalines ;+4 -8 +3 -10 0 -12 -3 -16 -4 -18 -5 -20 -11 -21 -14 -24 -15 -26 -300 -30

;run;

Tests in SAS…

/*to get all paired tests*/proc univariate data=paired;var diff;run;/*To get just paired ttest*/proc ttest data=paired;var diff;run;/*To get paired ttest, alternatively*/proc ttest data=paired;paired lossa*lossj;run;

Two-Sample T-test1. A two-sample t-test compares two groups on some

factor. 2. For example, one group could receive an

experimental treatment and the second group could receive a standard of care treatment or placebo.

3. Notice that in a two-sample t-test, two distinct groups are being compared, as opposed to the one-sample, where one group is compared to a general average, or a matched-pairs, where only one group is being measured twice.

Two independent samples t-test

• An independent samples t-test is used when you want to compare the means of a normally distributed interval dependent variable for two independent groups. For example, using the hsb2 data file, say we wish to test whether the mean for write is the same for males and females.

proc ttest data = "c:\hsb2"; class female;

var write;

run;

CLASS: CLASS statement giving the name of the classification (or grouping) variable must accompany the PROC TTEST statement in the two independent sample cases (TWO SAMPLE T TEST). The class variable must have two, and only two, levels.

Two Independent Samples: Distribution Free Tests

• There are times when the assumptions for using a t-test are not met.

• One common problem is that the data are not normally distributed, and your sample size is small.

• Another common problem is that the data values may only represent ordered categories.

• We need a nonparametric test to analyze differences in central tendencies for ordinal data.

• For very small samples, nonparametric tests are often more appropriate since assumptions concerning distributions are difficult to determine.

Distribution Free Tests

• The biggest difference between a parametric and nonparametric test is the fact that a parametric test assumes that the data under investigation is coming from a normal distribution.

• The SAS software provides several nonparametric tests such as the Wilcoxon rank-sum test and the Kruskal-Wallis test when dealing with two or more samples.

Non-parametric tests

• t-tests require your outcome variable to be normally distributed (or close enough).

• Non-parametric tests are based on RANKS instead of means and standard deviations (=“population parameters”).

Example: non-parametric tests10 dieters following Atkin’s diet vs. 10 dieters following Jenny Craig

Hypothetical RESULTS:Atkin’s group loses an average of 34.5 lbs.

J. Craig group loses an average of 18.5 lbs.

Conclusion: Atkin’s is better?

Enter data in SAS…data nonparametric;input loss diet $;datalines ;+4 atkins +3 atkins0 atkins-3 atkins-4 atkins-5 atkins -11 atkins-14 atkins-15 atkins-300 atkins-8 jenny-10 jenny-12 jenny-16 jenny-18 jenny -20 jenny-21 jenny-24 jenny-26 jenny-30 jenny

;run;

t-test doesn’t work…

• Comparing the mean weight loss of the two groups is not appropriate here.

• The distributions do not appear to be normally distributed.

• Moreover, there is an extreme outlier (this outlier influences the mean a great deal).

Statistical tests to compare ranks:

• Wilcoxon rank-sum test is analogue of two-sample t-test.

• Wilcoxon signed-rank test is analogue of one-sample t-test, usually used for paired data

NPAR1WAY Procedure

• The NPAR1WAY procedure provides the following location tests: Wilcoxon rank sum test (Mann-Whitney U test), Median test, Savage test, and Van der Waerden test.

• Also note that the Wilcoxon rank sum test can be obtained from the FREQ procedure.

Wilcoxon rank-sum test

• RANK the values, 1 being the least weight loss and 20 being the most weight loss.

• Atkin’s• +4, +3, 0, -3, -4, -5, -11, -14, -15, -300• 1, 2, 3, 4, 5, 6, 9, 11, 12, 20• J. Craig• -8, -10, -12, -16, -18, -20, -21, -24, -26, -30• 7, 8, 10, 13, 14, 15, 16, 17, 18, 19

Wilcoxon “rank-sum” test

• Sum of Atkin’s ranks:• 1+ 2 + 3 + 4 + 5 + 6 + 9 + 11+ 12 + 20=73• Sum of Jenny Craig’s ranks:7 + 8 +10+ 13+ 14+ 15+16+ 17+ 18+19=137

• Jenny Craig clearly ranked higher!

Wilcoxon rank-sum (Example 1)

/*to get wilcoxon rank-sum test*/proc npar1way wilcoxon data=nonparametric;class diet;var loss;run;

/*To get ttest*/proc ttest data=nonparametric;class diet;var loss;run;

Compare p-values0.0156 vs. 0.5962

Wilcoxon rank-sum test for two samples (Example 2)

1. Consider the following experiment. We have two groups, A and B. Group B has been treated with a drug to prevent tumor formation.

2. Both groups are exposed to a chemical that encourages tumor growth. The masses (in grams) of tumors in groups A and B are:

DATA TUMOR;INPUT GROUP $ MASS @@;DATALINES;A 3.1 A 2.2 A 1.7 A 2.7 A 2.5B 0.0 B 0.0 B 1.0 B 2.3;

PROC NPAR1WAY DATA =TUMOR WILCOXON;TITLE 'NONPARAMETRIC TEST TO COMPARE TUMOR MASSES’ ;CLASS GROUP;VAR MASS;

RUN;

Wilcoxon rank-sum test for two samples (Example 3)

• Consider the following example, Researcher B is interested in testing the difference between the effectiveness of two allergy drugs out on the market.

• He would like to administer drug A to a random sample of study subjects and then drug B to another random sample who suffer from the same symptoms as those individuals taking drug A.

• Researcher B would like to see if there is a difference between the two groups in the time, in minutes, for subjects to feel relief from their allergy symptoms.

Wilcoxon rank-sum test for two samples (Example 3)

data drugtest;input subject drug_group $ time;datalines;1 A 432 A 403 A 324 A 375 A 556 A 507 A 528 A 339 B 2810 B 3311 B 4812 B 3713 B 4014 B 4215 B 3516 B 43;run;proc means median min max; by drug_group; var time;run;

proc npar1way wilcoxon; class drug_group; var time;run;

The p-values (.3431) are above 0.05, you cannot reject the null hypothesis and must conclude that there is no difference between the median times to relief for both drug groups.

Wilcoxon “signed-rank” testH0: median weight loss in Atkin’s group = 0Ha:median weight loss in Atkin’s not 0Atkin’s• +4, +3, 0, -3, -4, -5, -11, -14, -15, -300

Rank absolute values of differences (ignore zeroes):Ordered values: 300, 15, 14, 11, 5, 4, 4, 3, 3, 0Ranks: 1 2 3 4 5 6-7 8-9 - Sum of negative ranks: 1+2+3+4+5+6.5+8.5=30Sum of positive ranks: 6.5+8.5=15

Signed-rank (Example 1)

/*to get one-sample tests (both student’s t and signed-rank*/

proc univariate data=nonparametric;var loss;where diet="atkins";run;

Compare p-valuesYou need to use the option ‘m0=’ to change the alternative hypothesis, not ‘h0=’ as in PROC TTEST

ANOVA• A one-way analysis of variance (ANOVA) is used

when you have a categorical independent variable (with two or more categories) and a normally distributed interval dependent variable and you wish to test for differences in the means of the dependent variable broken down by the levels of the independent variable.

• Just an extension of the t-test (an ANOVA with only two groups is mathematically equivalent to a t-test).

ANOVA (ANalysis Of VAriance)

• Idea: For two or more groups, test difference between means, for quantitative normally distributed variables.

• Like the t-test, ANOVA is “parametric” test—assumes that the outcome variable is roughly normally distributed

The “F-test”

groupswithinyVariabilit

groupsbetweenyVariabilitF

Is the difference in the means of the groups more than background noise (=variability within groups)?

amenorrheic oligomenorrheic eumenorrheic

0.7

0.8

0.9

1.0

1.1

1.2

SPINE

Between group variation

Spine bone density vs. menstrual regularity

Within group variability

Within group variability

Within group variability

The F-distribution• A ratio of sample variances follows an F-

distribution:

22

220

:

:

withinbetweena

withinbetween

H

H

The F-test tests the hypothesis that two sample variances are equal. F will be close to 1 if sample variances are equal.

mnwithin

between F ,2

2

~

The F-distribution

• The F-distribution is a continuous probability distribution that depends on two parameters n and m (numerator and denominator degrees of freedom, respectively):

ANOVA Table

Between (k groups)

k-1 SSB(sum of squared deviations of group means from

grand mean)

SSB/k-1 Go to

Fk-1,nk-k

chart

Total variation

nk-1 TSS(sum of squared deviations of observations from grand mean)  

Source of variation

d.f.

Sum of squares

Mean Sum of Squares F-statistic p-value

Within(n individuals

per group)

nk-k SSW (sum of squared deviations of observations from their group mean)

s2=SSW/nk-k

knkSSW

kSSB

1

TSS=SSB + SSW

ANOVA summary

• A statistically significant ANOVA (F-test) only tells you that at least two of the groups differ, but not which ones differ.

• Determining which groups differ (when it’s unclear) requires more sophisticated analyses to correct for the problem of multiple comparisons…

ANOVA• The following example studies the effect of bacteria on the nitrogen content of red clover plants. The

treatment factor is bacteria strain, and it has six levels. Five of the six levels consist of five different Rhizobium trifolii bacteria cultures combined with a composite of five Rhizobium meliloti strains. The sixth level is a composite of the five Rhizobium trifolii strains with the composite of the Rhizobium meliloti. Red clover plants are inoculated with the treatments, and nitrogen content is later measured in milligrams.

title1 'Nitrogen Content of Red Clover Plants';data Clover;

input Strain $ Nitrogen @@; datalines; 3DOK1 19.4 3DOK1 32.6 3DOK1 27.0 3DOK1 32.1 3DOK1 33.0 3DOK5 17.7 3DOK5 24.8 3DOK5 27.9 3DOK5 25.2 3DOK5 24.3 3DOK4 17.0 3DOK4 19.4 3DOK4 9.1 3DOK4 11.9 3DOK4 15.8 3DOK7 20.7 3DOK7 21.0 3DOK7 20.5 3DOK7 18.8 3DOK7 18.6 3DOK13 14.3 3DOK13 14.4 3DOK13 11.8 3DOK13 11.6 3DOK13 14.2 COMPOS 17.3 COMPOS 19.4 COMPOS 19.1 COMPOS 16.9 COMPOS 20.8

;run;proc anova data = Clover; class strain; model Nitrogen = Strain; run;

proc freq data = Clover; tables Strain; run;

ANOVA Graphs

ods graphics on; proc anova data = Clover; class strain; model Nitrogen = Strain; run;ods graphics off;

ANOVA

• The test for Strain suggests that there are differences among the bacterial strains, but it does not reveal any information about the nature of the differences. Mean comparison methods can be used to gather further information.

Another ANOVA Example

• Let’s assume that the researcher has data on individuals from three different diet camps.

• All the researcher is concerned with is seeing whether the mean weights of the individuals in each camp are significantly different from one another.

• Since we are comparing three different means, we must employ the use of ANOVA.

Another Exampledata expanova; input group weight;datalines;1 2231 2341 2541 2671 2342 2872 2132 2152 2342 2563 2343 3423 1983 2563 303;proc anova; class group; model weight = group; means group;run;

Adds little summary at the end

Non-parametric ANOVA

Kruskal-Wallis one-way ANOVA

Extension of the Wilcoxon Sign-Rank test for 2 groups; based on ranks

 Proc NPAR1WAY in SAS

Kruskal-Wallis (Example)

data Gossypol;input Dose n;do i=1 to n;input Gain @@;output;end;datalines;0 16 228 229 218 216 224 208 235 229 233 219 224 220 232 200 208 232 .04 11 186 229 220 208 228 198 222 273 216 198 213 .07 12 179 193 183 180 143 204 114 188 178 134 208 196 .10 17 130 87 135 116 118 165 151 59 126 64 78 94 150 160 122 110 178 .13 11 154 130 130 118 118 104 112 134 98 100 104;run;

1. The data consist of weight gain measurements for five different levels of gossypol additive.

2. Gossypol is a substance contained in cottonseed shells, and these data were collected to study the effect of gossypol on animal nutrition.

Kruskal-Wallis (Example)

proc npar1way data=Gossypol; class Dose; var Gain;run;

1. The p-value, or probability of a larger statistic under the null hypothesis, is <.0001.

2. This leads to rejection of the null hypothesis that there is no difference in location for Gain among the levels of Dose