statistical analysis of the two group post-only randomized experiment

Statistical Analysis of the Two Group Statistical Analysis of the Two Group Post-Only Randomized ExperimentPost-Only Randomized Experiment

Analysis RequirementsAnalysis Requirements

Two groupsTwo groups A post-only measureA post-only measure Two distributions, each with an average and variationTwo distributions, each with an average and variation Want to assess treatment effectWant to assess treatment effect Treatment effect = statistical (i.e., nonchance) Treatment effect = statistical (i.e., nonchance)

difference between the groupsdifference between the groups

R X OR O

Statistical AnalysisStatistical Analysis


Controlgroupmean


Controlgroupmean

Treatmentgroupmean


Controlgroupmean

Treatmentgroupmean

Is there a difference?

What Does What Does DifferenceDifference Mean? Mean?


Mediumvariability


Mediumvariability

Highvariability


Mediumvariability

Highvariability

Lowvariability


Mediumvariability

Highvariability

Lowvariability

The mean differenceis the same for all

three cases.


Mediumvariability

Highvariability

Lowvariability

Which one showsthe greatestdifference?

What Does What Does DifferenceDifference Mean? Mean? A statistical difference is a function of the A statistical difference is a function of the

difference between meansdifference between means relative to the relative to the variabilityvariability..

A small difference between means with large A small difference between means with large variability could be due to variability could be due to chancechance..

Like a Like a signal-to-noisesignal-to-noise ratio. ratio.

Lowvariability

Which one showsthe greatestdifference?

What Do We Estimate?What Do We Estimate?

Lowvariability


Lowvariability

Signal

Noise


Lowvariability

Signal

Noise

Difference between group means=


Lowvariability

Signal

Noise

Difference between group means

Variability of groups=


Lowvariability

Signal

Noise



=XT - XC

SE(XT - XC)

_ _

_ _


Lowvariability

Signal

Noise



XT - XC

SE(XT - XC)=

= t-value

_ _

_ _


The t-test, one-way analysis of variance The t-test, one-way analysis of variance (ANOVA) and a form of regression all test the (ANOVA) and a form of regression all test the same thing and can be considered equivalent same thing and can be considered equivalent alternative analyses.alternative analyses.

The regression model is emphasized here The regression model is emphasized here because it is the most general.because it is the most general.

Lowvariability

Regression Model for t-Test or One-Way Regression Model for t-Test or One-Way ANOVAANOVA

yi = 0 + 1Zi + ei

Regression Model for t-Test or One-Way Regression Model for t-Test or One-Way ANOVAANOVA

yyii = = outcome score for the ioutcome score for the ithth unit unit

00 == coefficient for the coefficient for the interceptintercept

11 == coefficient for the coefficient for the slopeslope

ZZii == 1 if i1 if ithth unit is in the treatment group unit is in the treatment group

0 if i0 if ithth unit is in the control group unit is in the control groupeeii == residual for the iresidual for the ithth unit unit

yi = 0 + 1Zi + ei

where:

In Graph Form...In Graph Form...


0(Control)

1(Treatment) Zi


0(Control)

1(Treatment)

Yi

Zi


0(Control)

1(Treatment)

0 is the intercepty-value when z=0.

Yi

Zi


0(Control)

1(Treatment)


1 is the slope.

Yi

Zi

Why Is 1 the Mean Difference?

0(Control)

1(Treatment)


1 is the slope.

Yi

Zi


0(Control)

1(Treatment)

Intuitive Explanation:Because slope is the change in y for a 1-unit change in x.

Yi

Zi

Change in y

Unit change in x (i.e., z)


0(Control)

1(Treatment)

Since the 1-unit change in x is the treatment-control difference, the slope is the difference between

the posttest means of the two groups.

Yi

Zi

Change in y

Why Why 11 Is the Mean Difference Is the Mean Difference

ininyi = 0 + 1Zi + ei


in in

First, determine effect for each group:

yi = 0 + 1Zi + ei


inin


For control group (Zi = 0):

yi = 0 + 1Zi + ei


inin



yi = 0 + 1Zi + ei

yC = 0 + 1(0) + 0


inin



yi = 0 + 1Zi + ei

yC = 0 + 1(0) + 0 ei averages to 0across the group.


inin



yi = 0 + 1Zi + ei

yC = 0 + 1(0) + 0

yC = 0

ei averages to 0across the group.


in in



For treatment group (Zi = 1):

yi = 0 + 1Zi + ei

yC = 0 + 1(0) + 0

yC = 0



inin




yi = 0 + 1Zi + ei

yC = 0 + 1(0) + 0

yC = 0

yT = 0 + 1(1) + 0



inin




yi = 0 + 1Zi + ei

yC = 0 + 1(0) + 0

yC = 0

yT = 0 + 1(1) + 0

yT = 0 + 1



ininyi = 0 + 1Zi + ei


in in

Then, find the difference between the two groups:

yi = 0 + 1Zi + ei


in in


yi = 0 + 1Zi + ei

yT = 0 + 1

yT

treatment


inin


yi = 0 + 1Zi + ei

yC = 0yT = 0 + 1

yT - yC =

controltreatment


in in


yi = 0 + 1Zi + ei

yC = 0yT = 0 + 1

yT - yC = (0 + 1)

controltreatment


inin


yi = 0 + 1Zi + ei

yC = 0yT = 0 + 1

yT - yC = (0 + 1) - 0

controltreatment


in in


yi = 0 + 1Zi + ei

yC = 0yT = 0 + 1

yT - yC = (0 + 1) - 0

controltreatment

yT - yC = 0 + 1 - 0


in in


yi = 0 + 1Zi + ei

yC = 0yT = 0 + 1

yT - yC = (0 + 1) - 0

controltreatment

yT - yC = 0 + 1 - 0

yT - yC = 1

ConclusionsConclusions

t-test, one-way ANOVA and regression t-test, one-way ANOVA and regression analysis all yield analysis all yield samesame results in this results in this case.case.

The regression analysis method utilizes The regression analysis method utilizes a a dummy variabledummy variable for treatment.for treatment.

Regression analysis is the most Regression analysis is the most generalgeneral model of the three.model of the three.

statistical analysis of the two group post-only randomized experiment

Documents

medium variability slide

difference mean

mean difference

treatment zizi slide

variability of groups

statistical analysis

treatment yiyi zizi

large variability