stats 95 experimental design –experimental design & lady tasting tea –type i and type ii...

Stats 95

• Experimental Design– Experimental Design & Lady Tasting Tea– Type I and Type II Errors– Null Hypothesis an Research Hypothesis

Lady Tasting Tea

• How would you design the experiment?• What task would you give her?• What would be the Independent Variable? Dependent

variable? Control condition• How many could she guess right by chance? • What if she can taste the difference, but she makes

mistakes?• Do you know for certain she can? Do you know for

certain she cannot?

Hypotheses

• H0 Null Hypothesis: there is nothing going on, Straw Man, the probability of guessing Tea in Milk is equal to guessing Milk in Tea

• H1 Research Hypothesis: something is going on, probability of correct identification is not equal to guessing between Milk in Tea and Tea in Milk.

NS+N

Hits(response “yes” on signal trial)

Criterion

Internal response

Pro

bab

ilit

y d

ensi

ty

Say “yes”Say “no”

NS+N

Correct rejects(response “no” on no-signal trial)

Criterion

Internal response

Pro

bab

ilit

y d

ensi

ty

Say “yes”Say “no”

NS+N

Misses(response “no” on signal trial)

Criterion

Internal response

Pro

bab

ilit

y d

ensi

ty

Say “yes”Say “no”

NS+N

False Alarms(response “yes” on no-signal trial)

Criterion

Internal response

Pro

bab

ilit

y d

ensi

ty

Say “yes”Say “no”

“What Cold Possibly Go Worng?”: Type I and Type II Errors

Reality

Perception

YES

(Signal + Noise)

NO

(Noise)

YES

(Signal + Noise)Hit

False Alarm (Type I)

False Positive

NO

(Noise)

Miss (Type II)

False NegativeCorrect Rejection

“What Cold Possibly Go Worng?”: Type I and Type II Errors

REALITY of

PREGNANCY

TEST RESULTS

YES NO

“PREGNANT”

(Reject Null)

HIT

(Pregnant & “+” on test)

FALSE ALARM (Type I)

Also called False Positive

(Not Pregnant & “+”)

“NOT PREGNANT”

(Fail to reject the Null)

Miss (Type II)

Also called False Negative

(Pregnant & “-”)

Correct Rejection

(Not Pregnant & “-”)

The End

Statistics in Correlations & Experiments

• Correlations measure Relationship– Strength and direction of relatioship

• Experiments measure the Differences– Statistical significance of the difference

Correlation: Measuring Relationship

• Sir Francis Galton (Uncle to Darwin– Development of behavioral statistics– Father of Eugenics– Science of fingerprints as unique– Retrospective IQ of 200– Drove himself mad just to prove

you could do it– Invented the pocket

13

2.3 The Science of Explanation

• Measuring correlation– more-more/less-less– more-less/less-more

• Correlation coefficient – measure of direction & strength– r = 1– r = -1– r = 0

14

Correlation

• What does correlation coefficient mean?

17

2.3 The Science of Explanation

• Experiment—2 critical features• (1) Manipulation

– independent variable – dependent variable—measured– Control Group Condition (or Variable)– Experimental Group Condition (or Variable)

• (2) Randomization - controls for a 3rd variable (you know exists but are not

interested in)– versus self-selection

Dependent VariablesWithout Demand Charcteristics

• DVs that aren’t subject to biased responses• Examples:– Is a painting in a museum popular?• There will be increased wear on the carpet near it.– Did a dental flossing lecture work?• Students will have cleaner teeth the next day.– Did a safer sex intervention for commercial sexworkers work?• There will be more condoms discarded in the park they work in.

Variation in IV Causes Variation in DV

1. Cause → Effect: whenever IV occurs, outcome DV should result.

Safe sex intervention Condoms in Park

2. Cause absent → Effect absentNo SS intervention no condoms

3. Cause variation → Effect variationMore or better interventions more condoms in park

Experimental & Control Groups• Experimental

Condition: Cause is valid– E.g., drug, alcohol

• Control Condition: cause is invalid– Placebo, juice

• Essence of experiment is to control conditions beforehand

21

The Science of Observation

• Validity—able to draw accurate inferences– construct validity: e.g., describing what intelligence is

and is not, “construct” refers to the “theory”– predictive validity: over time you find X predicts Y

• Reliability—same result each time?

- Test/Re-Test

- Parallel

- Inter-Item

Statistical Significance

• A finding is statistically significant if the data differ from what we would expect from chance alone, if there were, in fact, no actual difference.

• They may not be significant in the sense of big, important differences, but they occurred with a probability below the critical cutoff value, usually a z-score or p < .05

• Reject or Fail to Reject the NULL Hypothesis

Graphing Frequency

Discrete: Histogram Continuous: Frequency Polygon

Stem-and-Leaf: Exam 1 & 3

Selection of ranges & bins like Histogram, but usually simpler.

These plots represent the scores on an exam given to two different sections for the same course.