introducing concepts of statistical inference beth chance, john holcomb, allan rossman cal poly –...
Post on 19-Dec-2015
217 views
TRANSCRIPT
![Page 1: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/1.jpg)
Introducing Concepts of Statistical Inference
Beth Chance, John Holcomb, Allan Rossman
Cal Poly – San Luis Obispo, Cleveland State University
![Page 2: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/2.jpg)
22
Ptolemaic Curriculum?
“Ptolemy’s cosmology was needlessly complicated, because he put the earth at the center of his system, instead of putting the sun at the center. Our curriculum is needlessly complicated because we put the normal distribution, as an approximate sampling distribution for the mean, at the center of our curriculum, instead of putting the core logic of inference at the center.”
– George Cobb (TISE, 2007)
![Page 3: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/3.jpg)
3
Is this feasible?
Experience at post-calculus level Developed spiral curriculum with logic of inference
(for 2×2 tables) in chapter 1 ISCAM: Investigating Statistical Concepts,
Applications, and Methods (Chance, Rossman) New project (funded by NSF/CCLI)
Rethinking for lower mathematical level More complete shift, including focus on entire
statistical process as a whole
3
![Page 4: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/4.jpg)
Workshop goals
Enable you to: Re-examine how you introduce concepts of
statistical inference to your students Help your students to understand fundamental
concepts of statistical inference Develop students’ understanding of the process of
statistical investigations Introduce normal-based methods of inference to
complement randomization-based ones
![Page 5: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/5.jpg)
Workshop goals (cont.)
Enable you to: Implement activities based on real data from
genuine studies Assess student understanding of inference
concepts Make effective use of simulations, both tactile and
computer-based
![Page 6: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/6.jpg)
6CAUSE Webinar April 2009 6
Agenda
Mon pm: Inference for proportion Overview, introductions Statistical significance via simulation Exact binomial inference CI for proportion Transition to normal-based inference for
proportion
![Page 7: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/7.jpg)
77
Agenda (cont.)
Tues am: Inference for 2×2 table Simulating randomization test Fisher’s exact test Observational studies, confounding Independent random samples
Tues pm: Comparing 2 groups with quant response Simulating randomization test Matched pairs designs
![Page 8: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/8.jpg)
88
Agenda (cont.)
Wed am: Assessment issues Strategies for assessing student
understanding/learning Preliminary findings
Wed pm: More inference scenarios Comparing several groups (ANOVA, chi-square) Correlation/regression Discussion of implementation issues
![Page 9: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/9.jpg)
99
Some notes
Agenda is always subject to change Already has changed some!
We’ll discuss some assessment, implementation issues throughout
Please offer questions, comments as they arise Be understanding when we don’t have all the
answers! We’ll also discuss some thorny issues that
we have not resolved among ourselves
![Page 10: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/10.jpg)
Introductions
Who are you? Where/what do you teach? Why interested in this topic?
![Page 11: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/11.jpg)
1111
Example 1: Helper/hinderer?
Sixteen infants were shown two videotapes with a toy trying to climb a hill One where a “helper” toy pushes the original toy up One where a “hinderer” toy pushes the toy back down
Infants were then presented with the two toys as wooden blocks Researchers noted which toy infants chose
http://www.yale.edu/infantlab/socialevaluation/Helper-Hinderer.html
![Page 12: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/12.jpg)
1212
Example 1: Helper/hinderer?
Data: 14 of the 16 infants chose the “helper” toy Core question of inference:
Is such an extreme result unlikely to occur by chance (random selection) alone …
… if there were no genuine preference (null model)?
![Page 13: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/13.jpg)
1313
Analysis options
Could use a binomial probability calculation We prefer a simulation approach
To emphasize issue of “how often would this happen in long run?”
Starting with tactile simulation
![Page 14: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/14.jpg)
1414
Strategy
Students flip a fair coin 16 times Count number of heads, representing choices of
“helper” toy Fair coin represent null model of no genuine
preference Repeat several times, combine results
See how surprising to get 14 or more heads even with “such a small sample size”
Approximate (empirical) P-value Turn to applet for large number of repetitions:
http://statweb.calpoly.edu/bchance/applets/BinomDist3/BinomDist.html
![Page 15: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/15.jpg)
15
Results
Pretty unlikely to obtain 14 or more heads in 16 tosses of a fair coin, so …
Pretty strong evidence that infants do have genuine preference for helper toy and were not just picking at random
![Page 16: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/16.jpg)
Example 1: Helper/hinderer
Can do this on day 1 of course Logic of statistical inference/significance Null model, simulation, p-value, significance
![Page 17: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/17.jpg)
Example 2: Kissing
Study: 8 of 12 kissing couples lean to right Does this provide evidence against 50/50
model? Does this provide evidence against 75/25
model? What models does this provide evidence
against?
![Page 18: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/18.jpg)
Example 2: Kissing
Many new ideas here: Students describe rather than perform simulation Non-significant result (8/12) Null model other than 50/50 Looking at lower tail Sample size effect Big idea: Interval of plausible values (CI) Effect of confidence level Importance of random sampling
![Page 19: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/19.jpg)
Transition to normal-based inference Two methods to find p-value for proportion:
Approximation by simulation Exact binomial calculation
Why should we present normal approx at all? Because it’s commonly used (not good reason) Because even minimally observant student will
notice similarities of these simulated distributions Because z-scores convey additional information
Distance from expected, measured in SDs
![Page 20: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/20.jpg)
Example 1: Baseball Big Bang Some non-trivial aspects
Defining parameter Expressing hypotheses Sampling distribution
z = -5.75 conveys more information than p-value ≈ 0 95% CI:
Does this produce more/less understanding than forming CI by inverting test?
n
ppp
ˆ1ˆ96.1ˆ
![Page 21: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/21.jpg)
Example 2: Which tire?
Which tire would you choose? Fun, simple in-class data collection
Almost always in conjectured direction May or may not be significant
Can use simulation or binomial or normal Investigate effect of sample size
![Page 22: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/22.jpg)
Example 3: Cat Households
Sensible to use normal approx here H0: = 1/3, Ha: ≠ 1/3 z = -10.4, p-value ≈ .0000 99% CI: (.312, .320)
P-value and CI are complementary But provide different information
Statistical vs practical significance
![Page 23: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/23.jpg)
Example 4: Female Senators
95% CI for : (.096, .244) Beware of biased sampling methods If you have access to entire population: no
inference to be drawn!
![Page 24: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/24.jpg)
2424
Example 2: Dolphin therapy?
Subjects who suffer from mild to moderate depression were flown to Honduras, randomly assigned to a treatment
Is dolphin therapy more effective than control? Core question of inference:
Is such an extreme difference unlikely to occur by chance (random assignment) alone (if there were no treatment effect)?
Dolphin therapy Control group TotalSubject improved 10 3 13Subject did not 5 12 17
Total 15 15 30Proportion 0.667 0.200
![Page 25: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/25.jpg)
2525
Some approaches
Could calculate test statistic, P-value from approximate sampling distribution (z, chi-square) But it’s approximate But conditions might not hold But how does this relate to what “significance” means?
Could conduct Fisher’s Exact Test But there’s a lot of mathematical start-up required But that’s still not closely tied to what “significance” means
Even though this is a randomization test
![Page 26: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/26.jpg)
2626
Alternative approach
Simulate random assignment process many times, see how often such an extreme result occurs Assume no treatment effect (null model) Re-randomize 30 subjects to two groups (using cards)
Assuming 13 improvers, 17 non-improvers regardless Determine number of improvers in dolphin group
Or, equivalently, difference in improvement proportions Repeat large number of times (turn to computer) Ask whether observed result is in tail of distribution
Indicating saw a surprising result under null model Providing evidence that dolphin therapy is more effective
![Page 27: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/27.jpg)
2727
Analysis
http://www.rossmanchance.com/applets/Dolphins/Dolphins.html
![Page 28: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/28.jpg)
Non-simulation approach
Exact randomization distribution Hypergeometric distribution Fisher’s Exact Test p-value =
= .0127 0.30
0.25
0.20
0.15
0.10
0.05
0.00
X
Pro
bability
10
0.0127
3
Distribution PlotHypergeometric, N=30, M=13, n=15
15
30
2
17
13
13
3
17
12
13
4
17
11
13
5
17
10
13
![Page 29: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/29.jpg)
2929
Conclusion
Experimental result is statistically significant And what is the logic behind that?
Observed result very unlikely to occur by chance (random assignment) alone (if dolphin therapy was not effective)
![Page 30: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/30.jpg)
Example 2: Yawning
What’s different here? Group sizes not the same
So calculating success proportions more important
Experimental result not significant Lack of surprising-ness is harder for students to spot
than surprising-ness Well-stated conclusion is more challenging, subtle
Don’t want to “accept null model”
![Page 31: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/31.jpg)
Example 3: Murderous Nurse?
Murder trial: U.S. vs. Kristin Gilbert Accused of giving patients fatal dose of heart stimulant Data presented for 18 months of 8-hour shifts
Relative risk: 6.34
Gilbert on shift Gilbert not on shift TotalDeath occurred 40 34 74
No death 217 1350 1567Total 257 1384 1641
Proportion 0.156 0.025
![Page 32: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/32.jpg)
Example 3 (cont.)
Structurally the same as dolphin and yawning examples, but with one crucial difference No random assignment to groups
Observational study Allows many potential explanations other than “random
chance” Confounding variables Perhaps she worked intensive care unit or night shift
Is statistical significance still relevant? Yes, to see if “random chance” can plausibly be ruled
out as an explanation Some statisticians disagree
![Page 33: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/33.jpg)
Example 4: Native Californians? What’s different here? Not random assignment to groups Independent random sampling from
populations So …
Scope of conclusions differs Generalize to larger populations, but no cause/effect
conclusions Use different kind of randomness in simulation
To model use of randomness in data collection
![Page 34: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/34.jpg)
3434
Example 1: Lingering sleep deprivation? Does sleep deprivation have harmful effects
on cognitive functioning three days later? 21 subjects; random assignment
Core question of inference: Is such an extreme difference unlikely to occur by
chance (random assignment) alone (if there were no treatment effect)?
improvement
sleep c
onditio
n
4032241680-8-16
deprived
unrestricted
![Page 35: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/35.jpg)
3535
One approach
Calculate test statistic, p-value from approximate sampling distribution
68.2
93.5
92.15
1073.14
1117.12
90.382.1922
2
22
1
21
21
ns
ns
xxt
008.68.2Pr ? tvaluep
![Page 36: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/36.jpg)
3636
Randomization approach
Simulate randomization process many times under null model, see how often such an extreme result (difference in group means) occurs
Start with tactile simulation using index cards Write each “score” on a card Shuffle the cards Randomly deal out 11 for deprived group, 10 for unrestricted
group Calculate difference in group means Repeat many times
![Page 37: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/37.jpg)
Example 1 Sleep deprivation (cont.) Conclusion: Fairly strong evidence that sleep
deprivation produces lower improvements, on average, even three days later Justifcation: Experimental results as extreme as
those in the actual study would be quite unlikely to occur by chance alone, if there were no effect of the sleep deprivation
![Page 38: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/38.jpg)
Exact randomization distribution
Exact p-value 2533/352716 = .0072
![Page 39: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/39.jpg)
Example 2: Age discrimination? Employee ages:
25, 33, 35, 38, 48, 55, 55, 55, 56, 64 Fired employee ages in bold:
25, 33, 35, 38, 48, 55, 55, 55, 56, 64 Robert Martin: 55 years old Do the data provide evidence that the firing
process was not “random” How unlikely is it that a “random” firing process
would produce such a large average age?
![Page 40: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/40.jpg)
Exact permutation distribution Exact p-value: 6 / 120 = .05
56524844403632
20
15
10
5
0
mean age (fired)
Frequency
![Page 41: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/41.jpg)
Example 3: Memorizing letters You will be given a string of 30 letters Memorize as many as you can, in order, in 20
seconds
![Page 42: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/42.jpg)
Confidence Intervals based on Randomization Tests (Quantitative) Invert randomization test
Subtract from all subjects in group B, re-randomize, add from all subjects in group B, compare to observed difference
Similar to binomial example (kissing study) Get standard error from randomization distribution
and use observed +- 2 SEs Get percentiles from randomization distribution and
use observed +- percentiles t-interval Bootstrapping
![Page 43: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/43.jpg)
Series of Lab Assignments
Lab 1: Helper/Hinderer (Binomial test) Lab 2: Dolphin Therapy (2x2 table) Lab 3: Textbook prices (matched pairs from normal
population) or JFK/JFKC (randomization on quantitative variable)
Lab 4: Random Babies Lab 5: One-sample z-test for proportion (Reeses
Pieces) Lab 6: Sleepless nights (t-test, confidence interval) Lab 7: Sleep deprivation (randomization test) Lab 8: Study Hours and GPA (regression with
simulation and Minitab output)
![Page 44: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/44.jpg)
Random Babies
Suppose that 4 mothers give birth to baby boys at the same hospital on the same night
Hospital staff returns babies to mothers at random!
How likely is it that … … nobody gets the right baby? … everyone gets the right baby? …
![Page 45: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/45.jpg)
Random Babies
Last Names First Names Jones Jerry Miller Marvin Smith Sam Williams Willy
![Page 46: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/46.jpg)
Random Babies
Last Names First Names
Jones Marvin
Miller
Smith
Williams
![Page 47: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/47.jpg)
Random Babies
Last Names First Names
Jones Marvin
Miller Willy
Smith
Williams
![Page 48: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/48.jpg)
Random Babies
Last Names First Names
Jones Marvin
Miller Willy
Smith Sam
Williams
![Page 49: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/49.jpg)
Random Babies
Last Names First Names
Jones Marvin
Miller Willy
Smith Sam 1 match
Williams Jerry
![Page 50: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/50.jpg)
Random Babies
1234 1243 1324 1342 1423 1432
2134 2143 2314 2341 2413 2431
3124 3142 3214 3241 3412 3421
4123 4132 4213 4231 4312 4321
![Page 51: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/51.jpg)
Random Babies
1234 1243 1324 1342 1423 1432
4 2 2 1 1 2
2134 2143 2314 2341 2413 2431
2 0 1 0 0 1
3124 3142 3214 3241 3412 3421
1 0 2 1 0 0
4123 4132 4213 4231 4312 4321
0 1 1 2 0 0
![Page 52: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/52.jpg)
Random Babies
Probability distribution 0 matches: 9/24=3/8 1 match: 8/24=1/3 2 matches: 6/24=1/4 3 matches: 0 4 matches: 1/24
Expected value 0(9/24)+1(8/24)+2(6/24)+3(0)+4(1/24)=1
![Page 53: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/53.jpg)
Random Babies
First simulate, then do theoretical analysis Able to list sample space Short cuts when are actually equally likely Simple, fun applications of basic probability
![Page 54: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/54.jpg)
Naming Presidents
List as many U.S. Presidents as you can in reverse chronological order (starting with the current President)
Score = # correct before first error
![Page 55: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/55.jpg)
Naming Presidents
Obama Bush Clinton BushReagan Carter Ford NixonJohnson Kennedy Eisenhower TrumanRoosevelt Hoover Coolidge HardingWilson Taft Roosevelt McKinleyCleveland Harrison Cleveland ArthurGarfield Hayes Grant JohnsonLincoln Buchanan Pierce FillmoreTaylor Polk Tyler HarrisonVan Buren Jackson Adams MonroeMadison Jefferson Adams Washington
![Page 56: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/56.jpg)
Naming Presidents
Use sample data to determine 90% t-interval What percentage of sample values are within
this interval? Is this close to 90%?
![Page 57: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/57.jpg)
Naming Presidents
Lessons: Confidence interval is not a prediction interval Pay attention to what the parameter (“it”) is
![Page 58: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/58.jpg)
5858
Advantages
You can do this at beginning of course Then repeat for new scenarios with more richness Spiraling could lead to deeper conceptual understanding
Emphasizes scope of conclusions to be drawn from randomized experiments vs. observational studies
Makes clear that “inference” goes beyond data in hand Very powerful, easily generalized
Flexibility in choice of test statistic (e.g. medians, odds ratio) Generalize to more than two groups
Takes advantage of modern computing power
![Page 59: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/59.jpg)
59
Question #1
Should we match type of randomness in simulation to role of randomness in data collection? Major goal: Recognize distinction between random
assignment and random sampling, and the conclusions that each permit
Or should we stick to “one crank” (always re-randomize) in the analysis, for simplicity’s sake?
For example, with 2×2 table, always fix both margins, or only fix one margin (random samples from two independent groups), or fix neither margin (random sampling from one group, then cross-classifying)
![Page 60: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/60.jpg)
6060
Question #2
What about interval estimation? Estimating effect size at least as important as assessing
significance How to introduce this?
Invert test Test “all” possible values of parameter, see which do not put
observed result in tail Easy enough with binomial, but not as obvious how to
introduce this (or if it’s possible) with 2×2 tables Alternative: Estimate +/- margin-of-error
Could estimate margin-of-error with empirical randomization distribution or bootstrap distribution
![Page 61: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/61.jpg)
6161
Question #3
How much bootstrapping to introduce, and at what level of complexity? Use to approximate SE only? Use percentile intervals? Use bias-correction?
Too difficult for Stat 101 students? Provide any helpful insights?
![Page 62: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/62.jpg)
6262
Question #4
What computing tools can help students to focus on understanding ideas? While providing powerful, generalizable tool?
Some possibilities Java applets, Flash
Very visual, contextual, conceptual; less generalizable Minitab
Provide students with macros? Or ask them to edit? Or ask them to write their own?
R Need simpler interface?
Other packages? StatCrunch, JMP have been adding resampling capabilities
![Page 63: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/63.jpg)
6363
Question #5
What about normal-based methods? Do not ignore them!
Introduce after students have gained experience with randomization-based methods
Students will see t-tests in other courses, research literature
Process of standardization has inherent value A common shape often arises for empirical
randomization/sampling distributions Duh!
![Page 64: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/64.jpg)
6464
Assessment: Developing instruments that assess … Conceptual understanding of core logic of inference
Jargon-free multiple choice questions on interpretation, effect size, etc.
“Interpret this p-value in context”: probability of observed data, or more extreme, under randomness, if null model is true
Ability to apply to new studies, scenarios Define null model, design simulation, draw conclusion More complicated scenarios (e.g., compare 3 groups)
![Page 65: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/65.jpg)
Understanding of components of activity/simulation Designed for use after an in-class activity using
simulation. Example Questions
What did the cards represent? What did shuffling and dealing the cards represent? What implicit assumption about the two groups did the
shuffling of cards represent? What observational units were represented by the dots on
the dotplot? Why did we count the number of repetitions with 10 or
more “successes” (that is, why 10)?
65
![Page 66: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/66.jpg)
66
Conducting small classroom experiments Research Questions:
Start with study that has with significant result or non? Start with binomial setting or 2×2 table? Do tactile simulations add value beyond computer
ones? Do demonstrations of simulations provide less value
than student-conducted simulations?
![Page 67: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/67.jpg)
6767
Conclusions/Lessons Learned
Put core logic of inference at center Normal-based methods obscure this logic Develop students’ understanding with
randomization-based inference Emphasize connections among
Randomness in design of study Inference procedure Scope of conclusions
But more difficult than initially anticipated “Devil is in the details”
![Page 68: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/68.jpg)
Conclusions/Lessons Learned
Emphasize purpose of simulation Don’t overlook null model in the simulation Simulation vs. Real study Plausible vs. Possible
How much worry about being a tail probability How much worry about p-value = probability
that null hypothesis is true
68
![Page 69: Introducing Concepts of Statistical Inference Beth Chance, John Holcomb, Allan Rossman Cal Poly – San Luis Obispo, Cleveland State University](https://reader035.vdocuments.mx/reader035/viewer/2022081519/56649d365503460f94a0dc46/html5/thumbnails/69.jpg)
6969
Thanks very much!
Thanks to NSF (DUE-CCLI #0633349) Thanks to George Cobb, advisory group More information: http://statweb.calpoly.edu/csi
Draft modules, assessment instruments Questions/comments: