from data to information - olcinfo2.onlinelearningconsortium.org/rs/897-csm-305/... · from data to...
TRANSCRIPT
![Page 1: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/1.jpg)
From Data to Information in Online and Blended Learning Research
Chuck Dziuban Patsy Moskal
Research Initiative for Teaching EffectivenessUniversity of Central Florida
![Page 2: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/2.jpg)
Some of Chuck and Patsy’s Chapters
• Data Analysis
• Scholarship of Teaching and Learning
• Longitudinal Evaluation
• Big Data
• The Future
![Page 3: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/3.jpg)
Data to InformationResearch
and Data
Analysis
Interpretation
JudgmentInformation
![Page 4: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/4.jpg)
Data Analysis
![Page 5: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/5.jpg)
What do you do when someone asks:
“How large does my sample have to be?”
![Page 6: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/6.jpg)
Hypothesis testing answers this question
What is the chance of my sample data
The null hypothesis is true?
WHEN
H0
![Page 7: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/7.jpg)
Statistical (Classical) Hypothesis Tests are a Function of 3 Things:
1) Significance Level .05? .01? …or something else?
2) Sample Size
Tiny? Small? Medium? Large? Huge?3) Some Effect Size
A difference that means something to me∆1 –Doesn’t matter∆2 –Really important to me
![Page 8: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/8.jpg)
How much is enough?
∆2I don’t care about this
I care about this
∆1
![Page 9: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/9.jpg)
Statistical Significance Testing (SD = 15)Sample
Size
27502500225020001750150012501000750500
x1=100x2=101ES=.06
.01
.02
.03
.04
.05
.07
.10
.14
.20
.29
![Page 10: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/10.jpg)
So the strategy is…
1) Pick ∆2 first à This is important to me
2) Then pick a significance level .05, .01, or something else
3) Pick a sample size that will catch ∆2 but not ∆1
![Page 11: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/11.jpg)
Data Analysis Resources
• National Research Center for Distance Education and Technological Advancements (DETA) https://uwm.edu/deta/
• Ferguson, G. A., & Takane, Y. (1981). Statistical analysis in psychology and education (5th ed.). New York: McGraw-Hill.
• Nie, N. H. (1975). SPSS: Statistical package for the social sciences (2nd ed.). New York: McGraw-Hill.
• Practical Assessment, Research and Evaluation (PARE).
![Page 12: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/12.jpg)
Scholarship of Teaching and Learning(SoTL)
![Page 13: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/13.jpg)
So…what is SoTL?
• Involves…• Systematic reflection of the
teaching process• Research• Dissemination of the teaching
process and its impact on student learning
![Page 14: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/14.jpg)
A research context for SoTL
![Page 15: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/15.jpg)
But do they want us in their world?• Tim Brown – Communications• Amanda Groff – Anthropology
Are students interested in class tweets?
Not really
Prefer official channel, e-mails, CMS
Brown & Groff (2011)
![Page 16: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/16.jpg)
Word clouds online• Beatriz Reyes-Foster – Anthropology• Can word clouds help with concept formation?
deNoyelles & Reyes-Foster (2014)
![Page 17: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/17.jpg)
Susan B. Anthony
![Page 18: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/18.jpg)
John Lewis
![Page 19: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/19.jpg)
For more information on SoTL…
![Page 20: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/20.jpg)
Journals that publish SoTL
![Page 21: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/21.jpg)
Longitudinal Evaluation
![Page 22: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/22.jpg)
Longitudinal Evaluation
• Advantages• Allows for examination of trends over time• Can potentially follow cohorts• Aids in continuous quality improvement• Helps identify in “near real” time
• Challenges• Requires time and continuity!
Repeated studies, carried out over a period of time
![Page 23: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/23.jpg)
Students’ Desktop vs Laptop use from 2006-2009
71.0%
59.4%
49.7%44.0%
65.4%
72.8%
82.1%88.3%
40.0%
50.0%
60.0%
70.0%
80.0%
90.0%
100.0%
2006 2007 2008 2009N= (13,641) N= (12,861) N= (11,730) N= (10,180)
Desktop
Laptop
Note: Adapted from The ECAR study of undergraduate students and information technology, 2009,by S. D. Smith, G. Salaway, J. B. Caruso, & R. N. Katz, 2009, Boulder, CO: ECAR, p. 43
![Page 24: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/24.jpg)
Student Mobile Technology Use & Importance: 2012-2014
ECAR Study of Students and Information Technology (Dahlstrom & Bichsel, 2014)
![Page 25: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/25.jpg)
Student Perceptions of Instruction:Overall Excellent Ratings
50 49 52 53 54 5454 53 55 56 57 54
0
20
40
60
80
100
2009 2010 2011 2012 2013 2014
F2F (n=1,044,164) Blended (n=94,045)
![Page 26: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/26.jpg)
Student Perception of Instruction:The Form Items
• Organization of course • Explanation of requirements, grading, & expectations• Communication• Respect• Stimulation of interest• Creation of learning environment • Feedback• Aid in student success • Overall effectiveness of instructor• + 2 open-ended questions
![Page 27: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/27.jpg)
Achieve course objectives
Create learning environment
Then...The probability of an overall rating of Excellent = .99
If...
A decision rule for the probability of a faculty member receiving an overall rating of Excellent
Communicate ideas
n=58,156
Excellent Very Good Good Fair Poor
![Page 28: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/28.jpg)
A comparison of excellent ratings by course modalityunadjusted & adjusted for instructors satisfying the rule (n=431,261)
Course Modality Overall % Excellent
If Rule 1 % Excellent
Blended 55 99
Fully Online 57 99
F2F 54 99
Lecture Capture 51 99
Blended LC 48 99
![Page 29: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/29.jpg)
Longitudinal Resources
• 2015 Online Report Card - Tracking Online Education in the United States• http://onlinelearningconsortium.org/read/online-report-card-tracking-
online-education-united-states-2015/• ECAR Student and Technology Research Study
• https://library.educause.edu/resources/2016/6/2016-student-and-technology-research-study
• Pew Research Center on Internet, Science and Technology• http://www.pewinternet.org/
![Page 30: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/30.jpg)
Big Data
![Page 31: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/31.jpg)
The OldSTATISTICSSampling Estimation Hypothesis
Testing
![Page 32: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/32.jpg)
The New
BIG DATAModeling Prediction Machine
Learning
![Page 33: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/33.jpg)
Big Data and Statistical Thinking
Blended(n=53,476)Face to Face (n=726,342)Online (n=121,257)
Blended – Face to Face Blended – Online Online – Face to Face
Modality Mean P
Modality Bonferroni Effect Size
4.194.114.10
P = .000
= .000= .006= .013
.075
.093
.009
![Page 34: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/34.jpg)
Possibilities of Big Data• Identify strong and weak relationships• Develop useful if-then decision rules• Conduct network analysis• Identify affiliated classification groups• Discover patterns• Detect underlying clusters• Develop association rules• Construct new variables• Work with several variables simultaneously
![Page 35: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/35.jpg)
Big Data Resources• Levitt, Steven D., and Stephen J Dubner. Think Like a
Freak: The Authors of Freakonomics Offer to Retrain Your Brain. First edition. William Morrow, an imprint of HarperCollinsPublishers, 2014. *
• Lawson, J. (2015). Data science in higher education: Step-by-step machine learning for institutional researchers.
• Silver, N. (2012). The signal and the noise: Why so many predictions fail--but some don't. New York: Penguin Press.*
![Page 36: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/36.jpg)
The Future
![Page 37: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/37.jpg)
• Changing baselines• The speed of light• Complexity• Uncertain mediation• Being wrong• Beware of false positives• Research context• Collaboration
![Page 38: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/38.jpg)
The Future Resources• Complexity Academy http://complexityacademy.io/
• Schulz, K. (2010). Being wrong: Adventures in the margin of error. New York: Ecco. *
• Mullainathan, S., & Shafir, E. (2014). Scarcity: Why having too little means so much. Picador.
• Floridi, L. (2014). The fourth revolution: How the infosphere is reshaping human reality. Oxford University Press. *
![Page 39: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/39.jpg)
Questions?
![Page 40: From Data to Information - OLCinfo2.onlinelearningconsortium.org/rs/897-CSM-305/... · From Data to Information in Online and Blended Learning Research Chuck Dziuban Patsy Moskal](https://reader033.vdocuments.mx/reader033/viewer/2022042304/5ecf307468442779ed46531b/html5/thumbnails/40.jpg)
Research Initiative for Teaching Effectiveness
For more information contact:
Dr. Chuck Dziuban(407) 823-5478
Dr. Patsy Moskal(407) 823-0283
http://rite.ucf.edu