Microscopes and Telescopes:A Research Agenda for Studying Excellence and Equity in Online Learning Environments

Justin ReichDick Murnane, John Willett, Bob Wolf (Boston Consulting Group, retired), Hunter Gehlbach

MIT Lifelong Kindergarten Group 2010

1

AgendaI. Explore Nested Structure of Web 2.0

DataII. Schema of Web 2.0 Research MethodsIII. Prospectus for Researching Classroom

WikisI. Efficacy: How do we make them good?II. Equity: Do only certain kids get the good

ones?IV. Dip in to the data

2

What are wikis and why should we study them?• DEFINITION: Wikis are Web sites where any one can

edit any page at any time• GROWING IMPORT: Use of Web 2.0 tools and networks

(blogs, wikis, social networks, discussion forums) is growing exponentially in education, and transforming our society

• STUDENT-CENTERED: A democratic, citizen-centered ideology is “baked into” wiki architecture making them promising platforms for teaching 21st century skills

• ***NESTED, RETROSPECTIVE CORPUS***: Wikis preserve their entire history of activity in a nested data structure that allows research of infinitely variable granularity (like Google Earth) 3

4

What are wikis and why should we study them?• DEFINITION: Wikis are Web sites where any one can

edit any page at any time• GROWING IMPORT: Use of online learning

environments (blogs, wikis, social networks, discussion forums) is growing exponentially in education, and transforming our society

• STUDENT-CENTERED: A democratic, citizen-centered ideology is “baked into” wiki architecture making them promising platforms for teaching 21st century skills

• ***VAST, NESTED, RETROSPECTIVE CORPUS***: Wikis preserve their entire history of activity in a nested data structure that allows research of infinitely variable granularity (like Google Earth)

5

What do real wikis look like?

6

7

8

9/5/07 10/25/07 12/14/07 2/2/08 3/23/08 5/12/08 7/1/08 8/20/080

500

1000

1500

2000

2500

3000

Page SavesViews

Data from a single wiki

9

10

ALLPS

0

10

20

30

day

0 100 200 300 400 500 600 700 800 900

Day

Day

Pag

e S

ave

Pag

e S

ave

Moving average of wiki development in page saves

11

1,000K

10K

1,000

100

10

1

100K

# of Cases

Days Weeks Months Years

Design Research

Interviews

Surveys

Simulations

Semantic Analysis

Content Analysis

Discursive Analysis

Observational Research

Duration of data collection and capture

Time/Scale Web 2.0 Research State Space Modeling

Usage Statistics

Classroom Wiki Research Questions

• How do we make them good?– What best practices, attitudes and resources

produce wiki learning environments that promote and nurture 21st century skills?

• Do only certain kids get the good ones?– What is the distribution of high quality wikis across

schools serving different student populations?

12

13

Initial Quantitative Analysis to

Develop Sampling Strategy and Measurement

Protocol

Classroom Observations and Teacher Interviews to

Understand Wiki Practices

Development of

Wiki Quality Instrument

and Protocol

Content Evaluation of Wikis Using

Quality Instruments to

Answer Research Questions

Wiki User Surveys

Development of Semantic Wiki Quality Instrument

Literature Review of CSCL and 21st C. Skill

Scholarship

Path Diagram of Wiki Research

Excellence:How does wiki quality vary by

teacher attitudes, practices, and resources?

Are good wikis born or made?

Equity:How does wiki quality vary by

school SES variables like %FRPL, Per Pupil Expenditure, Title I status, or measures of racial

composition