emoocs 2014 research track 3_zahn

Carmen Zahn, Karsten Krauskopf, Jonas Kiener & Friedrich W. Hesse

Designing Video for Massive Open Online-EducationResearch on the effects of collaborative tasks framing the use of video

video source: https://www.coursera.org/course/inf4oec

Theory Background: Conceptual challenges from a learner-centered perspective

Experimental Study: Effects emerging from different tasks framing the usage ofonline-video (tools) on student collaboration and learning

Discussion: Future Research relating to MOOCs

Carmen Zahn 10.02.2014 2

Conceptual challenges in «MOOC-Designspaces»

(Taxonomy: General Features of Moocs, Schneider, 2013)

Carmen Zahn 10.02.2014 3

InstructionE.g. Lecture, readings, videos, interactivity

ContentE.g. Domain, Modularization, Pacing

AssessmentE.g. In-video quizzes, homework, group projects

CommunityE.g. Discussion board, socialmedia, video chat, text chat

Design challenge # 1

(Taxonomy: General Features of Moocs, Schneider, 2013)

«Design instructional video/video lectures that support individual understanding»

Carmen Zahn 10.02.2014 4

InstructionE.g. Lecture, readings, videos, interactivity

ContentE.g. Domain, Modularization, Pacing

Design Challenge # 2

(Taxonomy: General Features of Moocs, Schneider, 2013)

«Connect video materials to meaningful learning experiences and the community»

= Effective (collaborative) tasks & assessment»

Carmen Zahn 10.02.2014 5

AssessmentE.g. In-video quizzes, homework, group projects

CommunityE.g. Discussion board, socialmedia, video chat, text chat

Related research on video-based learning

Showing video is not the most efficient way to activate learners` mental effort -

especially problematic for learning of complex topics (e.g., compared to text, Salomon, 1984)

Digital video tools (e.g. highlighting, editing functions, annotation tools, hypervideo)

successfully enable learning activities necessary for learning of complex topics (e.g.,

Schwan & Riempp, 2004; Spiro, et al. 2007; Zahn, Pea, et al., 2005)

Embedding video in collaborative and creative tasks is a powerful strategy to stimulate

and enhance learning from video (e.g., Schwartz & Hartmann, 2007; Zahn et al., 2013)

6

More related research … in a nutshell

Video collaboration tools (e.g., hypervideo with discussion tool, webspaces for co-editing and sharing comments, etc.) improve learning of complex topics (e.g., Goldman,

2004; Zahn, Pea, Hesse & Rosen, 2010)

…not only in lab, but in real learning scenarios of school-based and university education (Pea et al., 2004; Pea et al., 2006; Stahl, Zahn, Finke, 2005; Zahn, Krauskopf, Hesse & Pea, 2010)

Extra-support for social interactions when learning with video tools increases success (e.g., Zahn, Krauskopf, Hesse & Pea, 2012)

7

Theory Background: Conceptual challenges from a learner-centered perspective

Experimental Study: Effects emerging from different tasks framing the usage ofonline-video (tools) on student collaboration and learning

Discussion:Future Research relating to MOOCs

Carmen Zahn 10.02.2014 8

Goals of the study

Test if specific video tools can be helpful for collaborative online-learning in a

complex domain like history

Initial exploration of the question “Which effects can we expect from different tasks

framing the use of those video tools on collaborative online-learning and outcome?”

9

Experimental Learning Environment

10

WebDIVER TM by Roy Pea, Stanford University (Pea et al., 2004)

Task and Experimental Design

Students used WebDiverTM for discussion and “multimedia” assignments

Online-Lesson: German History (“Berlin Airlift”)

Video & Materials: Original Newsreel (historical source) & Textbook information

Learning Goal: Integration of content knowledge and an understanding of the

propagandistic functions of the newsreel = Evaluation of historical source of evidence (e.g., Lindsay et al. 2013)

Experimental Variation: Discussion vs. design tasks framing video use

Group 1:…analyse and comment on the video within an online-discussion

Group 2:…analyse and comment on video in order to design a hypertext-like product, so that other

student learners can come to a good understanding of the newsreel (multimedia design)

11

Measures for Collaboration and Learning

12

Level Variable Measure

1: Cognitive learning outcome

History content knowledge acquisition

Factual Knowledge Test Picture Recognition Test

2: Surface level effects on collaboration and learning

Performance, collaboration and learning - quantity

Number of panels created in partnership Number of comments Length of commentsCollaboration index

3: Deeper level effects on collaboration and learning

Performance, collaboration and learning - quality

Number of panels referring to details Number of utterances in comments addressing historical contentNumber of utterances in comments addressing filmic styleNumber of utterances in commentsintegrating aspects of historical content and filmic style

MC-Tests, analyses of WebDiverTM – panels & comments and screen-videos

Level 1: Cognitive Outcome

13

Design-condition

(n = 19 dyads)

Discussion-condition

(n = 17 dyads)

Total

(N = 36 dyads)t-Test Effect size

Indicator M SD M SD M SD t(34) p d

Factual knowledge 33.4 2.5 34.0 1.7 33.7 2.1 -0.85 .40

Picture recognition

25.7 1.3 24.9 1.4 25.3 1.3 1.79 .08 0.7

In sum: Similar cognitive outcome from both tasks, marginal on picture recognition,

medium effect size (Cohen, 1988)

Level 2: Surface Level Collaborative Activity

14

Design-condition

(n = 19 dyads)

Discussion-condition

(n = 17 dyads)

Total

(N = 36 dyads)

t-Test Effect size

M SD M SD M SD t(10) p d

Number of comments 28 8.7 36.1 10.6 32 9.6 -2.5 .02** 0.8

Length of comments (in words) 426.7 161.1 610.6 290.6 518.7 225.9 -2.4 .02** 0.8

n = 6 dyads n = 6 dyads n = 12 dyads

Collaboration index 12.3 10.6 33.6 21.9 22.6 16.3 -2.08 .07+ 0.7

Dives created in partnership 4.2 4.2 12.0 6.9 8.1 5.6 -2.26 .05* 1.4

In sum: Discussion group = **More collaborative activities than design group

Level 3: Deeper Level Collaborative Activity

15

Design-condition

(n = 6 dyads)

Discussion-condition

(n = 4 dyads)

Total

(n = 10 dyads)

t-Test Effect size

Indicator M SD M SD M SD t(8) p dUtterances integrating historic and filmic aspects 5.2 1.7 3.3 1.2 4.3 1.5 2.23 .05* 1.3

Utterances addressing historical content 5.8 3.1 4.6 1.7 5.2 2.4 0.87 .41

Utterances addressing filmic style 12.7 5.2 18.1 2.9 15.4 4.1 -2.24 .05* 1.3

n = 19 n = 17 N = 36

Number of details 2.2 2.8 0.6 1.5 1.4 2.2 116a .16

In sum: Design group *stronger in discussing an integrative view of historic content and

filmic style (source evaluation!), while discussion group focuses on one aspect

Summary

Under the surface of apparently similar cognitive outcomes in multiple choice tests

asking for factual knowledge, fine-grained differences in important other aspects of

online learning became explicit:

While the discussion task stimulated significantly more collaborative surface activity

than the design task…

…the design task stimulated for more knowledge intensive elaboration than the

discussion task, such as an integrative view on video content and style

(Please remember the learning goals in our domain of history!)

16

Implications of the Results of this Study

If we know that tasks can influence “what we see” in an online-video Teachers can

decide according to their specific teaching goals which task is to prefer (discussion or

multimedia design assignments)

We might only speculate about how such differences (obtained in an ad-hoc

experiment, not MOOC) might add up in longer-lasting courses (e.g., tasks over

several weeks time) and then extend to cognitive outcomes (e.g., multiple choice

quizzes).

17

General Implications from Video-Research

Recommendations beyond design of instructional videos

Video editing and annotation tools provided with video lectures and video

sources make video a true “working medium” which students can use actively

and collaboratively for learning.

Think carefully about integration of video tools and tasks.

Research on MOOC video effects is definitely needed!

18

Thank you for your attention!

Carmen Zahn 10.02.2014 19

Theory Background: Conceptual challenges from a learner-centered perspective

Experimental Study: Effects emerging from different tasks framing the usage ofonline-video (tools) on student collaboration and learning

Discussion: Future Research relating to MOOCs

Carmen Zahn 10.02.2014 20