video annotation effects upon learning and...
TRANSCRIPT
VIDEO ANNOTATION EFFECTS UPON LEARNING AND METACOGNITIVE MONITORING
By
AARON OWEN THOMAS
A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY
UNIVERSITY OF FLORIDA
2016
© 2016 Aaron Owen Thomas
To Claudia, Isabella, and Julie
4
ACKNOWLEDGMENTS
I would like to thank the doctoral committee for their dedication and assistance
with every aspect of this project. I am indebted to Dr. Pavlo Antonenko for modeling the
highest standards of scientific inquiry and patiently helping me improve the quality of the
project. Without the guidance and support of Dr. David Therriault, this project would not
exist which was first conceived in his doctoral seminar. Dr. Carole Beal, early in the
development of this project, recognized the implications of segmentation and pausing of
the video timeline, which, in turn, influenced the design and functionality of the video
players for this project. Dr. Kent Crippen, throughout asked essential questions that kept
this project focused upon the primary purpose of educational research, namely how to
improve learning outcomes. Any and all errors, however, are solely my own.
In addition, this research could not have been conducted without the assistance of
Dr. Keith Thiede who provided the video scripts that formed the basis for the production
of the instructional videos and corresponding test questions. Additional thanks to Dr.
Kristen Apraiz, Dr. Kristy Boyer, Li Cheng, Robert Davis, James Kline, Elizabeth Kenney,
and Shilpa Sahay for contributing to participant recruitment. I would also like to thank AJ
Kleinheksel, Hope Kelly, Mark McCallister, Catherine Coe, and Brenda Lee for lending
support throughout the doctoral journey. I also am thankful for the encouragement and
help that I have received from Dr. Nicola Wayer and Ben Campbell throughout all of my
studies and professional pursuits as well.
In conclusion, I wish to thank my wife, Julie Thomas, for her support and my two
children, Isabella and Claudia Thomas, for their patience.
5
TABLE OF CONTENTS page
ACKNOWLEDGMENTS ................................................................................................... 4
LIST OF TABLES ............................................................................................................. 8
LIST OF FIGURES ........................................................................................................... 9
ABSTRACT .................................................................................................................... 10
CHAPTER
1 METACOGNITIVE CONSEQUENCES OF VIDEO ANNOTATION ......................... 12
Introduction .............................................................................................................. 12Theoretical Foundations .......................................................................................... 14Metacognition .......................................................................................................... 15Discrepancy-Reduction Model ................................................................................. 16Comprehension ....................................................................................................... 18Metacomprehension ................................................................................................ 20Multimedia and Metacognitive Monitoring ............................................................... 24Multimodality and Transience .................................................................................. 26Video Annotation and Segmentation ....................................................................... 27Segmentation and Metacognitive Monitoring: Interesting Interactions .................... 30Research on Video Annotation Systems ................................................................. 31Implications and Directions for Future Research ..................................................... 32
2 METACOGNITIVE CONSEQUENCES OF VIDEO SEGMENTATION ................... 34
Introduction .............................................................................................................. 34What is Video? ......................................................................................................... 35System-Controlled and Learner-Controlled Video Segmentation ............................ 35System-Controlled and Learner-Controlled Video Annotation ................................. 37Video-Based Learning as Self-Regulated Learning ................................................. 38Metacognition .......................................................................................................... 38Discrepancy-Reduction Model ................................................................................. 39Factors Impacting Metacognitive Monitoring Accuracy ........................................... 40Comprehension ....................................................................................................... 40Metacomprehension ................................................................................................ 42Model of Metacomprehension Accuracy .................................................................. 42Multimedia and Metacognitive Monitoring ............................................................... 44Experiment 1 ............................................................................................................ 45
Hypotheses ....................................................................................................... 45Random segmentation ................................................................................ 46Paragraph segmentation ............................................................................ 46Learner-controlled segmentation ................................................................ 47
6
No segmentation control ............................................................................. 48Method ..................................................................................................................... 48
Participants ........................................................................................................ 48Design ............................................................................................................... 49
Materials .................................................................................................................. 50Video Scripts ..................................................................................................... 50Hardware ........................................................................................................... 50Segmentation Conditions .................................................................................. 51Images, Animations, and Callouts ..................................................................... 52Judgments ......................................................................................................... 52Recall and Inference Performance .................................................................... 53
Procedure ................................................................................................................ 54Results ..................................................................................................................... 55
Recall and Inference Test Performance and Metacomprehension Accuracy ... 55Random Segmentation Effects .......................................................................... 57Paragraph Segmentation Effects ...................................................................... 58Learner-Controlled Effects ................................................................................ 59No Segmentation Effects (Control) .................................................................... 60
Discussion ............................................................................................................... 61Random Segmentation ...................................................................................... 61Paragraph Segmentation .................................................................................. 61Learner-Controlled Segmentation ..................................................................... 64No Segmentation ............................................................................................... 66Conclusion ......................................................................................................... 67
3 METACOGNITIVE CONSEQUENCES OF VIDEO ANNOTATION ......................... 72
Experiment 2 ............................................................................................................ 72Hypotheses ....................................................................................................... 72
Split-attention effects .................................................................................. 73Textbase and situation model disruption .................................................... 73Annotation effects ....................................................................................... 74Immediate annotation effects ...................................................................... 74
Method ..................................................................................................................... 75Participants ........................................................................................................ 75Design ............................................................................................................... 75
Materials and Procedure .......................................................................................... 76Results ..................................................................................................................... 78
Recall and Inference Test Performance and Metacomprehension Accuracy ... 78Textbase and Situation Model Disruption Effects .............................................. 78Metamemory and Metacomprehension Accuracy ............................................. 79Immediate Annotation Effects upon Metacomprehension ................................. 80Interactions between Experiment 1 and Experiment 2. ..................................... 80
Discussion ............................................................................................................... 82Comparison of Experiment 1 and Experiment 2 ................................................ 84Scientific and Practical Significance .................................................................. 87Limitations ......................................................................................................... 89
7
Conclusions ....................................................................................................... 89 APPENDIX
A EXPERIMENT 1 CONSENT FORM ........................................................................ 96
B EXPERIMENT 2 CONSENT FORM ........................................................................ 98
C FOUR VIDEO SCRIPTS ........................................................................................ 100
LIST OF REFERENCES .............................................................................................. 118
BIOGRAPHICAL SKETCH ........................................................................................... 127
8
LIST OF TABLES
Table page 2-1 Mean Test Scores and Judgment Magnitudes for Experiment 1. ....................... 69
2-2 Post Hoc Paired-T Test Comparisons ................................................................. 70
2-3 Metamemory and Metacomprehension Accuracy ............................................... 71
2-4 Relative Accuracy for POP for Recall and Inference ........................................... 71
3-1 Mean Test Scores and Judgment Magnitudes for Experiment 2. ....................... 92
3-2 Relative Metamemory and Metacomprehension Accuracy for Experiment 2 ...... 93
3-3 Relative Accuracy for POP for Recall, Inference, and Total Test Performance for Experiment 2 .................................................................................................. 94
9
LIST OF FIGURES
Figure page 2-1 Combined screenshot of the video player screen and subsequent
segmentation screen representative of both the random and paragraph segmentation conditions. ..................................................................................... 68
2-2 Combined screenshot of learner-controlled video screen and subsequent segmentation screen. .......................................................................................... 68
2-3 Comparison of recall and inference test performance across conditions. ........... 69
3-1 Combined screenshot of video screen and subsequent annotation screen for random and paragraph video annotation conditions. .......................................... 91
3-2 Combined screenshot of learner-controlled video annotation screen and annotation screen. ............................................................................................... 91
3-3 Screenshot of simultaneous video annotation screen. ........................................ 92
3-4 Comparison of recall and inference test performance across conditions. ........... 93
3-5 Comparison of recall test proportional means across experiments. .................... 94
3-6 Comparison of inference test proportional means across experiments. ............. 95
3-7 Comparison of total test proportional means across experiments. ..................... 95
10
Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy
VIDEO ANNOTATION EFFECTS UPON LEARNING AND METACOGNITIVE
MONITORING
By
Aaron Owen Thomas
August 2016
Chair: Pavlo Antonenko Major: Curriculum and Instruction
Video annotation is a developing technology that is beginning to be used in formal
and informal educational settings, yet how various affordances of video annotation
systems impact learning and metacognitive processes is an unexamined question in
both multimedia and metacomprehension literature. The metacomprehension paradigm
provides useful theoretical and methodological tools to generate hypotheses for the
interaction between multimodal media such as video and metacognitive processes.
Based upon a review of relevant literature, there is reason to believe that both learning
and metacognitive monitoring may be hindered in the context of video-based learning
conditions.
In two experiments, students viewed four instructional videos based upon
expository texts; made a judgment of learning for each video, and completed recall and
inference tests for each video. Experiment 1 evaluated the effects of three distinct
segmentation conditions (random, paragraph, and learner-controlled) and a no
segmentation control upon recall test performance, inference test performance, and
relative metacomprehension accuracy. Experiment 2 evaluated the effects of four distinct
annotation conditions (random, paragraph, learner-controlled, and simultaneous) upon
11
recall test performance, inference test performance, and between-subjects
metacomprehension accuracy. Between-subjects metacomprehension accuracy for each
condition was computed in two ways: as a correlation between judgments of learning
and inference test performance and as a correlation between predictions of performance
and inference test performance. Results from Experiment 1 indicated that segmentation
hindered recall and inference test performance. Results from Experiment 2 indicated that
video annotation had divergent effects upon recall and inference test performance.
Across all video annotation conditions, metacomprehension accuracy was low.
These results suggest that disruption of the video timeline either through
segmentation or an interpolated activity such as annotation can lead to a reduction in
metacomprehension accuracy and can result in significant reductions in performance
with the exception of the random video annotation. In the context of expository
multimodal video, non-segmented continuous video appears to provide the greatest
benefits both test performance (recall and inference tests) and metacognitive monitoring
accuracy. Thus, the following two studies suggest that the benefits of video-based
learning may be undermined by varying degrees of segmentation and interactivity in the
form of video annotation.
12
CHAPTER 1 METACOGNITIVE CONSEQUENCES OF VIDEO ANNOTATION
Introduction
While a substantial body of literature has addressed metacognitive monitoring
and control in the context of text-based comprehension (Maki & Berry, 1984; Thiede,
Anderson, & Therriault, 2003; Thiede, Dunlosky, Griffin, & Wiley, 2005), little conceptual
research has examined how video in general and video annotation impacts
metacognitive processes. For nearly thirty years, researchers have recognized the
potential to improve learning through interactive video. Smith (1987) discussed video
annotation as an advanced form of interactivity and reviewed numerous studies on
learning effectiveness concluding that the medium had much potential (n.b. although he
cautioned that rigorous studies examining the effects of video annotation are rare).
Twenty years later, Scheiter and Gerjets (2007) noted a continued lack of
methodological rigor in empirical research concerning interactive video. In spite of
ambiguous results, there is still great optimism that interactive video activities can
become an important tool for teaching and learning (Aubert, Prie, & Cannellas, 2014;
Bossewitch & Preston, 2011). The following article is an attempt to conceptualize the
metacognitive consequences of video annotation and specifically examine how the
metacomprehension paradigm (Dunlosky & Lipko, 2007) and discrepancy reduction
model of self-regulated learning (Butler & Winne, 1995) can enhance both the
theoretical foundations and methodological rigor in research on learning from interactive
video.
Educational uses of video continue to grow and expand (Kaufman & Mohan,
2009). The explosive growth in Massive Open Online Courses (MOOCs) can greatly be
13
attributed to the realization that high quality video lectures and demonstrations by the
world’s leading experts can be streamed at low cost to the learner when compared to
textbook costs (Baggaley, 2013; Guo, Kim, & Rubin, 2014). Online video streaming also
affords far greater interactivity and user control than either print text or the previously
used video formats such a telecasts and educational television (Merkt & Schwan, 2014).
Compared to traditional transient lecture environments, the learner in video-based
learning has the ability to pause, rewind, fast forward, and review selected frames and
clips, and in some cases produce annotations tied to specific portions of the video
player timeline. In addition, interactive video players allow instructional designers to
integrate formative and summative assessments such as multiple choice questions,
short response questions, drag and drop activities, interactive callouts, polls, and active
hyperlinks in the midst of a streaming video.
As video streaming and web development technologies continue to emerge and
influence video production methods, the affordances of interactive technologies in video
learning environments continue to expand and evolve (Pardo et al., 2015). There is
continued optimism that interactive features of 21st century video have the potential to
support self-regulatory monitoring and control processes at any time during the learning
process (Aubert et al., 2014; Greene & Azevedo 2009; Moos 2011). Video annotation
technologies, in particular, are of special interest because of the capacity to act as a
generative learning strategy (Wittrock, 1989) that may increase deep learning as a form
of notetaking (Henk & Stahl, 1985; Kobayashi, 2005).
Video annotation is one of many interactive features currently being employed in
video-based learning. VideoAnt™, a free Google™ application developed at the
14
University of Minnesota, has been widely adopted with over 5,000 new projects and
17,000 individual annotations as of 2010 (Hosack, 2010). Another video annotation
technology, Videonot.es™, has been installed by approximately 357,055 users as of
May 1, 2016 and was designed to integrate with Coursera™, Udacity™, edX™, Khan
Academy™, Vimeo™, and YouTube™. In addition, Lynda.com™, a for-profit online
video-based educational company, reports over 2 million subscribers and 144,000+
instructional videos that are streamed through a web-based player that allows for video
annotation. In conclusion, video annotation tools are becoming an important part of the
learning ecosystem in spite of lack of evidence concerning cognitive and metacognitive
consequences.
Theoretical Foundations
Given the context of video annotation in education and its growing
implementation, it is important to explore the theories, models, and frameworks that can
assist in understanding how video annotation systems may help or hinder cognitive and
metacognitive processes during video-based learning. While there is a robust literature
concerning the impact of multimedia upon learning and cognition (Mayer, 2014), at
present there is little work that has explored the interaction of multimodal instructional
video, video annotation, and metacognitive monitoring. The following sections will
describe metacognition, discrepancy-reduction models of self-regulated learning, and
the metacomprehension paradigm so as to inform a focused discussion of relevant
conceptual literature and identify areas of future research. Theories of comprehension
and metacomprehension, in particular, have been useful in understanding
metacognitive monitoring (Dunlosky & Lipko, 2007) and promise to offer a useful
15
theoretical paradigm and method in examining the interaction between video and
metacognitive processes.
Metacognition
The history of metacognition is long and owes much to the efforts of John Flavell
(1979) who described metacognitive experiences as “any conscious cognitive or
affective experiences that accompany and pertain to any intellectual enterprise” (p.
906). Metacognitive knowledge “consists primarily of knowledge or beliefs about what
factors or variables act and interact in what ways to affect the course and outcome of
the cognitive enterprises” (Flavell, 1979, p. 907). This metacognitive knowledge might
be summed up as the assumptions for a learner’s belief system for how people learn,
study, and manage the learning process. In other words, learners develop their own
assumptions and standards that they will use to evaluate their own learning processes.
The underlying assumption of metacognition theory is that if learners are able
effectively to monitor learning processes, then they will be better able to implement a
change in behavior to better meet their learning goals. Another key assumption is that if
learners have information derived from metacognitive monitoring, they will regulate their
cognition and try to determine what strategy to implement to address learning deficits as
identified by the monitoring process. The quality and effectiveness of metacognitive
monitoring, however, varies according to individual differences and specific learning
conditions (Griffin, Wiley, & Thiede, 2008). The accuracy of metacognitive monitoring is
assumed to impact the quality of metacognitive control in learning (Thiede, 1999;
Thiede, Anderson, & Therriault, 2003). Metacognitive monitoring consists of a learner’s
ability to evaluate his or her cognitive processes effectively, and metacognitive control is
16
the ability to employ the conclusions derived from metacognitive monitoring to change
learning behaviors (Son & Schwartz, 2002).
Discrepancy-Reduction Model
The relationship between metacognitive monitoring and control has been
described and examined in the literature through the discrepancy-reduction model of
self-regulated learning (Butler & Winne, 1995; Dunlosky, Hertzog, Kennedy, & Thiede,
2005; Nelson, Dunlosky, Graf, & Narens, 1994). The discrepancy-reduction model
posits that a learner establishes learning goals, monitors learning levels, and interprets
monitoring data so as to determine whether to terminate study or restudy the topic. If
monitoring information indicates a discrepancy between a learner’s established goals
and current knowledge level, restudy will continue until the current state of learning and
the desired learning goals reach zero (Butler & Winne, 1995). A major assumption of
this model is that accurate monitoring of the learning state is necessary for the
discrepancy-reduction mechanism to function effectively. The efficiency and
effectiveness of this regulatory loop depends upon the accuracy of self-evaluation
judgments, which are commonly called judgments of learning or JOLs in the literature.
As discussed above, metacognitive monitoring can be understood as a
metacognitive experience, but this monitoring process is mediated through what Flavell
(1979) refers to as metacognitive knowledge, which in turn produces cues used to judge
comprehension, recall, and performance levels (Flavell, 1979; Koriat, 1997; Maki,
1998). These cues are both theory-based, namely a learner’s beliefs about learning,
and heuristic-based cues that rely upon a learner’s fluency or ability to access specific
information in long-term or short-term memory employed in the judgment process
(Koriat, 1997).
17
Cue-utilization theory suggests that metacognitive judgments are based upon
inferences that learners make and are “accurate as long as the cues used at the time of
making the judgments are consistent with the factors that affect subsequent
performance” (Koriat, 1997, p. 350). In the context of learning from text, learners employ
memory for details of the text and memory of the situation model (Dunlosky & Thiede,
1998). If a learner bases a judgment of learning upon ability to remember details, this
judgment is likely not based upon an ability to construct a situation model, a micro-world
of causal and inferential relationships. Because the cues derived from memory for text
is not consistent with the situation model, which is the heart of comprehension and
meaning making, judgments of learning are expected to be error-laden and inaccurate.
Cues can introduce error into metacognitive judgments and are often referred to
as heuristics (Serra & Dunlosky, 2010). Heuristic cues, for example, could arise if a
learner’s preconceived beliefs about the efficacy of multimedia produce overconfidence
in future test performance (Serra & Dunlosky, 2010). Other examples of heuristics that
could introduce error into metacognitive judgments include interest in the topic, feelings
of fluency, or mood (Griffin, Jee, & Wiley, 2009; Rawson, Dunlosky, & Thiede, 2000). In
the context of learning from video, it is possible that heuristics such as the belief that
video is easy and text is hard (Salomon, 1984) may impact metacognitive monitoring
which in turn prompts a learner to produce inaccurate JOLs. Attitudes and perceptions
concerning a particular technology may in fact be an important variable to consider with
respect to metacomprehension research as applied to multimedia.
Building upon the cue-utilization theory, researchers have attempted to create
conditions that strengthen consistency between cues and the factors that influence
18
subsequent performance. One effective strategy to increase metacognitive monitoring
accuracy is summarization of text material. Summarization of text material can increase
metacognitive monitoring accuracy but only after a long delay as compared to
immediately producing summaries after reading (Thiede & Anderson, 2003; Thiede et
al., 2003). Delayed-summarization positively impacts metacognitive monitoring because
it focuses a learner upon creating a mental representation of the text that is not
influenced by cues formed from short-term memory. Immediate summarization results in
a mental representation of the text that is often flawed because of an overreliance and
an abundance of remembered details derived from short-term memory. Situation model
cues are more robust memory aids when compared to the surface level cues because
of the effects of short-term memory (Kintsch, Welsch, Schmalhofer, & Zinny, 1990). By
forcing a delay in summarization, greater alignment is achieved between cues and
factors that impact performance.
Comprehension
As discussed above, much of the existing metacognitive research concerns text-
based learning conditions and, as a result, theories of comprehension have been
fundamental to the development of metacognitive monitoring literature. Comprehension,
a complex cognitive process and foundation for critical thinking and problem solving,
has been examined almost exclusively in text-based conditions (McNamara & Magliano,
2009). Although there are numerous theories of comprehension, the Construction-
Integration (CI) model (Kintsch & van Dijk, 1978), the situation model (van Dijk &
Kintsch, 1983), and the mental model (Johnson-Laird, 1983) have provided seminal
foundations for empirical research in the area of comprehension. Although these
models differ in important ways, these models all assume that a reader produces
19
multiple mental representations of the text during the act of reading (Kintsch & Van Dijk,
1978; Kintsch, 1998). Comprehension of text requires both understanding and memory
in order to build or construct a situation-model of the mental representation (Graesser,
Millis, & Zwaan, 1997; Zwaan, Magliano, & Graesser, 1995).
The CI model, in particular, is composed of three levels. First, the surface level
includes the encoding of specific words and syntactical relationships. For example, the
surface level includes a reader’s ability to determine what the subject, verb, and object
of a sentence may be. Second, the textbase level refers to the meaning of a sentence.
The situation model of representation includes the linking of ideas, propositions,
generation of inferences, and connection to a learner’s prior knowledge. Third, the
situation model provides a global or broad context in which a learner participates in the
interpretation of explicit language and symbols along with inferences.
In terms of multimedia, the literature suggests that comprehension processes are
similar for text and multimedia on the back-end although there may be important
differences in front-end processing (Magliano et al., 2013). Differences in front-end
processing between text and video media manifest in terms of orthographic, gist
processing, object processing, motion processing, and perhaps the textbase (Magliano
et al., 2013). Because of reduced demands of the cognitive system in the midst of front-
end processing, many empirical studies have found that oral or audio narratives support
comprehension (Gough & Tunmer, 1986; Mayer & Moreno, 1998; Mousavi, Low, &
Sweller, 1995). This suggests that multimedia stimuli may have positive effects on
metacognitive processes such as JOLs.
20
Metacomprehension
Before proceeding, it is important to establish clear operational definitions for the
terms we are adopting from text-based metacognitive literature. Metacognitive
monitoring is an inclusive term for metacomprehension and metamemory processes
(Thiede et al., 2003). Metacomprehension is a learner’s assessment of his or her
comprehension of text or other learning materials (Hacker, 1998), while metamemory is
a learner’s assessment of his or her ability to retrieve facts or details after reading
(Jaeger & Wiley, 2014; Dunlosky & Thiede, 1998). Metacomprehension accuracy is the
ability of learners to predict accurately levels of comprehension of a specific topic after
the topic has been presented (Dunlosky & Lipko, 2007). This is to be distinguished from
metamemory accuracy, which is a learner’s ability to predict accurately his or her ability
to recall details after instruction (Rawson, Dunlosky, & McDonald, 2002).
Metacomprehension and metamemory are important mechanisms to consider when
evaluating metacognitive monitoring and control in light of discrepancy-reduction
models of self-regulated learning. In the case of interactive technologies that allow for
review and restudy, these constructs are likely to provide fruitful areas of future
research in evaluating the discrepancy-reduction feedback loop in computer-based
learning environments.
If comprehension is the degree to which a learner accurately constructs a
situation model, metacomprehension is a process of evaluating the quality of the
situation model itself (Wiley, Griffin, & Thiede, 2005). Previous research has suggested
that the more a learner focuses upon cues aligned to the situation model, the more
accurate his or her metacomprehension (Anderson & Thiede, 2008; Thiede, Griffin,
21
Wiley, & Anderson, 2010). More accurate metacomprehension is hypothesized to
impact the efficiency of restudy decisions.
Metacognitive monitoring accuracy tends to be low in text-based learning
conditions (Maki & Berry, 1984). In light of extremely low measures of relative
metacognitive monitoring accuracy, researchers began to explore what factors were
contributing to low monitoring accuracy in the context of text-based learning. Domain
familiarity was found to have little impact upon monitoring accuracy (Griffin et al., 2009;
Maki & Serra, 1992). Surprisingly, comprehension skill was also not found to be a
significant contributor to relative monitoring accuracy (Maki, Jonas, & Kallod, 1994).
Text difficulty, however, did appear to negatively impact monitoring accuracy because
easy texts cause readers to engage in automatic reading mode as compared to difficult
texts that hinder accuracy because of the scarcity of cognitive resources to monitor
accurately (Weaver & Bryant, 1995). Essentially, text difficulty has been associated with
an inverted U relationship to monitoring accuracy where the easiest and most difficult
texts result in low accuracy and medium difficulty texts result in higher accuracy.
Accordingly, researchers need to consider the quality and nature of the text because
some texts are more likely to support rich situation models as compared to others
(Griffin, Wiley, & Salas, 2013).
There is need for future research to examine how script difficulty and inherent
situation model complexity interact with visual-audio components of video. It is easy to
imagine how multimedia could render text material easier in which learners enter into an
automatic viewing mode where cognitive resources are not engaged by the learner
because of perceptions of ease. At the same time, Weaver and Bryant (1995) suggest
22
that if difficult material could be reduced in difficulty level, then there should also be a
corresponding increase in monitoring accuracy. How multimedia can mediate text
difficulty is at this time an unexplored area of research. In text-based conditions,
coherence of text (Rawson & Dunlosky, 2002) appeared to significantly impact relative
monitoring accuracy in text-based learning. How to quantify and qualitatively categorize
video difficulty and video coherence will likely become one of the major challenges
facing research concerning video-based learning.
Although metacognitive monitoring accuracy has been found to be quite low in
text-based learning contexts (Maki, 1998), rereading was found to substantially increase
metacognitive monitoring accuracy by allowing the reader to allocate more resources to
situation model construction as compared to allocating resources to textbase
construction on a second reading, especially in aiding low working memory readers
(Griffin et al., 2008; Rawson et al., 2000). These results suggest that if interactive
multimedia like video can ease the degree of cognitive resources required, then there
should be a corresponding increase in metacognitive monitoring accuracy because the
learners will have the resources to focus upon situation model construction and avoid
distraction with the lexical and textbase levels of comprehension.
Even though rereading appears to be an important strategy in text-based
learning, there is evidence to suspect that learners in video-based learning conditions
do not engage in comprehension strategies such as rewatching to the degree found in
text-based conditions (Rayner & Serano, 1994). Hasler, Kersten, and Sweller (2007)
reported that only one of the 18 participants in the stop-play group used the play/stop
button consistently. Consistency across studies, Learners appear not to engage the
23
pause button to the degree expected (Bassili & Joordans, 2008; Hasler, et al., 2007;
Tabbers & de Koeijer, 2010). The metacognitive monitoring literature above suggests
that rewatching an instructional video should result in more accurate monitoring but how
such rewatching behaviors can be encouraged in learners and how the differences in
reading and watching behaviors impact metacognitive monitoring is yet to be
addressed. On a practical level, there appears much opportunity to improve learning
through proper training in the educational use of interactive video (Merkt & Schwan,
2014).
Another strategy employed to increase metacognitive monitoring accuracy is
summarization of text material. Summarization of text material can increase
metacognitive monitoring accuracy but only after a long delay as compared to
immediately producing summaries after reading (Thiede & Anderson, 2003). The
importance of learners actively generating gist keywords has also been demonstrated
such that the act of keyword production as compared to merely viewing a list of expert
derived keywords did have a significant positive impact upon metacognitive monitoring
accuracy (Thiede et al., 2005). These findings suggest that producing gists, through the
physical act of typing or writing, is a significant factor in improving accuracy. Merely
thinking of gist words or reading gists produced by experts appears to have little positive
impact upon monitoring accuracy.
The conditions described above have direct applicability to video annotation in
the sense that learners can annotate during the video, immediately after the video, and
at a delay. It is easy to imagine learners simultaneously producing summaries of video
content, which the literature suggests should result in lower metacomprehension
24
accuracy, while delayed-annotation conditions are expected to improve metacognitive
monitoring accuracy relative to immediate annotation conditions. Simultaneous video
annotation consists of the video playing during which a learner generates annotations
without pausing the timeline. A variation of simultaneous annotation is where the video
timeline pauses as soon as the learner attempts to generate an annotation.
The timing of a generative activity, however, is not an absolute factor as
demonstrated by the positive effects of immediate self-explanation (Griffin et al., 2008).
It appears that there are some types of interventions that overcome the tendency of
learners to rely upon cues that introduce error into metacognitive judgments even when
there is no delay. In the context of multimedia, it may be the case that the negative
effects of short-term working memory upon metacognitive judgments may be overcome
as a result of multimedia effects which, in turn, could foster deeper situation-model
processing and a corresponding increase in metacomprehension accuracy.
Multimedia and Metacognitive Monitoring
The metacognitive monitoring research discussed above dealt exclusively with
the reading of expository texts without audio, illustrations, or a combination of both
modalities. For our purposes, we are basing our most broad understanding and
definition of multimedia learning as the process of learning from spoken or printed texts
and pictures that could include illustrations, photos, maps, graphics, animations, or
instructional video (Mayer, 2014).
One of the first multimedia and metacognitive monitoring studies examined
whether diagrams embedded in texts could improve metacomprehension accuracy in
comparison to text alone in a population of 59 undergraduate students enrolled in a
psychology course who were assigned to complete tutorials on airline parts, flight
25
movements, and flight instruments (Cuevas, Fiore, & Oser, 2002). The results indicated
that there was a significant positive correlation between the judgment of learning and
the test for the text with illustrations group and no significant correlation was identified
for the text-only group. The reasoning behind the gains in monitoring accuracy rested
upon an assumption that conceptual diagrams would support processing and
integration.
There is also evidence, however, that subjective factors such as beliefs
concerning the medium can have a deleterious effect upon learning. Multimedia
heuristics (cues that introduce error into JOLs) have been observed which can result in
overconfidence (Serra & Dunlosky, 2010). Although the multimedia group outperformed
the non-multimedia group in this study, there were no differences in absolute monitoring
accuracy, yet decorative graphics did appear to produce overconfidence as a function of
a multimedia heuristic whereby beliefs concerning the efficacy of multimedia, no matter
whether the multimedia is effective or ineffective, biased judgments of learning (Serra &
Dunlosky, 2010). This multimedia heuristic was also detected in the case of worked
examples with illustrations (Ackerman, Leiser, & Shpigleman, 2013). In terms of relative
metacognitive monitoring accuracy, monitoring accuracy for a decorative illustration
group was significantly lower than either the no illustration group or the conceptual
illustration group which suggests little metacognitive monitoring advantage for
conceptual graphics (Jaeger & Wiley, 2014).
From a review of the literature, the evidence is both limited and conflicting.
Multimedia, understood as text and static images, appears in some cases to improve
metacognitive accuracy (Cuevas et al., 2002), while in other cases appears to
26
negatively impact both absolute and relative monitoring accuracy (Ackerman et al.,
2013; Dunlosky & Serra, 2010; Jaeger & Wiley, 2014). These conflicting results might
be explained as a result of the quality and affordances of the specific multimedia used
and differences in metacognitive monitoring measurements.
Metacognitive monitoring in the context of multimodal media such as video is
only now beginning to be explored. Recently, relative metacognitive accuracy was
examined in three distinct conditions: video with simultaneous annotation, video with
long delayed annotation, and video-only (Thomas et al., 2016). Relative monitoring
accuracy was high for both the video-only group and the video group with long delayed
annotation which was surprising because previous text-based metacomprehension
literature had suggested superior monitoring accuracy for long delayed summarization
in comparison to the no summarization. These results suggest that there may be
important differences between text or illustrated enhanced texts and multimodal media.
In conclusion, most of the multimedia conditions tested in the literature can be
classified as static and unimodal. Some research has shown the negative effects of
decorative images without finding positive multimedia effects for conceptual images
(Jaeger & Wiley, 2014). Overall, the literature suggests that static multimedia introduces
error into metacognitive judgments. In contrast to static multimedia, there is emerging
evidence that multimodal conditions have the potential to produce higher levels of
metacognitive monitoring accuracy
Multimodality and Transience
The positive relationship between multimedia and situation model construction in
the case of multimodal instructional video may be dependent upon whether modality or
reverse modality effects manifest. On the one hand, the use of both visual and auditory
27
modes in learning has been shown to improve working memory processing, which the
literature refers to as the modality principle (Low & Sweller, 2005; Mousavi et al., 1995).
On the other hand, there is also the possibility that the transience of visual and auditory
information in video could pose reverse modality effects when the processing of both
information streams becomes a burden to cognitive processing (Leahy & Sweller, 2011;
Ng, Kalyuga, & Sweller, 2013; Wong, Leahy, Marcus, & Sweller, 2012). Thus, there is
evidence that the benefits of multimodal video may extend to more effective
metacognitive monitoring. If, on the other hand, multimodal video’s transient information
streams pose a burden upon working memory, then metacognitive processes may be
hindered as well. One strategy for addressing potential transience or reverse modality
effects is to employ system- or learner-controlled segmentation. The following section
will address how segmentation may impact both cognitive and metacognitive systems.
Video Annotation and Segmentation
Much literature exists concerning the positive benefits of segmentation upon
cognition and suggests that performance will improve where complex video or
animation is segmented in comparison to non-segmented video (Chandler & Sweller,
1996; Mayer, Dow, & Mayer, 2003; Mayer, 2005, Moreno, 2007). Pausing the video is
perhaps one means to counteract transience effects. Again, the literature has yet to
explore the impact of segmentation upon metacognitive monitoring or control processes
within the context of educational video.
Segmentation can be system-controlled or learner-controlled. Annotation
systems that pause upon annotation generation provide a unique case where the
resulting segments consist of varying durations and locations on the video timeline that
or may not represent conceptually meaningful units of information. Important questions
28
arise as to whether a learner’s decision to initiate an annotation may segment a video in
a way that disrupts the coherence of the instructional message. Specifically, if the pause
disrupts forming a textbase or situation model representation, this is likely to impact
both performance and metacognitive monitoring. Addressing how learner-initiated
pauses impact both cognition and metacognition would begin to offer insights to
instructional designers and instructors as to whether they should build in system-
controlled annotation points or allow learners to control when and where annotations
are generated.
Early work in the area of segmentation found that learner-controlled conditions
may help lower cognitive load effects through simple user interactions as pacing
controls in a multimedia presentation (Mayer & Chandler, 2001). Yet many studies have
found that learners often do not use the pause feature (Bassili & Joordans, 2008; Hasler
et al., 2007; Tabbers & de Koeijer, 2010). In a qualitative study concerning learner-
controlled pause, one participant reported, “A pause in watching video is worse than a
break in reading a book, because I felt I have no place to return to. I lost context”
(Caspi, Gorsky, & Privman, 2005, p. 40). This is an important finding because it
suggests that learners employ reading strategies in the midst of video learning. Yet if
front-end comprehension processes and user behaviors are different for text as
compared to instructional video (Magliano et al., 2013), this suggests the need to begin
a systematic investigation of the interaction between comprehension and viewing
strategies as compared to an interaction between comprehension and reading
strategies. The affordances of interactive technologies and text-based strategies may
have a significant impact upon learning and self-regulated learning processes.
29
Although some learners may find pausing a distraction (Caspi et al., 2005),
system-controlled pausing has been shown to increase performance on both retention
and transfer tests (Moreno, 2007). The positive effects of learner-controlled
segmentation have been replicated recently in terms of procedural knowledge (Stiller,
Freitag, Zinnbauer, & Freitag, 2011) and transfer (Tabbers & de Koeijer, 2010).
How learner-controlled segmentation interacts with static or dynamic visuals has
been examined by Hoffler and Schwartz (2011) who found a significant interaction
between the type of pacing and type of visual representation with the result that learning
with animations was better with learner-controlled pacing, while learning with static
graphics was better with system-controlled pacing. The findings suggest that learner-
controlled pacing is an important factor that may also improve performance in the
context of multimodal instructional video with dynamic slides and animations.
Mayer and Moreno (2003) examined whether segmentation provides benefits to
learning through reflection time; whereas, others have suggested that segmentation
supports learning through temporal cues that reinforce the “underlying structure” of the
instructional material (Spanjers, van Gog, & van Merriënboer, 2010, p. 279). The
perspective that pausing or even temporal cues can assist in recognition of global
information structures suggests that these pausing or segmenting tools may support the
construction of a situation model in video-based learning which, in turn, would be
expected to improve metacognitive monitoring accuracy.
The quantity of segmentation (no pause, 7 pauses, 14 pauses, and 28 pauses) in
instructional video has been associated with greater performance on recall and transfer
test performance although there were substantial increases in perceptions of
30
annoyance as the number of segments increased (Doolittle, Bryant, & Chittum, 2014).
Questions remain, however, as to whether the location of the segmentation impacts
learning and metacognitive monitoring.
In conclusion, video annotation as a pause mechanism needs to be considered
within the framework of segmentation literature. It is unclear, however, if similar
increases in performance will follow as the number of pauses initiated by annotation
increases since there may be unforeseen interactions between segmenting the timeline
through pauses and the process of producing annotations. If automatic pause via video
annotation allows a learner to perceive structural cues as suggested in the literature
(Spanjers et al., 2012), segmentation may be an important factor in improving
comprehension and metacomprehension.
Segmentation and Metacognitive Monitoring: Interesting Interactions
Segmentation initiated by annotations might improve cognitive performance yet
undermine metacognitive monitoring processes because of the immediacy of the gist or
summary annotation as discussed above (Anderson & Thiede, 2008; Thiede et al.,
2003; Thiede & Anderson, 2003; Thiede et al., 2005). According to long-delayed
hypothesis examined in metacomprehension literature, summaries or gists produced
immediately after reading create conditions in which learners rely upon homogeneous
cues, which tend to introduce error into metacognitive judgments, because more topic
information manifests in short-term memory immediately after reading (Dunlosky &
Nelson, 1992). After the mental network of short-term memory has decayed, a learner’s
access to long-term memory produces far more accurate summaries based upon
heterogeneous cues (Thiede et al., 2003). If comprehension of video is akin to text
comprehension as suggested by previous research (Magliano et al., 2001; Magliano et
31
al., 2015), video annotation introduces an even more immediate condition than has
been examined in text-based studies and may negatively impact metacognitive
monitoring processes.
The corollary of this is the scenario in which video is continuous and annotations
are produced at a long delay. Segmentation research suggests that non-segmented
multimodal presentations may undermine cognition, yet these same conditions are
hypothesized to improve metacognitive monitoring if delayed-summarization or gist
production effects are a factor in video annotation conditions as they are in text-based
metacomprehension research (Thiede et al., 2005).
Research on Video Annotation Systems
Empirical research specifically focused upon the cognitive and metacognitive
effects of video annotation upon learning is beginning to be produced. Most of the
empirical work has focused upon software architecture and features (Sadallah, Aubert,
& Prié, 2014) or upon the reflective practices that allow learners to annotate
performance whether in teaching practices (Colasante, 2011) or athletics (Assfalg,
Bertini, Colombo, & Del Bimbo, 2002). Little work has addressed how instructional
videos streamed through interactive video players may impact specific cognitive and
metacognitive processes to improve recall, transfer, or inference test performance. One
recent study addressed the effects of an embedded interactive online video annotation
tool and learning system upon learner performance as a function of self-regulation
(Delen, Liew, & Wilson, 2014). Because video annotation behavior was not examined in
a controlled setting but rather in conjunction with other supplemental materials and
formative questions, the effects of video annotation were not able to be isolated. In a
recent quasi-experimental study, video annotation was positively correlated to exam
32
performance but detailed trace data concerning the quality and purpose of each
annotation was not examined (Pardo et al., 2015). These studies suggest the positive
effects of video annotation but the specific factors and conditions aiding performance
require greater examination.
Implications and Directions for Future Research
Video annotation is a complex cognitive and generative process. The
affordances of video technology appear to support self-regulated learning behaviors
such as restudy and review, but how interactive features such as video annotation may
support or hinder cognitive and comprehension processes within a larger framework of
self-regulated learning is an unexamined area. If one of the primary advantages of
instructional video is the ability to pause, rewind, and review, it is necessary to use
relevant theories of metacognition, multimedia, and comprehension to begin to
formulate specific research questions that address why specific conditions can lead to
greater monitoring accuracy which in turn may lead to more efficient restudy decisions.
At this point, we simply do not know how video functionality such as annotation will
impact metacognitive monitoring and metacognitive control.
Although the metacomprehension paradigm has produced insights concerning
metacognition in text-based conditions, there are also reasons to believe that important
differences in text-based and video-based learning conditions may contribute to
differences in the accuracy of metacognitive monitoring processes across media. On a
theoretical level, future research, informed by theories of comprehension and
multimedia, can begin to identify differences of comprehension and metacomprehension
between text and video so as to test theory-based hypotheses concerning learning with
interactive multimedia.
33
Next steps in video annotation research include establishing a working taxonomy
to classify instructional video according to functional affordances, script difficulty,
complexity of graphics, and degree of adherence to effective multimedia principles. In
addition, future research needs to evaluate whether expository and narrative videos
have a similar impact upon cognitive and metacognitive performance. One of the great
challenges in evaluating learning in the context of instructional video is the fact that any
one frame or scene can shift from multimodal to unimodal information streams at any
point. For example, in the midst of narration an image can fade to black screen while
text appears on the screen with or without narration. In many ways, instructional video is
similar to Proteus, the old man of the sea, constantly changing and morphing into
something else. If science concerning cognitive and metacognitive processes is to
progress, there is a need to reformulate the question of how video impacts learning into
how specific types and combinations of instructional video impact learning.
The greater question, however, is how self-regulated learning processes such as
metacognitive monitoring and control in video-based and interactive learning
environments impact learning. The promise of video is the ability to revisit portions of
the timeline to restudy when learning deficiencies are detected. One hope for future
work is that researchers will identify specific strategies and practices to improve the
effectiveness and efficiency of restudy within the context of video-based learningThere
is great opportunity to improve learning by identifying conditions that allow for efficient
monitoring and control during both study and restudy conditions.
34
CHAPTER 2 METACOGNITIVE CONSEQUENCES OF VIDEO SEGMENTATION
Introduction
Educational uses of video continue to grow and expand (Kaufman & Mohan,
2009). Compared to traditional transient lecture environments, the learner in video-
based learning has the ability to pause, rewind, fast-forward, and review selected
frames and clips, and in some cases produce annotations tied to specific portions of the
video player timeline (Hosack, 2010). The unique and evolving affordances of 21st
century video streaming technology have the potential to support self-regulatory
monitoring and control processes at any time during the learning process (Aubert, Prié,
& Canellas, 2014; Azevedo 2009). In particular, accurate monitoring of comprehension
has been shown to be an important factor in efficient and effective self-regulated
learning in the context of text-based learning (Thiede, Anderson, & Therriault, 2003) and
beginning to be examined in multimedia contexts (Jaeger & Wiley, 2014; Pilegard &
Mayer, 2015; Serra & Dunlosky, 2007).
Just as in text-based learning, effective self-regulation in video-based learning is
assumed to be important, yet our understanding of metacognitive monitoring processes
in video is complicated by potential cognitive differences in processing (Magliano,
Loschky, Clinton, & Larson, 2013) and by interactive features of video players such as
pausing and video annotation. Accordingly, the primary purpose of the two experiments
reported in this article is to examine how segmentation and video annotation may either
hinder or harm recall and inference test performance along with metacognitive
monitoring.
35
What is Video?
Video can be understood as a simultaneous presentation of a continuous stream
of visual and auditory information (Cennamo, 2012). Research concerning the impact of
video upon learning has led to divergent results depending upon the degree of
correspondence between auditory and visual information and the degree of transience
of the presentation (Grimes, 1990). In some contexts, the use of both visual and
auditory modes in learning can improve working memory processing, which the
literature refers to as the modality principle (Low & Sweller, 2005; Mousavi, Low, &
Sweller, 1995). The modality principle is based upon a dual-processing model of
working memory which contains one system for visual information and another system
for verbal information (Baddeley, 1992). Overreliance upon one information system is
postulated to overload the system. More recent literature, however, suggests that there
is also the possibility that the transience of visual and auditory information in video could
pose transience effects when the processing of both information streams becomes a
burden to cognitive processing (Leahy & Sweller, 2011; Ng, Kalyuga, & Sweller, 2013;
Wong, Leahy, Marcus, & Sweller, 2012). One potential solution to transience effects in
video-based learning is segmentation of the video into shorter clips to account for the
limited attention span of learners (Middendorf & Kalish, 1996) or potential cognitive load
issues (Mayer & Chandler, 2001, Moreno, 2007).
System-Controlled and Learner-Controlled Video Segmentation
Video allows for segmentation but the time and agent of segmentation are
important variables to consider in video-based learning. System-controlled
segmentation occurs when an educator segments a long instructional video into shorter
film clips. Learner-controlled segmentation most often occurs when the learner clicks a
36
pause button. Early work found that learner-controlled segmentation reduced cognitive
load effects through simple interactions such as pacing controls (Mayer & Chandler,
2001) although learners often do not use the pause feature (Bassili & Joordans, 2008;
Hasler et al., 2007; Tabbers & de Koeijer, 2010). System-controlled pausing was found
to increase performance on both retention and transfer tests (Moreno, 2007). The
positive effects of learner-controlled segmentation have been observed in terms of
procedural knowledge (Stiller & Zinnbauer, 2011) and transfer (Tabbers & de Koeijer,
2010). There is some evidence, however, that learner-controlled pausing is especially
helpful for dynamic video animations as compared to system-controlled pausing (Hoffler
& Schartz, 2011). On a theoretical level, there are varying accounts as to why
segmentation aids learning. Some researchers suggest that segmentation benefits
learning because of a reduction in cognitive load (Moreno, 2007), others explain that
segmentation provides structure through temporal cues so that the “underlying structure
of the information” becomes explicit to a learner (Spanjers, van Gog, Wouters, & van
Merriënboer, 2010, p. 279). In sum, segmentation of video whether learner-controlled or
system-controlled appears to improve learning although learner-controlled pausing is
often underutilized.
If segmentation can be manipulated to support the structure of information, there
may be segmentation conditions such as random segmentation of the timeline that
could disrupt the structure of the information and hinder recall and inference test
performance. Random segmentation, however, can be either system-controlled or
learner-controlled. For example, if an educator segments the video timeline at random
points to create smaller clips, this would be an example of system-controlled random
37
segmentation. If a video player allows a learner to pause and in effect segment the
timeline, an intuitive hypothesis suggests that learner-controlled pausing introduces
some form of random segmentation since a delay between a learner’s decision to pause
and the initiation of the pause will cut the video timeline at an unknown point.
System-Controlled and Learner-Controlled Video Annotation
How video annotation interacts, however, with segmentation conditions is
unclear. Video annotation as a form of note-taking can be viewed as a generative
learning strategy (Wittrock, 1989) that may increase deep learning (Henk & Stahl, 1985;
Kobayashi, 2005) and the benefits of video annotation and interpolated short-response
prompts between system-controlled segments have been demonstrated (Cheon,
Chung, Crooks, Song, & Kim, 2014; Cheon, Crooks, & Chung, 2014; Szpunar, Khan, &
Schacter, 2013). In contrast to system-controlled video annotation (Cheon et al., 2014),
video annotation tools allow the learner to control if and when an annotation is
produced. In some cases, video annotation systems automatically pause the video as
soon as an annotation is initiated, while other systems allow the learner to type
annotations during video playback. Video annotation systems that pause upon initiation
of annotation provide a unique case where the resulting segments consist of varying
durations and locations on the video timeline that may or may not represent
conceptually meaningful units of information. Important questions arise as to whether
learner-controlled annotation disrupts the coherence of the instructional message and
hinders learning or whether learner-controlled annotation provides an opportunity for
increased integration and reflection (Scheiter & Gerjets, 2007), thereby improving recall
and inference test performance.
38
Video-Based Learning as Self-Regulated Learning
One of the primary advantages of video-based learning is the ability to control
pace, pause, fast-forward, and rewind for review and restudy (Petty & Rosen, 1987).
The affordances of interactive video player systems mentioned above, however, also
pose potential challenges to learners in the effective use of these tools (Merkt &
Schwan, 2011). Specifically, learners are responsible for regulating their educational
experience in terms of attention, use of specific video player affordances, and restudy
behaviors. Accordingly, learning from interactive video systems is dependent upon the
quality of self-regulated learning, which requires a learner to plan, manage, and sustain
the learning process (Zimmerman & Schunk, 2011). Although self-regulated learning is
an essential component within education (Zimmerman, 1998), little work has addressed
self-regulation in respect to interactive video-based learning conditions let alone how
segmentation and video annotation may impact self-regulatory processes.
Metacognition
Self-regulated learning has been heavily influenced by theories of metacognition.
The history of metacognition is long and owes much to the efforts of John Flavell (1979)
who described metacognitive experiences as “any conscious cognitive or affective
experiences that accompany and pertain to any intellectual enterprise” (p. 906). These
metacognitive experiences might reflect a learner’s feeling or sense that a particular
lesson, text, or lecture is unclear. Metacognitive knowledge “consists primarily of
knowledge or beliefs about what factors or variables act and interact in what ways to
affect the course and outcome of the cognitive enterprises” (Flavell, 1979, p. 907). This
metacognitive knowledge might be summed up as the assumptions for a learner’s belief
system for how people learn, study, and manage the learning process.
39
Nelson and Narens (1990) developed a framework for metacognition that
described metacognition as a dynamic interaction between monitoring (evaluating
knowledge levels) and control (modification of the study behavior) across acquisition,
retention, and retrieval stages of learning. For example, during a video-based
presentation, a learner may make a Judgment of Learning (JOL) as to whether he or
she believes the lesson will be remembered or understood. This type of JOL is a
product of metacognitive monitoring and has been shown to have a causal impact upon
metacognitive control processes such as restudy behaviors (Metcalfe, 2009; Thiede et
al., 2003). In ideal learning conditions, metacognitive monitoring provides high quality
data concerning knowledge levels which in turn allows for more effective metacognitive
control. Unfortunately, metacognitive monitoring accuracy in general is quite low
(Dunlosky & Lipko, 2007; Maki, 1998).
Discrepancy-Reduction Model
The relationship between metacognitive monitoring and control has been
described and examined in the literature through the discrepancy-reduction model of
self-regulated learning (Butler & Winne, 1995; Nelson, Dunlosky, Graf, & Narens, 1994).
The discrepancy-reduction model posits that a learner establishes learning goals,
monitors learning levels, and interprets monitoring data so as to determine whether to
terminate study or restudy the topic. If monitoring information indicates a discrepancy
between a learner’s established goals and current knowledge level, restudy will
continue until the current state of learning and the desired learning goals reach zero. A
major assumption of this model is that accurate metacognitive monitoring of the learning
state is necessary for the discrepancy-reduction mechanism to function effectively.
40
Factors Impacting Metacognitive Monitoring Accuracy
Accurate metacognitive monitoring is mediated through what Flavell (1979) refers
to as metacognitive knowledge, which in turn produces cues used to judge
comprehension, recall, and performance levels (Koriat, 1997). These cues are both
theory-based, namely a learner’s beliefs about learning, and heuristic-based cues that
rely upon a learner’s fluency or ability to access specific information in long-term or
short-term memory employed in the judgment process (Koriat, 1997). Some cues are
more reliable than others (Koriat, 1997), while other cues introduce error into
metacognitive judgments and are often referred to as heuristics (Serra & Dunlosky,
2010). Heuristic cues, for example, could arise if a learner’s preconceived beliefs about
the efficacy of a specific medium (text, audio, illustrations, video) create overconfidence
in future test performance (Serra & Dunlosky, 2010). While there is evidence to suggest
that learners believe video-based learning as “easier” which may lead to shallow
processing (Salomon, 1984), it has yet to be established whether epistemological
beliefs about learning from video would introduce error into the accuracy of
metacognitive judgments.
Comprehension
Much of early metacognitive monitoring research was conducted for cued-recall
learning conditions but was soon extended to examine text-based learning conditions
(Dunlsoky & Lipko, 2007). Because of a focus upon text-based learning, theories of
comprehension have been fundamental to the development of metacognitive monitoring
literature. Comprehension, a complex cognitive process and foundation for critical
thinking and problem solving, has been examined almost exclusively in text-based
conditions (McNamara & Magliano, 2009). Although there are numerous theories of
41
comprehension, the Construction-Integration (CI) model (Kintsch & van Dijk, 1978), the
situation model (van Dijk & Kintsch, 1983), and the mental model (Johnson-Laird, 1983)
have provided the most seminal foundations for empirical research in the area of
comprehension. According to these models, a reader produces multiple mental
representations of the text during the act of reading (Kintsch & Van Dijk, 1978; Kintsch,
1998). Comprehension of text requires both understanding and memory in order to build
or construct a situation-model of the mental representation (Graesser, Millis, & Zwaan,
1997).
The CI model of comprehension is composed of three levels. First, the surface
level includes the encoding of specific words and syntactical relationships. For example,
the surface level includes a reader’s ability to determine what the subject, verb, and
object of a sentence may be. Second, the textbase level refers to the meaning of
sentences. Third, the situation model provides a global or broad context in which a
learner participates in the interpretation of explicit language and symbols along with
inferences. The situation model of representation includes the linking of ideas,
propositions, generation of inferences, and connection to a learner’s prior knowledge.
Comprehension processes are similar for text and video on the back-end
although there may be important differences in front-end processing (Magliano et al.,
2013). Differences in front-end processing between text and video manifest in terms of
orthographic, gist processing, object processing, motion processing, and perhaps the
textbase (Magliano et al., 2013). Because of reduced demands of the cognitive system
in the midst of front-end processing, many empirical studies have found that oral or
audio narratives support comprehension (Gough & Tunmer, 1986; Mayer & Moreno,
42
1998; Mousavi et al., 1995). This suggests that there may be positive multimedia effects
for metacognitive processes such as evaluation of comprehension, also referred to as
metacomprehension.
Metacomprehension
Metacomprehension is a learner’s assessment of his or her comprehension of
text or other learning materials, while metamemory is a learner’s assessment of his or
her ability to recall facts or details after reading (Jaeger & Wiley, 2014; Dunlosky &
Thiede, 2013). Metacomprehension accuracy is the ability of learners to predict
accurately levels of comprehension of a specific topic after the topic has been
presented (Dunlosky & Lipko, 2007). This is to be distinguished from metamemory
accuracy, which is a learner’s ability to predict accurately his or her ability to recall
details after instruction (Rawson, Dunlosky, & McDonald, 2002).
Model of Metacomprehension Accuracy
Metacognitive judgments are “accurate as long as the cues used at the time of
making the judgments are consistent with the factors that affect subsequent
performance” (Koriat, 1997, p. 350). Cues can be superficial (beliefs, familiarity, or
interest), memory-based (recallability), and comprehension-based (related to situation
model) (Thiede, Griffin, Wiley, & Anderson, 2010). Comprehension-based cues, in
particular, have been found to be more reliable and correlated to more accurate
metacognitive monitoring (Thiede et al., 2010).
One effective strategy to increase monitoring accuracy is delayed-summarization
of text material (Thiede et al., 2005). Delayed-summarization positively impacts
metacognitive monitoring because it either requires retrieval from long-term memory or
creates a condition in which the surface and textbase levels of representation become
43
less available (Thiede et al., 2005). Immediate-summarization results in a mental
representation of the text that is often flawed because of an overreliance and an
abundance of remembered details derived from short-term memory (Thiede et al.,
2003).
Summarization in the context of video annotation raises an interesting condition
where summarization is even more immediate than the conditions examined in previous
metacomprehension research. It is easy to imagine learners producing summaries of
video content immediately as gists come to mind, which the literature suggests should
result in lower metacomprehension accuracy. Although recent research suggests that
simultaneous video annotation without pause is detrimental to both inference test
performance and monitoring accuracy (Thomas et al., 2016), this may be more a result
of split-attention effects rather than reliance upon superficial or memory-based cues.
One of the goals of the current study is to test whether immediate video
annotation with either system or learner-controlled pauses will confirm the delayed-
summarization paradigm that hypothesizes that the immediacy of the annotation will
result in poor metacomprehension accuracy.
Based upon previous research, it can be hypothesized that the immediacy of the
gist should result in lower metacomprehension accuracy (Thiede et al., 2005), yet there
is also evidence that the delay of a video annotation, however, may not be an absolute
factor as demonstrated by the positive effects of immediate self-explanation upon
metacomprehension accuracy (Griffin et al., 2008). It appears that there are some types
of generative activities that overcome the tendency of learners to rely upon superficial or
memory-based cues that introduce error into metacognitive judgments even when there
44
is no delay. In the context of learning from video, it may be the case that the negative
effects of immediate annotation may be overcome as a result of beneficial multimedia
effects which may in fact foster deeper processing at the situation model level of
comprehension. Although Mayer’s (2014) Cognitive Theory of Multimedia Learning
(CTML) does not directly reference situation models, it can be inferred that effective
multimedia can support robust construction of the situation model through more efficient
encoding and deeper integration with prior knowledge.
Multimedia and Metacognitive Monitoring
The intuitive hypothesis that multimedia could support accurate metacognitive
monitoring processes has been tested in text-based conditions. In fact, multimedia
heuristics (superficial cues that introduce error into JOLs) have been observed to result
in overconfidence no matter whether the media is effective or ineffective (Ackerman,
Leiser, & Shpigleman, 2013; Serra & Dunlosky, 2010). Even conceptual illustrations did
not appear to provide advantages in metacomprehension accuracy in comparison to a
text-only group, which suggests little metacognitive monitoring advantage for text
augmented with illustrations (Jaeger & Wiley, 2014). These studies suggest that
requiring learners to integrate illustrations with the text, although sometimes beneficial
with respect to recall and problem solving (Mayer & Gallini, 1990), does not foster
conditions that focus a learner upon comprehension-based cues dependent upon the
situation model. This may be a result of an increased burden in integrating text and
image or even the fact that the illustrations used in these studies were not especially
effective at reinforcing the situation model.
Application of these results to video-based learning, however, is problematic
because of the difference between static unimodal media (text with illustrations) and
45
dynamic multimodal media (text, narration, illustration, and animation) in terms of
processing (Magliano et al., 2013; Mayer, 2014). There is some evidence to suspect
that video-based narrations augmented with illustrations and animations can support
both recall and inference test performance along with accurate metacomprehension. In
particular, metacomprehension accuracy without annotation of video content has been
demonstrated to be high (G > .55) and was as high as a long-delayed video annotation
group (Thomas et al., 2016). This suggests that multimodal video can positively support
metacognitive monitoring processes even without delayed-annotation.
In conclusion, the current study in a set of two experiments examined how
system-controlled and learner-controlled segmentation and video annotation impact
recall and inference test performance in a series of novel and unexamined conditions. A
secondary purpose, however, was to examine how segmenting and video annotation
conditions impact metacomprehension accuracy. How metacognitive monitoring
operates in the context of video-based learning is an important question to address
because of the ability of video technology to allow for pause, review, and restudy, that
is, metacognitive control behaviors derived from accurate metacognitive monitoring
(Nelson & Narens, 1990; Butler & Winne, 1995).
Experiment 1
Hypotheses
Although there is much conceptual literature as well as empirical evidence to
hypothesize the positive benefits of video segmentation, the effects of video
segmentation upon metacomprehension accuracy have yet to be examined empirically.
This experiment explored the effects of 4 experimental conditions, two of which
reflected system-controlled segmentation (random segmentation and paragraph
46
segmentation), one that afforded learner-controlled segmentation, and a control
condition (continuous, non-segmented video).
Random segmentation
Random segmentation was hypothesized to interrupt the textbase level of
comprehension (Hypothesis 1a) because the basic unit of the textbase, namely the
sentence, was broken by random segmentation of the narrative. Weakening of the
textbase through random segmentation was also hypothesized to hinder the
construction of a situation model because of a loss of attention to the global
relationships among the textbase (sentences) and paragraphs that allow learners to
generate a situation model (Hypothesis 1b) (Kintsch, 1998). It was further hypothesized
that significant disruptions at the textbase level would result in poor relative
metacomprehension monitoring accuracy (Hypothesis 1c) (Rawson & Dunlosky, 2002).
Paragraph segmentation
In contrast to the hypothesized detrimental effects of randomized segmentation
to recall and inference performance, a paragraph segmentation condition was employed
to test whether system-controlled segmentation at the paragraph level (that is, at the
level of meaningful content chunks) would support the textbase level of comprehension
by not interrupting the sentence unit which would result in higher recall performance
relative to random segmentation (Hypothesis 2a). It was further hypothesized that
paragraph segmentation could support inference test performance by reinforcing the
global structure of the information across the video as suggested by previous research
(Spanjers et al., 2010) or reduction of cognitive load (Moreno, 2007) (Hypothesis 2b). If
multimodal video could aid in recognition of global structures and causal relationships at
the situation model, then improved metacomprehension accuracy may result
47
(Hypothesis 2c). Although lack of disruption at the textbase level may result in increases
in recall or inference test performance, metacomprehension accuracy was expected to
be low if segmentation at the paragraph level would introduce disruptions that would
undermine coherence of the situation model (Hypothesis 2d) (Rawson & Dunlosky,
2002).
Learner-controlled segmentation
A learner-controlled segmenting condition allowed participants to pause the video
at will up to five times (the participants were informed prior to the experiment). Based
upon previous literature (Bassili & Joordans, 2008; Hasler et al., 2007; Tabbers & de
Koeijer, 2010), it was expected that participants would rarely employ the pause button.
It was hypothesized that learner-controlled segmentation would introduce a condition
similar to randomized segmentation because the delay between the decision to pause
and the actual initiation of segmentation (pause) button would likely segment the
narration at the textbase (sentence) in unexpected ways which in turn would negatively
impact recall test performance (Caspi et al., 2005). It was hypothesized that learner-
controlled segmentation would result in no significant differences between learner-
controlled and random segmentation in recall test performance (Hypothesis 3a).
Although it is also to be noted that negative effects were expected to be less
severe because of hypothesized underutilization of the pause button. Inference test
performance also was expected to be hindered as well, but again the disruption of the
global situation model would be limited due to underutilization of the pause button
(Hypothesis 3b). Learner-controlled segmentation was also expected to negatively
impact metacomprehension accuracy because the inherent randomization of learner-
48
controlled segmentation would undermine coherence through disruption of the textbase
(Rawson & Dunlosky, 2002) (Hypothesis 3c).
No segmentation control
The control was a non-segmented or continuous video condition and was
hypothesized to underperform in recall (Hypothesis 4a) and inference test performance
(Hypothesis 4b) in comparison to both the random and paragraph segmentation
conditions because of cognitive load burdens (Moreno, 2007) or transience effects (Ng
et al., 2013). A competing hypothesis (Hypothesis 4c), however, suggests that
segmentation that is disruptive at the textbase would hinder recall and inference test
performance (Kintsch, 1998). Paragraph segmentation although not disruptive to the
textbase was hypothesized to infer with construction of the situation model (Hypothesis
4d).
In terms of metacomprehension accuracy, it was hypothesized (Hypothesis 4e)
that potential transience of non-segmented video could hinder metacomprehension
accuracy because of increased concurrent processing demands (Griffin et al., 2008). A
competing hypothesis (Hypothesis 4f), however, suggests that non-segmented video
could increase coherence of the audio narration and an increase in coherence is
associated with improvements in relative metacomprehension accuracy (Rawson &
Dunlosky, 2002).
Method
Participants
Fifty-four undergraduate students enrolled in either Introduction to Programming
or Educational Technology at a major university in the Southeast United States
participated in the study in partial completion of a course requirement. Three
49
participants were removed from the analysis because they did not complete the study
which resulted in 51 participants (28 males, 23 females). The average age of the
participants was 20.2 (SD = 4.29).
Design
A repeated measures ANOVA design was used to explore the hypotheses. The
within-subjects variables included: random segmentation, paragraph segmentation,
learner-controlled segmentation, and a continuous non-segmented video control group.
Dependent measures included recall and inference test performance along with
Judgment of Learning (JOL) and Prediction of Performance (POP) responses. Because
of four distinct treatment groups and four distinct videos, complete counterbalancing of
conditions was not feasible. Accordingly, a Latin Square was employed to address
ordering effects for the four videos in addition to partially counterbalancing the order of
the segmenting conditions.
Between-subject metacomprehension accuracy was calculated as a Goodman
and Kruskal’s gamma correlation based upon average JOL and inference test
performance for each condition. Between-subject metamemory accuracy was calculated
as a Goodman and Kruskal’s gamma correlation based upon average JOL and recall
test performance for each condition. Because participants also completed a POP,
between-subject metacomprehension accuracy was also calculated as a Pearson’s
correlation coefficient based upon average POP and inference test performance. In
addition a second measure of metamemory accuracy was calculated as the Pearson’s
correlation based upon average POP and recall test performance. Between-subject
metacomprehension and metamemory accuracy is especially relevant for longer single
lessons with complex learning outcomes (Pilegard & Mayer, 2015). Accordingly,
50
between-subject metacomprehension accuracy was an appropriate operationalization of
metacomprehension since the current study required each participant to complete one
lesson for each of the four segmentation conditions.
Materials
Video Scripts
The four video scripts were adopted from Thiede & Anderson (2003) and
included the following topics: Norse Settlements, Naval Warfare, Alcohol and Sleep,
and Experimental Design. The video scripts were narrated by a male voice with no
change to the original Thiede & Anderson (2003) text. The average narration speed
across videos was 155.6 words per minute which is recommended for narration, books
on tape, and voice-over video (Williams, 1998). The script length varied between 1137
and 1319 words. Flesch-Kincaid grade levels for the video scripts ranged between
grades 11 through 15. The audio narration ranged between -12 dB and -6 dB, which is
considered an optimal range for audio narrations (Williams, 1998). Videos lasted
between 7:32 minutes and 8:12 minutes which is considered a reasonable duration for
instructional video in online settings before learners lose interest (Guo et al., 2014).
Hardware
Participants viewed the instructional videos on Apple iMac desktop computers
with a 27 inch screen, keyboard, and mouse. The videos were delivered via Google
Chrome browser, Version 50.0. During the video portion of the experiment, participants
were not able to close the browser window or skip ahead in each video presentation.
Participants were able to adjust volume through the keyboard volume controls and all
other keyboard commands were disabled during the video portion of the experiment.
51
Segmentation Conditions
The experiment included four distinct video conditions: random segmentation,
paragraph segmentation, learner-controlled segmentation, and no segmentation
(control). The duration between segments was 10 seconds during which a screen
appeared to the participant with the following text, “The video will continue in 10
seconds.” Ten seconds was determined to be an appropriate time based upon previous
research that had determined participants in ideal conditions need between 50 and 60
seconds (Thiede et al., 2008). See Figure 2-1 for a combined screenshot of the video
screen and subsequent segmentation frame.
For the random segmentation condition, randomization was achieved by splitting
each script into five equal and distinct sections. The next step included counting and
numbering each word, and using a random number generator to provide a breakpoint in
the timeline. For example, if the random number generator produced the number
twenty-one, a segment was created before the twenty-first word. The paragraph
segmentation condition for a video included five segments but these segments occurred
at five natural breaking points in the script, namely at the end of a paragraph. It is to be
noted that some segments at the paragraph level included multiple paragraphs. The
learner-controlled condition included a pause button that allowed a user to pause the
video up to five times. If a participant clicked the pause button, the same screen and
prompt appeared as in the random and paragraph annotation conditions. See Figure 2-
2 for a combined screenshot of the learner-controlled segmentation condition. The no
segmentation condition neither included segments nor the ability to pause the video.
This no segmentation condition played the video continuously.
52
Images, Animations, and Callouts
The videos included high-quality graphics, maps, timelines, coordinated cues,
callouts, and simple animations. Using the guidelines proposed by Clark and Lyons
(2010), the graphics that were employed in all of the videos were evaluated to what
degree they served representational, organizational, relational, transformational, and
interpretive purposes. If a word or image was displayed on the screen, it corresponded
to a word in the narration so that no extra text content was added to the intervention. In
addition, videos were produced to achieve high correspondence between images and
narration (Grimes 1990). Decorative graphics that did not directly support learning were
not employed.
Judgments
After viewing four videos through each of the four conditions, participants were
prompted to answer a JOL and a POP for each video in the same order as presented
(Griffin et al., 2009; Jaeger & Wiley, 2014; Thiede et al., 2005, 2009). For the JOL, a
seven-point Likert JOL item, common in previous metacomprehension literature (see
Anderson & Thiede, 2003; Thiede et al., 2005) was used to predict how well a
participant understood material before taking the performance assessment. Specifically,
the participants saw the title of the video and responded to the following question: “How
well do you think you understood this video? 1 (very poorly) to 7 (very well).” This was a
one-item measure displayed on a separate web page.
After completing the JOL, participants were required to answer a POP on a
screen that prompted the participant on a separate web page, “If you were to take a test
on the video title listed above, how many questions out of 12 would you answer
correctly?” In the dropdown menu for the POP, each point value was converted to
53
proportional score as well. For example, if a participant predicted six out of twelve
questions correct, this selection would appear as “6/12 (50% correct).” This is a
common prompt used in metacognitive monitoring studies (Schraw, 2009). In sum, each
participant was required to produce four JOLs and four POPs for each of the four video
topics.
Recall and Inference Performance
Each participant completed a 12-item multiple-choice assessment for each video
that contained six recall questions and six inference questions. Each multiple-choice
question included three distractors and one correct response. The inference and recall
questions for each of the videos were adopted from the Thiede and colleagues’ studies
(Thiede & Anderson, 2003; Thiede et al., 2003; Thiede et al., 2008) that have been
used in a number of empirical studies with a high reliability score (α = .80). Reliability
analysis of the items in this study yielded an acceptable reliability score (α = .71).
Recall questions evaluated the degree to which a learner could correctly recall
details from each of the four videos. Recall questions were designed to test the textbase
level of representation because they examine whether a participant can recall verbatim
facts from the text. For example, for the Norse Settlements video participants were
asked to recall historical dates and the names of various medieval ships.
Inference test scores were used to test the learner’s situation model of the videos
(i.e., access the learner’s situation model; Graesser et al., 1997; Kintsch, 1988). For
example, an inference question for Naval Warfare asked the participant to select the
best title for the video that summed up the essence of the instructional video. The
purpose of including two types of learning tests follows the same rationale established
54
in much metacomprehension literature, namely evaluating a participant’s ability to judge
the textbase level of representation (i.e., recall) and ability to judge the situation-model
(i.e., inference) (Thiede & Anderson, 2003).
Test questions were presented to the participant one at a time for each video
topic. Testing followed the same order as the videos had been presented to the
participant (Griffin et al., 2009; Thiede et al., 2005, 2009). Type of test question varied
systematically between inference and recall questions which were randomly selected
from a question pool for either inference or recall questions for a particular video topic.
All participants responded to the same set of questions. As a result, consecutive
ordering of inference or recall questions for one video topic was avoided among a set of
twelve multiple-choice questions. Participants completed a total of 48 questions for all
video topics.
Procedure
Participants were randomly assigned to a computer station after completing a
participation agreement form. Each participant was instructed to put on headphones
and click a link on the desktop that would launch the experiment. A multimedia
presentation introduced the participants to the study, required a sound check, prompted
participants to complete demographic survey questions, and presented the following
instruction:
“You will soon view four instructional videos on various topics. Your goal is to learn as much as you can about each topic. Two videos will be segmented into shorter clips. One video will allow you to pause at will (up to 5 times). One video will play continuously. After viewing the videos, you will be asked to judge how well you understood each topic and predict how you would do on a test on each topic. You will then be asked to complete a multiple-choice test for each topic. Do the best you can and if you have technical difficulties, please raise your hand.”
55
Participants viewed a sample video, completed a sample JOL question and POP,
and then answered a sample inference multiple-choice question based upon the sample
video to counteract participant tendencies to base judgments upon surface memory
alone (Jaeger & Wiley, 2014; Thiede et al., 2011). Participants were then encouraged to
ask clarifying questions about the procedure.
After being randomly assigned to one of the combinations of video and
segmentation conditions based upon a Latin Square to account for ordering effects,
participants viewed four instructional video topics. The videos were presented in three
segmentation conditions (random, paragraph, and learner-controlled) and a continuous
no segmentation control. After viewing all four videos, participants completed a JOL and
POP for each video topic. After completing the judgments of learning above,
participants completed a learning performance test that included six inference and six
recall items for each video. After completing tests for all four video topics, a global score
was displayed for overall performance for all videos. Participants spent approximately
60 minutes to complete the study.
Results
Recall and Inference Test Performance and Metacomprehension Accuracy
Descriptive data on test performance and metacognitive judgments is reported in
Table 2-1. Average test scores for recall and inference questions are proportions of
correct responses based upon a maximum of 6. Average total test scores are
proportions of correct responses based upon a maximum of 12. JOL means were
calculated for each condition by dividing the sum of all predictions by the total number of
participants. The same procedure was applied to calculate POP means for each
condition.
56
A 4 (segmentation condition: random segmentation, paragraph segmentation,
learner-controlled segmentation, and no segmentation) X 2 (test type: recall, inference)
repeated measures ANOVA on test performance indicated a main effect for test type
with recall (M = .59) significantly outperforming inference (M = .53) test performance,
F(1, 100) = 6.15, p = .015, partial η2 = .06. According to Cohen (1988), this can be
classified as a medium effect size. See Figure 2-3 for recall and inference test
performance comparisons.
Two one-way repeated measures ANOVAs were conducted to evaluate
differences across each of the four conditions with respect to recall and inference score
performance. In all analyses of means, no violations of sphericity were detected.
Significant effects of segmentation condition were detected on recall test performance,
F(3, 150) = 5.24, p =.002, partial η2 = .10 and inference test performance F(3, 150) =
2.93, p =.04, partial η2 = .06. Two one-way repeated measures ANOVAs were
conducted to evaluate differences across each of the four conditions with respect to
JOL and POP magnitudes. No significant differences across conditions were identified
for either JOL, F(3, 150) = 1.89, p =.15, or for POP, F(3, 150) = 1.82, p =.163. Paired-
samples T-tests were conducted to evaluate specific comparisons across conditions for
recall and inference test scores since significant effects were detected. See Table 2-2
for comparisons.
Between-subjects metacomprehension accuracy was calculated as gamma
correlation between the JOL and inference and recall test performance respectively
following the method established by Pilegard and Mayer (2015). This measure of
metacomprehension accuracy is especially useful for long multimedia videos such as
57
those used in the current study. It is also comparable to within-subjects
metacomprehension accuracy. Table 2-3 includes calculations for metamemory and
relative metacomprehension accuracy. As expected relative metamemory accuracy was
low and in all conditions the correlation between the JOL and recall performance was
not significant since these judgments were not focused upon the situation model (Wiley
et al., 2005). Metacomprehension accuracy, however, was found to be both significant
and moderate for the no segmentation group.
Since the following study employed a POP item, it was also possible to calculate
metacomprehension and metamemory accuracy between a POP and recall and
inference test performance respectively. Because the correlation was between two
continuous variables, Pearson’s r was more appropriate than the non-parametric
gamma correlation. Interpretation of the Pearson r is similar to Gamma in which
correlations range from -1 to +1 with correlations at or below 0 indicating poor accuracy.
Pearson correlations between predictions and performance are reported in Table 2-4.
Metamemory accuracy for both measures was insignificant and low.
Metacomprehension accuracy was insignificant and low for conditions other than the
significant and moderate levels of the no segmentation control condition.
Random Segmentation Effects
As predicted (Hypothesis 1a), random segmentation hindered both recall and
inference test performance. Specifically, participants in the random segmentation
condition (.49) significantly underperformed in recall test performance than participants
in the paragraph segmentation condition (.64) [t(50) = 3.34, p =0.001, d = .62] and the
no segmentation condition (.64) [t(50) = 3.12, p =0.003, d = .61]. Random segmentation
(.48) significantly underperformed in inference test performance in comparison to the no
58
segmentation condition (.58) [t(50) = -2.62, p =0.01, d = .41] (Hypothesis 1b). Inference
test performance did not differ significantly between random and paragraph
segmentation conditions. As predicted (Hypothesis 1c), the hypothesized disruption in
comprehension resulted in low relative metacomprehension accuracy. Specifically, two
separate correlations indicated that metacomprehension accuracy was insignificant and
low for random segmentation (G = .03; r = .01). In other words, the random
segmentation condition resulted in extremely poor relative metacomprehension
accuracy.
Paragraph Segmentation Effects
Paragraph segmentation was hypothesized to support recall and inferential
processing through reduced cognitive load (Moreno, 2007) or increased focus upon the
structure of information (Spjaners et al., 2010) (Hypothesis 2a-b). Surprisingly,
participants in the paragraph segmentation condition (.64) did not significantly differ in
recall test performance than participants in the no segmentation condition (.64). In
addition, although paragraph segmentation (.51) was not significantly lower in inference
test performance in comparison to the no segmentation condition (.58) [t(50) = -1.96, p
=0.06], it is important to note that significance was nearly achieved.
Paragraph segmentation was hypothesized to support inferential processes
(Hypothesis 2c) but in fact may have weakened situation model construction based
upon the nearly significant differences between paragraph segmentation and no
segmentation. Accordingly, two separate measures of metacomprehension accuracy
indicated that metacomprehension accuracy was low and insignificant for paragraph
segmentation (G = .03; r = .01).
59
As expected, the learner-controlled condition did not fully utilize the pause button
(M = 2.1 pauses initiated). It was hypothesized that learner-controlled segmentation
would introduce disruptive effects similar to those encountered in random segmentation
and would hinder both recall and inference test performance (Hypothesis 3a).
Interestingly, the learner-controlled segmentation condition (.59) significantly
outperformed the random segmentation condition in recall (.49) [t(50) = 2.64, p =0.001,
d = .42]. Hypothesis 3b was not confirmed since no significant differences in inference
test performance were identified between the learner-controlled and random
segmentation condition. As expected (Hypothesis 3c), the disruptive effects of learner-
controlled segmentation to the situation model hindered metacomprehension accuracy.
Metacomprehension accuracy for learner-controlled segmentation was both insignificant
and low (G = .18; r = .21).
Learner-Controlled Effects
As expected, underutilization of the pause button resulted in no significant
differences in recall and inference test performance between the no segmentation
condition and the learner-controlled condition. In spite of a lack of significant
differences, however, it is important to note that overall learner-controlled segmentation
resulted in lower recall and inference scores relative to the no segmentation condition.
This result is congruent with underutilization of the learner-controlled segmentation
button (M = 2.1 pauses initiated), which may have resulted in a video condition that was
nearly continuous and far less disrupted in comparison to the random segmentation
condition.
60
No Segmentation Effects (Control)
Contrary to Hypothesis 4a-b that non-segmented video would introduce cognitive
load (Moreno, 2007) and transience effects (Ng et al., 2013) with detrimental effects
upon recall and inference test performance, the no segmentation condition increased
recall [t(50) = 3.12, p =0.003, d = .61] and inference performance [t(50) = -2.62, p =0.01,
d = .41] in comparison to random. This supports Hypothesis 4c that disruption at the
textbase would undermine recall and inference test performance (Kintsch, 1998).
Hypothesis 4d that paragraph segmentation would disrupt the situation model without
disrupting the textbase was partially confirmed since no significant differences in recall
test performance were identified between the no segmentation and paragraph
segmentation condition. Although differences in inference test performance between the
no segmentation and paragraph condition did not reach significance at a critical value of
.05, significance was nearly achieved [t(50) = -1.96, p =0.06]. This suggests that
disruption to global comprehension processes may be effected by paragraph
segmentation.
With respect to metacomprehension accuracy, two competing hypothesis were
tested. Hypothesis 4e that cognitive load and transience effects would result in poor
metacomprehension accuracy because of concurrent processing was not confirmed
(Griffin et al., 2008). The competing hypothesis that non-segmented video would result
in accurate metacomprehension as a result of greater coherence and less disruption at
either the textbase or situation model (Rawson and Dunlosky, 2002) was confirmed
since metacomprehension accuracy of the no segmentation condition was both
significant and moderate (G = .44; r = .42) in contrast to the low and insignificant values
of the three segmented conditions.
61
Discussion
Random Segmentation
The results from Experiment 1 suggest that recall and inference test performance
were affected by various segmentation conditions. Segmentation was assumed to
disrupt various levels of comprehension depending upon where the system or learner-
controlled segmentation was initiated. As predicted, random segmentation as form of
disruption to the textbase (Hypothesis 1a) appeared to hinder recall performance in
comparison to the no segmentation condition control and had a negative impact upon
inference test performance (Hypothesis 1b). Poor inference test performance is
indicative of a poorly developed situation model (Kintsch, 1994), which was likely a
result of an unstable textbase (Kintsch & van Dijk, 1978). Whatever advantages random
segmentation may have had in the reduction of cognitive load (Moreno, 2007), potential
cognitive advantages of segmentation are overshadowed by conditions that disrupt the
textbase and in turn disrupt the situation model.
As predicted by the disruption hypothesis (Rawson & Dunlosky, 2002)
(Hypothesis 1c), weakening of the textbase and lack of coherence was associated with
poor metacomprehension accuracy (G = .00; r = .12). Because the disruption occurred
at the level of the textbase, participants likely focused attention upon textbase cues
unrelated to the more diagnostic and reliable cues related to the situation model. When
there is a lack of alignment between the cues and comprehension levels,
metacomprehension accuracy is expected to be low (Koriat, 1997).
Paragraph Segmentation
In comparison to the disruptive effects of random segmentation, paragraph
segmentation was an experimental condition hypothesized to support recall by not
62
disrupting the textbase (Hypothesis 2a) and to support inference test performance
(Hypothesis 2b) either through reduction in cognitive load (Moreno, 2007) or support of
information structures (Spanjers et al., 2010). As predicted (Hypothesis 2a), a lack of
disruption to the textbase resulted in improvement in recall test performance (M = .64) in
comparison to random segmentation (M = .49).
Surprisingly, recall test performance was not significantly different between
paragraph segmentation (M = .64) and the no segmentation control (M = .64) as
predicted by segmenting literature (Moreno, 2007; Spanjers et al., 2010). The lack of
beneficial segmenting effects upon recall test performance suggests that the videos
used in this study did not appear to result in transience effects (Leahy & Sweller, 2011;
Wong, Leahy, Marcus, & Sweller, 2012). This result, however, corresponds to previous
research in video-based learning that has demonstrated that high correspondence
between image and narration can significantly reduce transience and improve recall
performance (Grimes 1990). Correspondence between narration and images allowed
for efficient encoding and minimized the need to make sense of or resolve information
being processed through the auditory and visual channels. In this case, segmentation
does not offer potential benefits in encoding.
Contrary to Hypothesis 2b and surprisingly, segmentation did not improve
inference test performance significantly in comparison to random segmentation in spite
of potential gains from a reduction in cognitive load (Moreno, 2007) or gains from an
increased awareness of the structure of the information (Spjaners et al, 2010). This
finding, however, is consistent with text comprehension literature that has demonstrated
63
that conditions that can significantly improve memory for details are frequently different
from conditions that improve comprehension of the text (Wiley et al., 2005).
In fact, differences in inference test performance between the no segmentation
control condition (M = .58) and the paragraph segmentation condition (M =51) were
moderately significant (p = .06). In other words, segmentation at the paragraph level
likely undermined the ability to construct a situation model. This is a surprising finding in
light of substantial research that has demonstrated the benefits of segmentation of
multimedia presentations (Mayer & Pilegard, 2014). This unexpected finding, however,
may be explained as a result of the global nature of the situation model where
inferences and principles are derived across multiple paragraphs (Kintsch, 1998). For
example, if paragraph segmentation occurred between paragraphs three and four, this
might allow for reflection upon relationships between the first three paragraphs but
disrupt the relationship between the first three paragraphs and subsequent paragraphs.
Just as random segmentation disrupted the textbase level, so paragraph segmentation
appears to have disrupted the situation model.
Another possible explanation for poor inference test performance in comparison
to a no segmentation control condition is the characteristic of the videos used in this
study. For example, the videos used in Moreno’s 2007 study that provided significant
evidence for the positive benefits of segmentation were videos of an expert teacher
applying pedagogical and classroom management skills. This type of video is quite
different from videos used in this study that were developed from expository texts
whose complexity allows for the opportunity to evaluate situation model level inferences
(Wiley et al., 2005).
64
It was hypothesized (Hypothesis 2c) that paragraph segmentation would aid
recognition of global relationships inherent in the videos (Spanjers et al., 2010) and
would result in a significant and high levels of relative metacomprehension accuracy.
The insignificant and low metacomprehension results do not support this hypothesis. In
fact, the poor metacomprehension accuracy of paragraph segmentation (G = -.05; r =
.05) supports Hypothesis 2d that paragraph segmentation although less disruptive than
random segmentation would result in decreased coherence and poor
metacomprehension accuracy (Rawson & Dunlosky, 2003).
Learner-Controlled Segmentation
The learner-controlled segmentation condition had the ability to pause the video
timeline up to five times. As predicted (Bassili & Joordans, 2008; Hasler et al., 2007;
Tabbers & de Koeijer, 2010), the pause button was underutilized (M = 2.1 pauses
initiated). The findings did not support the hypothesis that learner-controlled
segmentation would disrupt the textbase to the same degree as the random
segmentation condition (Hypothesis 3a). Additionally, the hypothesis that learner-
controlled segmentation would harm the situation model to the same degree as random
segmentation was confirmed since no significant differences in inference test
performance were identified (Hypothesis 3b) although it is to be noted that there was a
slight increase in inference test performance score (See Table 2-2). In other words,
contrary to our predictions, learner-controlled segmentation resulted in higher
performance in recall test performance than random segmentation, and at the same
time no differences in inference test performance. This is an important finding because
it suggests that affordances of the technology such as learner-controlled segmentation
(pause) may impact different comprehension levels. As previous comprehension
65
literature has noted, there are some types of interventions that benefit recall but not
necessarily comprehension (Wiley et al., 2005).
Two competing hypotheses were evaluated with respect to learner-controlled
segmentation as a factor that may impact learning from video. The disruption of
comprehension hypothesis based upon Kintsch (1998) suggests that the disruptive
effects of learner-controlled segmentation would undermine the textbase and situation
model with resulting lower performance in terms of both recall and inference tests. On
the other hand, the learner-controlled hypothesis (Scheiter & Gerjets, 2007) suggests
that learner-controlled environments could aid learning. Since no significant differences
in recall or inference test performance were identified between learner-controlled
segmentation and a no segmentation control, neither hypothesis was confirmed in the
current study. There is, however, some evidence to suggest some disruption occurred
since recall and inference test scores were lower for the learner-controlled condition in
comparison to the no segmentation control. As to why potential learner-controlled
disruptions did not result in a significant difference in comparison to the no
segmentation control, it may be the case that the underutilization of the pause button
did not produce a critical quantity of disruptive segments required to significantly hinder
learning performance.
Metacomprehension accuracy for learner-controlled segmentation was
hypothesized to be low because of disruption to the situation model through disruption
of the textbase and situation model (Dunlosky & Rawson, 2005) (Hypothesis 3c).
Metacomprehension accuracy was in fact both low and insignificant (G = .18; r = .21)
which suggests that learner-controlled segmentation (pause) although underutilized (M
66
= 2.1 pauses initiated) proved to create a disruption on the metacognitive level. Thus,
learner-controlled segmentation in comparison to a no segmented condition did not
appear to significantly undermine objective performance measures of recall and
inference ability, but rather undermined the accuracy of metacognitive monitoring
processes.
No Segmentation
As discussed above, the hypotheses (Hypothesis 4a-b) that no segmentation
would hinder recall and inference test performance as a result of excessive cognitive
load (Moreno, 2007) or transience (Ng et al., 2013) were not confirmed. In fact, the no
segmentation condition did not differ with the paragraph segmentation in recall test
performance and significantly outperformed paragraph segmentation in inference test
performance which supports Hypothesis 4d that modality effects (Kalyuga et al., 1999;
Mayer & Moreno, 1998) without the disruptive effects of segmentation could aid
learning.
The benefits of improved comprehension in the no segmentation condition
extended to metacognitive monitoring processes as well. As predicted (Hypothesis 4e),
the findings supported the hypothesis that metacomprehension accuracy is related to
the quality of the situation model (Griffin et al., 2008) as observed by high inference test
performance and that multimedia can positively impact metacognitive monitoring
processes (Thomas et al., 2016) as long as there is also a high degree of coherence
(Dunlosky & Rawson, 2002). In contrast to the low and insignificant correlations of all
three segmentation conditions (random, paragraph, and learner-controlled), the no
segmentation (control) resulted in significant and moderate metacomprehension
accuracy correlations (G = .44; r = .42). Across conditions, as disruption to the situation
67
model became less severe and coherence increased, metacomprehension accuracy
improved as predicted by previous research (Rawson & Dunlosky, 2002).
Conclusion
In conclusion, Experiment 1 provides evidence that segmentation can have
deleterious effects upon recall and comprehension depending upon where the segment
occurs depending on what level of comprehension is disrupted. Although failure to
detect positive effects for either recall or inference test performance was an unexpected
finding in light of substantial research supporting the cognitive benefits of segmentation,
the results and corresponding trends reported in Experiment 1 can be accounted for
according to theories of comprehension and metacomprehension (Kintsch, 1998;
Rawson & Dunlosky, 2005). In addition, there was significant evidence to support the
principle that multimodal video could support both comprehension levels and
metacomprehension processes in a multimodal video context.
68
Figure 2-1. Combined screenshot of the video player screen and subsequent segmentation screen representative of both the random and paragraph segmentation conditions.
Figure 2-2. Combined screenshot of learner-controlled video screen and subsequent
segmentation screen.
69
Table 2-1. Mean Test Scores and Judgment Magnitudes for Experiment 1.
Random segmentation
Paragraph segmentation
Learner-controlled segmentation
No segmentation
Recall % Correct
.49 (SE .04)
.64 (SE .03)
.59 (SE .03)
.64 (SE .03)
Inference % Correct
.48 (SE .03) .51 (SE .03) .55 (SE .03) .58 (SE .03)
Total % Correct
.48 (SE .03) .57 (SE .02) .57 (SE .03) .61 (SE .03)
Judgment of Learning (JOL)
4.4 (SE .15) 4.4 (SE .18) 4.8 (SE .17) 4.5 (SE . 18)
Prediction of Performance (POP)
.63 (SE .04) .62 (SE .03) .68 (SE .02) .63 (SE .03)
Figure 2-3. Comparison of recall and inference test performance across conditions.
70
Table 2-2. Post Hoc Paired-T Test Comparisons 95 %
Confidence
Mean Diff.
SD
SE Lower Upper t df Sig. (2-tailed)
Recall % Correct Random – Paragraph
.15 .32 .05 .06 .24 3.34 50 .001* (d = .62)
Random – Learner-controlled
.10 .28 .04 .02 .18 2.64 50 .001* (d = .42)
Random – No segmentation
.15 .34 .05 .05 .25 3.12 50 .003* (d = .61)
Inference % Correct
Random – No segmentation
.11 .29 .04 -19 -.02 -2.62 50 .01* (d = .41)
Paragraph – No segmentation
.08 .29 .04 -.15 .00 -1.96 50 .06
71
Table 2-3. Metamemory and Metacomprehension Accuracy Group JOL to Recall JOL to Inference (metamemory)
(metacomprehension)
M (SE) M (SE) Random segmentation
. .03 (.11)
.00 (.11)
Paragraph segmentation
.14 (.15) -.05 (.14)
Learner-controlled segmentation
-.07 (.14) .18 (.16)
No segmentation
.20 (.14) .44 (.14)*
Note. A * indicates statistically significant correlation at p < .05.
Table 2-4. Relative Accuracy for POP for Recall and Inference Group POP to Recall
Metamemory Accuracy POP to Inference
Metacomprehension Accuracy M (SE) M (SE) Random segmentation
.01 (.11)
.12 (.10)
Paragraph segmentation
.24 (.14)
.05 (.14)
Learner-controlled segmentation
-.01 (.14)
.21 (.14)
No segmentation
.21 (.13)
.42 (.14)*
Note. A * indicates statistically significant correlation at p < .05.
72
CHAPTER 3 METACOGNITIVE CONSEQUENCES OF VIDEO ANNOTATION
Experiment 2
Since results from Experiment 1 suggest that some forms of segmentation could
have a deleterious effect upon cognitive and metacognitive performance, Experiment 2
was designed to evaluate whether a generative activity such as annotation would
interact with the segmentation conditions addressed in Experiment 1 to support learning
and metacognitive monitoring, specifically metacomprehension. Encouraging learners to
generate summaries of instructional material has been shown to improve
comprehension and is theorized to aid learners in building relations in the instructional
materials as well as supporting integration with prior-knowledge (Wittrock, 1989).
Summarization or summative annotation can also improve comprehension through self-
testing and encourage learners to repair comprehension (Winne & Hadwin, 1998). How
video annotation as a form of summarization will interact with various video annotation
conditions to impact cognitive and metacognitive processes was the primary focus of
Experiment 2. The video annotation conditions included the following: random video
annotation, paragraph video annotation, learner-controlled video annotation, and
simultaneous video annotation.
Hypotheses
In Experiment 2, random video annotation is a novel condition that combines
random segmentation and the generative act of summarization. It was unknown
whether the summarization activity allowed for sufficient repair of both comprehension
and recall that were greatly hindered in the random segmentation condition in
Experiment 1.
73
Split-attention effects
One hypothesis (Hypothesis 1a) suggests that simultaneous annotation should
be significantly lower than either system or learner-controlled annotation conditions that
pause the video timeline because of deleterious split-attention effects (Chandler &
Sweller, 1996). Based upon Hypothesis 1a, we would expect random, paragraph, and
learner-controlled video annotation to be significantly higher in both recall and inference
test performance because learners would not need to focus both upon transient
information streams (Ng et al., 2013) and the mechanical demands of keyword
production (Kobayashi, 2005).
Textbase and situation model disruption
Among the three video annotation conditions that lack split-attention effects
(random, paragraph, and learner-controlled video annotation), a second hypothesis
suggests that the benefits of annotation may be negatively impacted by where the
annotation occurs on the timeline just as the location of video segmentation had
impacted both recall and inference test performance in Experiment 1 (Hypothesis 1b).
This hypothesis is based upon theories of comprehension that posit that the textbase
level of comprehension is foundational to the situation model (Kintsch, 1998). If random
video annotation disrupts the textbase, it may result in a situation where the learner
attempts to repair the textbase level since the audio narration was paused midstream.
Paragraph video annotation, on the other hand, would allow a learner to generate
annotations that relate more to the situation model because of a lack of disturbance to
the textbase and greater attention to information structures (Spanjers et al., 2010).
Learner-controlled video annotation would likely not be as effective as paragraph
74
segmentation because of unintended disruptions to the textbase as a function of the
delay between the decision to annotate and manually clicking the annotation field.
Annotation effects
With respect to a comparison of segmentation and video annotation conditions,
notetaking literature suggests potential increases in recall and comprehension across
conditions as a result of encoding effects associated with note-taking (Hypothesis 2a)
(Kobayashi, 2005). In light of system and learner-controlled conditions that
automatically pause the video for annotation production, it is assumed that the
mechanical demands of annotation production will not interfere with encoding
processes.
A final hypothesis arises that there are in fact no significant gains in recall or
inference performance from the act of annotation or keyword production (Hypothesis
2b) which has been the case in previous metacomprehension research based upon the
texts and tests used in this study (Thiede et al., 2005). A lack of significant effects upon
recall and inference test performance from keyword production may be a result of the
fact that production of one keyword (the case with prior studies) does not provide
enough of an opportunity to integrate new information with prior knowledge.
Immediate annotation effects
Video annotation was expected to hinder metacomprehension accuracy for all
four video annotation conditions because of an overreliance upon memory-based and
superficial cues derived from short-term memory (Hypothesis 3a) (Thiede et al., 2005).
Annotation as keyword production can be considered a simple word recall task, but
when performed immediately, learners are more likely to rely upon surface level cues
related to the textbase as compared to more stable cues derived from the situation
75
model (Thiede et al., 2005). This is the long-delayed hypothesis and has been validated
in numerous studies (Thiede et al., 2008). Thus, metacomprehension accuracy was
expected to be low across all four video annotation conditions.
A competing hypothesis, however, suggests that if video annotation supports
comprehension by directing learners to the situation model because of positive
multimedia effects (Mayer, 2014), then metacomprehension accuracy should improve
with the exception of simultaneous annotation (Hypothesis 3b). This hypothesis is
based upon the fact that immediate manipulations such as self-explanation (Griffin et al,
2008) or concept mapping (Redford et al., 2012) have supported relative
metacomprehension accuracy. In other words, because of the potential benefits to
processing from multimodal media (Mayer, 2014) and differences between text and
video in terms of front-end processing (Magliano et al., 2013), it is possible that
immediate annotation could in fact support metacomprehension accuracy.
Method
Participants
Forty-nine undergraduate students (27 males, 22 females) enrolled in
Introduction to Programming and Educational Technology at a major university in the
Southeast United States in the study in partial completion of a course requirement. The
average age of the participants was 19.8 (SD = 2.39).
Design
The design was identical to Experiment 1 in terms of employing a repeated
measures ANOVA with Latin Square design for video order and a partially
counterbalanced ordering of video annotation conditions. The within-subjects variables
included: random video annotation, paragraph video annotation, learner-controlled
76
video annotation, and simultaneous video annotation. Dependent measures included
the same recall and inference test performance along with JOL and POP measures
used in Experiment 1.
Materials and Procedure
The videos and the computer laboratory were the same as Experiment 1. In
addition, for the randomized and paragraph video annotation conditions, which were
system-controlled, the segmentation for the video annotation prompt and response
screen occurred at the same location points as in Experiment 1. For the random video
annotation condition and paragraph video annotation, at the end of a video segment, a
new screen with a text box would appear to prompt the participant to produce a gist.
See Figure 3-1 for a combined screenshot of a video screen and subsequent annotation
screen.
The learner-controlled video annotation condition included a right side annotation
area which, when clicked upon, would automatically pause the video and hide the video
background. The annotation screen only allowed one annotation at a time. See Figure
3-2 for a screenshot of the learner-controlled annotation video and annotation screens.
This learner-controlled video annotation mirrors how some popular video annotation
players currently function in which the video pauses as soon as an annotation is
initiated (Hosack, 2010). The simultaneous video annotation condition included the
same type of right side annotation area as the learner-controlled video annotation player
but did not pause the video when an annotation was initiated as is the case with
annotation systems such Videonot.es™ and Lynda.com™. See Figure 3-3 for a
screenshot of the simultaneous video annotation conditions.
77
Participants were randomly assigned to a computer station after completing a
participation agreement form. Each participant was instructed to put on headphones
and click a link on the desktop that would launch the experiment. A multimedia
presentation introduced the participants to the study, required a sound check, and
presented the following instruction:
“You will soon view four instructional videos on various topics. Your goal is to learn as much as you can about each topic. Two videos will be segmented into shorter clips at which you will be prompted to generate a gist keyword that sums up the content you just viewed. One video will allow you to create gist keywords while the video plays. In this condition, if you initiate an annotation, the video will pause and will play once you click the play button. One video will play continuously and you will need to create gist keywords while it plays. This video will not allow you to pause. After viewing the videos, you will be asked to judge how well you understood each topic and predict how you would do on a test on each topic. You will then be asked to complete a multiple-choice test for each topic. Do the best you can and if you have technical difficulties, please raise your hand.”
All participants were instructed that they would first complete a brief profile
survey to collect demographic information and report their familiarity with video
annotation systems. Participants viewed a sample video, produced a practice
annotation, completed a sample JOL question and POP, and then answered a sample
inference multiple-choice question based upon the sample video to counteract
participant tendencies to base judgments upon surface memory alone (Jaeger & Wiley,
2014; Thiede et al., 2011). Participants were then encouraged to ask clarifying
questions about the procedure.
After being randomly assigned a condition, participants viewed four instructional
video topics. The videos were delivered through four video annotation conditions
(random video annotation, paragraph video annotation, learner-controlled video
annotation, and simultaneous video annotation). After viewing the videos, participants
78
completed a JOL and POP for each video topic. After completing the judgments of
learning above, participants completed a learning performance test that included six
inference and six recall items. After completing tests for all four video topics, a global
score was displayed for overall performance for all videos. Participants spent
approximately 60 minutes to complete the study.
Results
Recall and Inference Test Performance and Metacomprehension Accuracy
Descriptive data on test performance and metacognitive judgments are reported
in Table 3-1. A 4 (Video annotation condition: random video annotation, paragraph
video annotation, learner-controlled video annotation, and simultaneous video
annotation) X 2 (Test type: recall, inference) repeated measures ANOVA on test
performance indicated no significant main effects for test type with recall (M = .58) and
inference (M = .54) test performance, F(1, 96) = .011, p = .92.
Textbase and Situation Model Disruption Effects
Two one-way repeated measures ANOVAs were conducted to evaluate
differences across each of the four video annotation conditions with respect to recall
and inference test performance. In all two analyses, assumptions of sphericity had not
been violated. There was no significant effect of video annotation group on recall test
performance, F(3, 144) = 1.45, p =.23, or inference test performance F(3, 144) = 2.23, p
=.14 which was contrary to Hypothesis 1a that posited that split-attention effects would
significantly hinder recall and inference test performance of the simultaneous video
annotation condition (Chandler & Sweller, 1996). Further, the disruption of
comprehension hypothesis (Hypothesis 1b) was not confirmed in Experiment 2 in that
79
no significant differences were observed between random and paragraph video
annotation in either recall or inference test performance. See Figure 3-4.
Significant effects, however, were detected for total test performance across
video annotation conditions F(3, 144) = 2.61, p =.05, partial η2 = .05. Specifically,
participants in the random video annotation condition significantly outperformed in total
test performance (.62) than participants in the paragraph annotation condition (.53)
[t(48) = 2.44, p =0.02, d = .48] and the simultaneous condition (.53) [t(48) = 2.43, p
=0.02, d = .42]. It is to be noted that there were no differences in total test performance
between random segmentation and learner-controlled annotation. This supports
Hypothesis 1b that video annotation may impact performance differently dependent
upon when or how the annotation was initiated.
Metamemory and Metacomprehension Accuracy
Two one-way repeated measures ANOVAs were conducted to evaluate
differences across each of the four video annotation conditions with respect to JOL and
POP magnitudes. No significant differences across conditions were identified for either
JOL, F(3, 144) = 2.15, p =.10, or for POP, F(3, 144) = 2.07, p =.10.
Just as in Experiment 1, two separate measurements of between-subjects
metacomprehension and metamemory accuracy were calculated. Table 3-2 reports
gamma correlations for Experiment 2. See Experiment 1 for a description of how
accuracy was calculated and interpreted. Among the four video annotation conditions,
metamemory and metacomprehension accuracy was significant for paragraph
annotation although low.
A second measure of metamemory and metacomprehension accuracy was
calculated as a Pearson r correlation between POP and scores for recall and inference
80
test performance respectively. See Table 3-3. Overall, this second measure of accuracy
provided convergent evidence that accuracy was low and insignificant for all annotation
conditions with the exception of paragraph video annotation condition.
Immediate Annotation Effects upon Metacomprehension
Hypothesis 3a was confirmed in that metacomprehension accuracy was
insignificant and low across all conditions although paragraph video annotation did
result in significant metamemory accuracy and reached a moderate level of significance
for metacomprehension accuracy. A second measure of metacomprehension accuracy
that used the relationship between the POP and actual inference test performance,
however, was significant for paragraph video annotation (r = .27). There was no
evidence to support Hypothesis 3b that video annotation would aid comprehension and
result in improved relative metacomprehension accuracy.
Interactions between Experiment 1 and Experiment 2.
Three repeated measures split-plot ANOVAs were employed to evaluate whether
there were significant interactions between segmentation and video annotation
conditions with respect to recall, inference, and total test performance. There were
statistically significant interactions between the effects of segmentation and video
annotation on recall test performance, F(3, 294) = 5.94, p =.001, partial η2 = .06. Post-
hoc independent t-test comparisons for recall test performance across conditions
indicated significant differences between random segmentation and random video
annotation t(98) =2.96, p = .004 with a medium effect size (d = .60). No other significant
differences in recall test performance between segmentation and annotation conditions
were identified, (paragraph segmentation vs. paragraph video annotation, learner-
controlled segmentation vs. learner-controlled video annotation, and no segmentation
81
vs. simultaneous annotation). In general, it appears that random video annotation
assisted with recall of facts. See Figure 3-5.
There were statistically significant interactions between the effects of
segmentation and video annotation on inference test performance, F(3, 294) = 3.89, p
=.01, partial η2 = .04. For inference test performance across conditions, there were
significant differences between random segmentation and random video annotation
t(98) =3.12, p = .002 with a medium effect size (d = .62). No other differences in
inference test performance between segmentation and annotation conditions were
identified, (paragraph segmentation vs. paragraph video annotation, learner-controlled
segmentation vs. learner-controlled video annotation, and no segmentation vs.
simultaneous annotation). In general, it appears that random video annotation assisted
with comprehension. See Figure 3-6.
There were statistically significant interactions between the effects of
segmentation and video annotation on total test performance, F(3, 294) = 7.71, p =.00,
partial η2 = .07, which can be considered a moderate effect size (Cohen, 2013).
Independent t-tests indicated differences between random segmentation and random
video annotation t(98) =3.73, p = .00 with a large effect size (d = .75) (Cohen, 2013),
and also between no segmentation and simultaneous video annotation t(98) = -1.95, p =
.05 with a medium effect size (d = .39). See Figure 3-7 for comparisons across
conditions and experiments. Differences in total test performance between paragraph
segmentation vs. paragraph video annotation and learner-controlled segmentation vs.
learner-controlled video annotation were insignificant. As predicted, potential effects
82
from split-attention in the simultaneous video annotation hindered test performance
(Chandler & Sweller, 1996).
As expected, metacomprehension accuracy across Experiment 2 was low and
insignificant although paragraph video annotation did reach a significant but low level of
metacomprehension accuracy (G = .21, p = .06; r = .27) in comparison to the low and
insignificant level of accuracy for paragraph segmentation.
Discussion
The purpose of Experiment 2 was to evaluate the impact of various video
annotation conditions upon learning and metacognitive monitoring performance.
Hypothesis 1a that simultaneous annotation would be significantly lower than the other
conditions that pause the video was not confirmed. This is surprising considering
mechanical demands of simultaneous annotation during video playback (Kobayashi,
2005) or the cognitive load demands of split-attention effects (Chandler & Sweller,
1996). One potential explanation for a lack of significant differences was the fact that
participants produced few annotations in the simultaneous annotation group (M = 3.2
annotations) which is a little over half of the number of annotations in either random or
paragraph video annotation conditions. Underutilization of the annotation feature even
when instructed to use the annotation tool while under observation suggests that the
task of viewing an instructional video supersedes that of producing annotations. Where
on the timeline these annotations were generated may also be an important factor to
consider in future research since many participants were observed delaying annotation
until the end of the video as if attention was first directed at the video and annotation
was a secondary thought. Because trace data was not collected as to where on the
timeline the annotations were produced, it was not possible to test whether the timing of
83
the annotation was a significant factor. Future research is needed to determine the
subjective factors as to why learners use or do not make use of specific affordances.
Additionally, Hypothesis 1b that disruption to the textbase (Kintsch, 1998)
through random annotation would significantly hinder learning was not confirmed. In
fact, there was evidence to suggest that random video annotation actually aided total
test performance. Although no significant differences were identified for recall and
inference test performance across all four video annotation conditions, significant
differences were identified for the total scores with random annotation significantly
outperforming paragraph and simultaneous video annotation. This finding was
surprising in light of the negative effects of random segmentation upon learning in
Experiment 1. One potential explanation for this finding is the fact that the random video
annotation created germane cognitive load conditions (Paas & van Merrienboer, 1994;
Sweller, 2010) and greater attention through reduced mindwandering through an
interpolated activity (Szpunar, Jing, & Schacter, 2014). Based upon this reasoning,
random annotation may have served as a stimulus to engage comprehension at the
textbase and situation model as compared to paragraph annotation which may have
resulted in automatic viewing and shallow processing associated with instructional video
(Salomon, 1984).
It is also important to note that a lack of significant differences between random
video annotation and learner-controlled annotation in total test performance supports
the hypothesis that learner-control introduces disruption akin to the random annotation
condition. Potential disruptions of learner-controlled video annotation to comprehension,
however, were likely moderated by underutilization of annotation tool (M = 2.8). While
84
the literature suggests the positive effects of interpolated activities in the midst of
instructional video (Szpunar et al., 2013), the results from Experiment 2 support the
principle that interpolated activities can aid learning but the timing of the interpolation
may be an important factor for video annotation conditions.
Metacomprehension accuracy across all four video annotation conditions was
low. This supports Hypothesis 3a that immediate keyword production facilitates
conditions that introduce error into metacognitive monitoring judgments (Thiede &
Anderson, 2004; Thiede et al., 2005). There was no significant evidence to support
Hypothesis 3b that potential benefits of multimodality in processing and integration with
prior-knowledge (Mayer 2014) would in turn benefit metacomprehension accuracy as
had been observed with immediate manipulations such as self-explanation (Griffin et al.,
2008) or immediate concept mapping (Redford et al., 2012). In fact, random video
annotation produced extremely high performance in both recall and inference tests, but
extremely low metacomprehension accuracy (G = -.12; r = 16).
Comparison of Experiment 1 and Experiment 2
Comparisons in total test performance across Experiment 1 and 2 indicated
significant interactions between segmentation and annotation conditions. Hypothesis 2a
that annotation could aid learning was confirmed in the case of random video annotation
(Kobayashi, 2005). Random video annotation had positive significant effects upon
learning, while random segmentation hindered learning. Random segmentation harmed
learning because of a disruption to the textbase and situation model (Kintsch, 1998), but
a generative activity such as annotation appeared to provide a means to improve
learning when inserted in the randomized segment. This interaction was unexpected. As
discussed above, random annotation may lead to germane cognitive load conditions
85
(Paas & van Merrienboer, 1994; Sweller, 2010) and greater attention through reduced
mindwandering (Szpunar et al., 2014).
The benefits of paragraph and learner-controlled video annotation, however,
were not confirmed. This confirms Hypothesis 2b that annotation operationalized as a
gist keywords would not improve learning although immediate annotation might have an
impact upon metacomprehension monitoring (Thiede et al., 2005).
Significant differences in recall and inference scores were also observed
between the no segmentation and simultaneous annotation condition which confirmed
hypothesized split-attention effects of simultaneous annotation in comparison to a
control without segmentation (Chandler & Sweller, 1998; Thomas et al., 2016). The
moderate effect size (d = .39) of simultaneous annotation in comparison to the no
segmentation condition can, in part, be explained by underutilization of annotation
production (M = 3.2). While underutilization of pause or annotation is an intuitive
explanation, future work is needed to test the degree to which specific levels of
utilization impact performance.
Across both experiments, metacomprehension accuracy varied according to
condition. The no segmentation condition reached significance and a moderate level of
metacomprehension accuracy (G = .44; r =42). Paragraph annotation reached
significance and a low level of metacomprehension accuracy (G = .21; r = .27) in
comparison to the insignificant and low paragraph segmentation condition (G = -.05; r =
.05). This suggests that annotation at the paragraph level resulted in utilization of more
reliable cues as compared to error-laden cues in the case of paragraph segmentation
(Koriat, 1997). One explanation is that the paragraph segmentation condition resulted in
86
perceptions of ease and shallow processing (Salomon, 1984) as compared to the more
cognitive intensive task of generating annotations.
While metacomprehension accuracy was low for both random segmentation and
annotation, this occurred in the context of significant and divergent total test
performance scores. This suggests that judgments, even after a generative activity has
repaired learning performance, are based upon cues derived from a broken situation
model as compared to the repaired situation model. In essence, the disruption becomes
the point of focus as compared to a robust mental model that arises from an effective
generative activity such as summarization. An analogy to the random conditions
employed in the two experiments might consist of a ceramic vase that has been
dropped and broken, while annotation acts as means to repair the structure. Attention,
in this case, centers upon the fracture lines as compared to the overall structure even
when it is possible to repair the vase. In this way, these fracture points become the non-
comprehension based cues upon which JOLs are made and account for low
metacomprehension accuracy (Rawson & Dunlosky, 2002). In the case of paragraph
annotation, because the fracture points are less disruptive, there is less focus upon the
disruption, and as a result metacomprehension accuracy improves. This principle
extends to the least disruptive condition, namely, the no segmentation condition which
had the highest level of metacomprehension accuracy. The conditions discussed above
appear to support Rawson and Dunlosky’s (2002) hypothesis that as coherence of text
increases so does metacomprehension accuracy.
Metacomprehension accuracy for learner-controlled segmentation and
annotation was low. Although the pause button and annotation field for both conditions
87
were underutilized, metacomprehension accuracy was low. This suggests that learner-
control whether in the form of pause or video annotation weakens the ability to
accurately monitor comprehension levels. Future work is needed to understand how
learner-control may influence a learner’s reliance upon cues that introduce error into
metacognitive judgments.
Comparison of metacomprehension accuracy for no segmentation and
simultaneous video annotation conditions indicated moderate levels of accuracy for the
no segmentation condition and low levels for the simultaneous condition. Although
simultaneous annotation confounds split-attention effects and immediate annotation
effects, the findings are a replication of the benefits of non-segmented video in
comparison to simultaneous video annotation (Thomas et al., 2016). Viewing a video
from beginning to end without interruption appears to support metacomprehension
accuracy.
Scientific and Practical Significance
Video annotation is a complex activity which often results in a convergence of
conditions in the form of segmentation and interpolated generative activity. In the
context of expository instructional video, the results from Experiment 1 do not provide
support for the segmenting principle which predicts that segmentation should improve
learning outcomes in terms of recall and inference test performance. These results,
however, do provide support for theories of comprehension that predict that disruptions
at specific levels of comprehension (surface, textbase, and situation model) impact
recall and inference test performance in significant and differing ways. In addition, the
results further support the hypothesis that segmentation as a form of disruption to either
the textbase or situation model of representation undermines metacognitive monitoring
88
processes. One potential explanation for the deleterious effects of segmentation upon
metacomprehension accuracy is a learner’s greater attention to superficial cues that
introduce error into metacognitive judgments. Future work, however, is needed to
evaluate the specific types of cues that inform judgments in the context of segmentation
of video-based learning.
The results from Experiment 2 provide partial support for the potential of
generative activities such as video annotation to aid learning. In particular, the positive
effects of the random video annotation condition upon recall and inference scores
suggest that the timing of annotation may result in differing degrees of germane
cognitive load in which learners are more likely to invest greater mental effort. In this
case, random annotation allowed for substantial repair to both the textbase and
situation model in contrast to paragraph annotation or learner-controlled annotation. The
results from all four video annotation conditions support the long-delayed hypothesis
that immediate summarization weakens the ability to make accurate judgments and
results in low metacomprehension accuracy. Even in the case of random video
annotation which resulted in high recall and inference scores, metacomprehension
accuracy was low. This is an important finding because this suggests that even when
video annotation aids learning, there may be a weakening of metacognitive monitoring
processes.
Unlike traditional transient lecture or broadcast media environments, one of the
primary advantages of video-based learning is the ability to pause and review. This
study suggests that system or learner-controlled segmentation and video annotation
may in fact undermine a learner’s ability to monitor learning levels and thereby may
89
result in less effective metacognitive control behaviors. The degree to which
metacognitive monitoring influences metacognitive control processes is an essential
question for future research to address in video-based learning contexts since there is a
possibility that the affordances of interactive video technology such as video annotation,
which can aid learning in specific circumstances, undermine the ability to implement
effective metacognitive control behaviors such as restudy.
Limitations
Although the four videos supported the educational relevance of the research in
terms of grade level, content and duration, the videos were carefully designed with the
purpose of maintaining coherence between the narration and images along with an
attempt to avoid decorative graphics. To apply these findings to lecture-capture formats,
typical of post-secondary settings such as a recorded lecture, in which there is little
coherence between the narration and images presented would be inappropriate. In
addition, the annotation condition merely required each participant to produce five
keywords to summarize content as compared to the likely variance in annotation
quantity and quality to be encountered in an ecological setting. Future work is needed to
identify current usage behaviors of video annotation tools in educational settings and
how usage of specific tools support or hinder learning.
Conclusions
The use of video annotation is likely to grow as web-based streaming and video
annotation tools become ubiquitous in educational settings, yet how affordances such
as annotation and pause-initiated segmentation impact learning and metacognitive
monitoring has not been fully addressed. The two experiments presented here provide
evidence that segmentation and annotation of video impact learning performance and
90
metacognitive monitoring in significant and at times differing ways. One objective was to
identify conditions, if any, where both test performance (recall and inference) and
metacognitive monitoring performance were high. In general, the non-segmented video
without annotation condition produced substantial learning in both recall and inference
test performance along with high metacomprehension accuracy. Viewing a multimodal
video of expository content without interruption appears to aid learning and to support
effective metacognitive monitoring processes. This study also demonstrates that
learning and metacognition in video-based environments is dependent upon when
segments are initiated, who or what initiates segments, and what type of activity occurs
during a segment of a video timeline. Examining why specific video-based learning
conditions impact metacognitive monitoring and control is important in light of the
continued growth and use of instructional video in online and blended learning
environments where learner success depends to a large degree on the ability to self-
regulate cognition and learning.
91
Figure 3-1. Combined screenshot of video screen and subsequent annotation screen for random and paragraph video annotation conditions.
Figure 3-2. Combined screenshot of learner-controlled video annotation screen and annotation screen.
92
Figure 3-3. Screenshot of simultaneous video annotation screen.
Table 3-1. Mean Test Scores and Judgment Magnitudes for Experiment 2. Random video
annotation Paragraph video annotation
Learner-controlled video annotation
Simultaneous video annotation
Recall % Correct
.63 (SE .03)
.55 (SE .03)
.59 (SE .03)
.55 (SE .03)
Inference % Correct
.60 (SE .03) .52(SE .03) .53 (SE .03) .53 (SE .03)
Total % Correct .62 (SE .02) .53 (SE .03) .56 (SE .03) .53 (SE .03)
Judgment of Learning (JOL)
4.2 (SE .15) 4.7 (SE .18) 4.5 (SE .17) 4.7 (SE . 18)
Prediction of Performance (POP)
.60 (SE .03) .67 (SE .03) .66 (SE .02) .66 (SE .03)
Note. Average test scores for recall and inference questions are proportions based upon a maximum of 6.
93
Figure 3-4. Comparison of recall and inference test performance across conditions.
Table 3-2. Relative Metamemory and Metacomprehension Accuracy for Experiment 2 Group JOL to Recall JOL to Inference (metamemory) (metacomprehension) M (SE) M (SE) Random video annotation
-.08 (.11)
-.12 (.16)
Paragraph video annotation
.25 (.12) * .21 (.13) (p = .06)
Learner-controlled video annotation
.09 (.15) .16 (.16)
Simultaneous video annotation
-.01 (.15) .13 (.15)
Note. A * indicates statistically significant correlation at p < .05.
94
Table 3-3. Relative Accuracy for POP for Recall, Inference, and Total Test Performance for Experiment 2
Group POP to Recall Metamemory
accuracy
POP to Inference Metacomprehension
accuracy M (SE) M (SE) Random video annotation
-.06 (.12)
.16 (.15)
Paragraph video annotation
.21 (.11)
.27 (.10)*
Learner-controlled video annotation
.05 (.15)
.25 (.14)
Simultaneous video annotation
.08 (.13)
.12 (.13)
Note. A * indicates statistically significant correlation at p < .05.
Figure 3-5. Comparison of recall test proportional means across experiments.
95
Figure 3-6. Comparison of inference test proportional means across experiments.
Figure 3-7. Comparison of total test proportional means across experiments.
96
APPENDIX A EXPERIMENT 1 CONSENT FORM
Dear participant, Thank you so much for taking the time to participate in this study to learn about how people learn from instructional video. The following study is interested in exploring how instructional video impacts learning and the ability to self-evaluate learning.. Title of Study: VIDEO SEGMENTATION EFFECTS UPON LEARNING AND METACOGNITIVE MONITORING Investigators: Aaron Thomas [email protected] Contact Phone Number: (352) 273-1575 Scientific Purpose of the Study: The purpose of this study is to determine how specific use cases (segmentation of the video timeline) of instructional videos based upon various historical and scientific topics impact learning performance on recall and inference tests and metacomprehension accuracy, namely a learner’s ability to accurately predict performance across video topics. In other words, how does instructional video impact a learner’s ability to predict their level of comprehension and ability to remember details from the instructional video under specific segmentation conditions. The specific conditions include the following: segmentation equally distributed in the video timeline but inserted between paragraphs of the video text, random segmentation throughout the video timeline, learner-controlled segmentation, and a control with no segmentation. Procedure: If you choose to participate in this study, you are allowing us to use information that is collected during the normal educational practices of this course. The following types of information may be collected: Survey: A variety of means may be used to document your perspective and learning practices. Artifacts: A variety of artifacts may be collected to document your perspective and learning practices. These may include but are not limited to responses to questions and summaries of instructional content. Online archives: Responses that are housed within the learning management system may be archived. Observations: Observation data in the form of field notes or video recordings may be collected to document your interactions in the computer lab during your session.
An anonymous coding scheme will be applied to all information at the end of the study and prior to analysis. Data will be analyzed and reported in an aggregated fashion and participants will not be identified by name in any reports of our research. Data is maintained on UF secure
97
servers and will be anonymized on the servers. Any and all data downloaded will be deidentified. At the conclusion of the study, the UF database will be deleted.
Risks and Benefits of Participation There are risks involved in all research studies. However, minimal risk is envisioned for participating in this project. You will not be identified by name in any reports of this research; pseudonyms will be used. There are no direct benefits for participating in this research although you may learn something about historical and scientific topics. Time Required and Compensation The study will occur in a computer lab between February and May 2016 at days and times announced in your classes. We anticipate requiring a total of 70-80 minutes of your time to complete this intervention. There will be no compensation for participating in this study although your instructor may award you extra credit if they wish. If you participate, you will be given a certificate of completion that you may present to your professor or instructor for credit. Confidentiality All information gathered in this study will be kept confidential to the extent provided by law. No reference will be made in written or oral materials that could link you to this study. All records will be stored in an encrypted desktop computer in the Principal Investigator's office or in a locked office which is monitored 24-7 by surveillance cameras. When the study is completed and the data have been analyzed, the information will be shredded and/or electronically erased. Voluntary Participation Your participation is strictly voluntary. Non-participation or denied consent to collect some or all of the data listed above will not affect your grades or your status as a student. In addition, you may request at any time that your data not to be included. Contact Information If you have any questions or concerns about the study, you may contact Aaron Thomas, at [email protected] or (352) 273-2243. Questions regarding your rights as a research participant in this study you may contact the UFIRB Office, Box 112250, University of Florida, Gainesville, FL 32611-2250; ph (352) 392-0433. Participant Consent I have read the above information and agree to participate in this study. I am at least 18 years of age.
Print Name: ______________________________________________
Date:____________________________________________________
Signature:________________________________________________
98
APPENDIX B EXPERIMENT 2 CONSENT FORM
Dear participant, Thank you so much for taking the time to participate in this study to learn about how people learn from instructional video. The following study is interested in exploring how instructional video impacts learning and the ability to self-evaluate learning.. Title of Study: VIDEO ANNOTATION EFFECTS UPON LEARNING AND METACOGNITIVE MONITORING Investigators: Aaron Thomas [email protected] Contact Phone Number: (352) 273-1575 Scientific Purpose of the Study: The purpose of this study is to determine how specific use cases (annotation of the video timeline) of instructional videos based upon various historical and scientific topics impact learning performance on recall and inference tests and metacomprehension accuracy, namely a learner’s ability to accurately predict performance across video topics. In other words, how does instructional video annotation impact a learner’s ability to predict their level of comprehension and ability to remember details from the instructional video under specific annotation conditions? The specific conditions include the following: system-controlled segmentation and annotation, learner controlled segmentation initiated by annotation, no segmentation with simultaneous annotation, no segmentation with immediate annotation, no segmentation with delayed annotation, and no segmentation with no annotation. Procedure: If you choose to participate in this study, you are allowing us to use information that is collected during the normal educational practices of this course. The following types of information may be collected: Survey: A variety of means may be used to document your perspective and learning practices. Artifacts: A variety of artifacts may be collected to document your perspective and learning practices. These may include but are not limited to responses to questions and summaries of instructional content. Online archives: Responses that are housed within the learning management system may be archived. Observations: Observation data in the form of field notes or video recordings may be collected to document your interactions in the computer lab during your session.
An anonymous coding scheme will be applied to all information at the end of the study and prior to analysis. Data will be analyzed and reported in an aggregated fashion and participants
99
will not be identified by name in any reports of our research. Data is maintained on UF secure servers and will be anonymized on the servers. Any and all data downloaded will be deidentified. At the conclusion of the study, the UF database will be deleted.
Risks and Benefits of Participation There are risks involved in all research studies. However, minimal risk is envisioned for participating in this project. You will not be identified by name in any reports of this research; pseudonyms will be used. There are no direct benefits for participating in this research although you may learn something about historical and scientific topics. Time Required and Compensation The study will occur in a computer lab between February and May 2016 at days and times announced in your classes. We anticipate requiring a total of 70-80 minutes of your time to complete this intervention. There will be no compensation for participating in this study although your instructor may award you extra credit if they wish. If you participate, you will be given a certificate of completion that you may present to your professor or instructor for credit. Confidentiality All information gathered in this study will be kept confidential to the extent provided by law. No reference will be made in written or oral materials that could link you to this study. All records will be stored in an encrypted desktop computer in the Principal Investigator's office or in a locked office which is monitored 24-7 by surveillance cameras. When the study is completed and the data have been analyzed, the information will be shredded and/or electronically erased. Voluntary Participation Your participation is strictly voluntary. Non-participation or denied consent to collect some or all of the data listed above will not affect your grades or your status as a student. In addition, you may request at any time that your data not to be included. Contact Information If you have any questions or concerns about the study, you may contact Aaron Thomas, at [email protected] or (352) 273-2243. Questions regarding your rights as a research participant in this study you may contact the UFIRB Office, Box 112250, University of Florida, Gainesville, FL 32611-2250; ph. (352) 392-0433. Participant Consent I have read the above information and agree to participate in this study. I am at least 18 years of age.
Print Name: ______________________________________________
Date:____________________________________________________
Signature:________________________________________________
100
APPENDIX C FOUR VIDEO SCRIPTS
Video 1: Naval Warfare
Though to some extent it involved operations against surface raiders, the contest
in the Atlantic was primarily a war between German submarines, striving to sink the
ships on which the United Kingdom depended, and Allied surface and air antisubmarine
forces.
As in the Pacific war, the beginning of the war in Europe saw the opening of an
unlimited submarine campaign. The U-boats available to the Germans at the time
operated mainly in the ocean approaches to England, making daylight attacks from
periscope depth. Early results were good, and aggressive submarines exposed
themselves with sonar. Antisubmarine counter-measures proved more effective than the
Germans had anticipated, and submarines shifted their attention to lone ships.
The fall of France in 1940 gave Germany advanced bases on the French Atlantic
coast, allowing U-boats to patrol farther into that ocean and thus greatly expanding the
area open to attack. British ships and planes had to be diverted from antisubmarine duty
to the protection of their own coasts. As a counter to British antisubmarine tactics, the
U-boat force changed their own doctrine and began surface night attacks on convoys.
Again, these were successful and huge tonnages of shipping were sunk. U-boats,
including some "high scorers," were lost, but the balance was profitable and the
initiative was firmly in the submariners' hands.
During the early months of the war, a few German warships such as the Admiral
Graf Spee and "merchant cruisers" such as the Atlantis accounted for some merchant
tonnage. Operating at long range in the face of overwhelming British naval strength,
101
surface raiders never had more than a minor influence on the course of the Atlantic war.
As the war went on, the "fleet in being" tactics employed with such ships as Tirpitz,
Scharnhorst, and Gneisenau carried their share of weight in the minds of Allied
admirals. The destruction of Convoy PQ-17 on the North Atlantic Murmansk Run was
due, in largest part, to apprehension over the chance of a surface sortie from Norwegian
bases.
British defensive measures against the new night attacks included centralized
convoy routing, wide dispersal routing and strengthening of escorts. This made it more
difficult for U-boats to find and attack convoys. On the other hand, expanded German
submarine construction programs now began to produce results. Beginning in 1941,
about 20 new U-boats a month entered service, bolstering the relatively small force with
which the Kriegsmarine had entered the war. As the undersea army expanded,
however, there was some loss in crew training and experience.
During the summer of 1941 individual surface night attacks on convoys gave way
to the "wolfpack" attack. Submarines patrolled areas where convoys could be expected.
On making a contact, U-boats did not attack but shadowed the convoy, signaling other
submarines to join the attack. Multiple night attacks meant the chance of higher kills at
less risk to the submarines. Land-based long-range Focke-Wulf FW 200 patrol planes
also participated in these operations.
As U-boats extended their operating areas farther across the Atlantic, it became
necessary to escort convoys through the entire transoceanic passage instead of only
the western approaches to England. Since President Roosevelt involved the U.S. Navy
to an increasing degree in the Atlantic war, however, pressure on the Royal Navy was
102
somewhat eased. After Pearl Harbor, the U.S. Navy was officially in the submarine war,
but lack of experience made it less than fully effective at first.
U-boat operations in the Caribbean and in American coastal waters were highly
productive during the spring and summer of 1942. American antisubmarine measures
were largely improvised and ineffective at first; it was not until the fall of that year that
interlocking coastal convoys and air patrols made U-boats tend to return to the open
ocean.
Two technical developments, radar and airborne depth bombs, were by now
contributing to the antisubmarine war. Patrol planes, equipped with underwater bombs
and search radar as well as high-intensity searchlights for night attack, made the U-boat
transit area in the Bay of Biscay increasingly dangerous. Ship and aircraft radar could
detect surfaced submarines at a distance, even at night or in foul weather. Convoy
escorts' radar and HF/DF gave them effective means of defense against wolfpacks.
Large-scale shipbuilding programs were well under way in the United States and the
United Kingdom. These were intended not only to produce cargo hulls faster than they
could be sunk but to provide antisubmarine patrol and escort ships in more adequate
numbers. Large numbers of what the British called frigates and the United States called
destroyer escorts (DE), as well as escort aircraft carriers (CVE), were aimed directly at
the submarine threat. Ahead-throwing antisubmarine weapons, such as Hedgehog and
Squid, increased ships' capabilities.
The main action shifted back into the North Atlantic late in 1942. This area was
still too distant for the long-range land-based patrol planes from the United Kingdom or
North America, and some extremely large wolfpacks were frequently assembled to
103
overwhelm a convoy's escort force. Shipping losses were heavy during the winter of
1942-43.
In 1943, the balanced shifted. New escort ships and CVEs were increasing in
quantity. The addition of carrier antisubmarine planes to cover the mid-Atlantic area had
a positive effect. The increase in numbers of available antisubmarine ships enabled
hunter-killer groups, one CVE with a number of DEs, to patrol submarine operating
areas. Higher frequency search radar proved valuable. U-boat wolfpacks continued to
operate into the spring of 1943, but in May a pack was decisively defeated in an attack
on Convoy ONS-5. Not only did ship sinkings decrease, but submarines losses rose.
During May 37 U-boats were lost; 34 went down in July. Many of these were sunk by
airplanes, and a sizable proportion were sent to the bottom of the Bay of Biscay,
departing on patrol or returning from it.
The U-boat force tried various expedients to right the balance. Dispersing at first
into the South Atlantic to avoid an attack, they moved north again in the fall to try
acoustic bombing torpedoes against escort convoys. In October, one escort ship and
three merchant ships were sunk - at a cost of 22 U-boats.
Many U-boats had their antiaircraft batteries considerably augmented, receiving
37-mm guns and twin or quadruple 20-mm mounts. The tactic was now to remain on the
surface and "shoot it out" with an attacking airplane. Results were not worthwhile, and
submarine losses continued to be heavy.
U-boat warfare was primarily defensive through the winter of 1943-44. Relatively
few boats went to sea, and the toll they took was meager. Attempts were made to attack
104
the enormous concentration of shipping taking part in the invasion of France (June
1944), but massive antisubmarine screening made the efforts useless.
During the latter part of 1944, the Germans introduced snorkel, allowing their
submarines to operate without surfacing. The snorkel could not be detected by current
search radar, and by using this new device and resorting to "bottoming" tactics the
submarines were able to gain some protection from radar and sonar. As French bases
were lost, submarines shifted to ports in Norway and Germany. Some successes were
achieved during the winter of 1944-45, but by the spring of 1945, new techniques and
more sensitive radar had again tipped the scale. A new high-performance U-boat, the
hydrogen-peroxide-fueled Type XXI, was an excellent design with unprecedented
underwater performance, but it was completed too late for war service.
Throughout the war, convoy operations proved the most effective measure both
in protecting convoys and in sinking U-boats. Patrol measures were far less efficient.
During 1939-45, a total of 2,753 Allied ships, of 14,557,000 gross tons, were sunk at a
cost of 733 German and 79 Italian submarines.
Video 2: Norse Settlements
The Viking Age Scandinavians had a lasting impact upon the peoples of western
Europe. Their settlements, commercial ventures, and raids affected cultures from the
Russian plains to the Irish Sea and from northernmost arctic Norway to the
Mediterranean. During the Viking period (ca. 790 – 1100), Scandinavians also ventured
across the North Atlantic, settling the Shetland and Faroe islands, Iceland, and
Greenland and making a brief appearance on the shores of America. This North
Atlantic arm of the Viking Age expansion connected the eastern and western
105
hemispheres, and, for a few years at the end of the tenth century, a single language and
culture reached from Kiev to the gulf of St. Lawrence.
By the beginning of the Viking Age, most of Scandinavia was organized into a
maze of local chieftainships. Chieftains were expected to be effective in protecting their
clients and aggressive in pressing for every advantage for themselves and their
supporters in their struggles with rival chieftains. Traditional law codes (which became
increasingly formalized during the Viking period and were written down soon after) and
the independence of farmer-clients served somewhat as a restraint on chiefly ambition,
but warfare and blood feuds were still commonplace. While Norway, Sweden, and
Denmark were known as geographical terms, nothing resembling a nation-state (even
by eighth-century standards) existed in pre-Viking Scandinavia.
As wealth from abroad entered Scandinavia, and as Scandinavian merchants,
travelers, and mercenaries learned more of the kingdoms of the outside world, the
combination of new resources and new ideas seems to have sparked increased
competition among local chieftains and petty kings. Agriculture also prospered as a
period of warm climate (now known as the Little Climatic Optimum) lengthened growing
seasons in northwestern Europe. Population seems to have enlarged, which led to the
settlement of the uplands and the extension of Norse farms into arctic Norway. The
expansion of territorial boundaries during the Viking Age provided an outlet for this
growing rural population and yielded new territory for the losers in the intensifying
struggles among chieftains for dominance.
Neither a growing population nor competing chieftains would have produced the
Viking expansion had the means for overseas travel, trade, and conquest been
106
lacking. Through the efforts of maritime archaeologists, we know a good deal about
Viking period ships and their construction. By the late eighth century, Scandinavian
clinker-built ships had reached a high level of perfection, combining lightness and
shallow draft with great strength and sea-keeping ability. Viking ships could land on any
beach, penetrate far up rivers, and survive North Atlantic storms on the open sea.
While strong and elegant, the clinker-built Viking ships had two significant
limitations. They required a long run of high-quality timber (preferably oak) for the keel
and naturally curved timbers for the stem and stern pieces. Since this quality timber
was absent in the North Atlantic islands, settlers in Iceland and Greenland found it hard
to replace oceangoing ships lost at sea. The Viking design also sharply limited cargo
capacity—even the knarrs (trading vessels) could carry only a fraction of the cargo of
the later carvel-built Hanseatic cogs that came to dominate European commerce in the
later Middle Ages. Viking ships could reach distant points, but they could not carry
enough passengers and supplies to ensure a viable transatlantic foothold. Population
movement across the North Atlantic thus required a chain of settlements, each
providing population and resources for successive ventures westward.
Scandinavian North Atlantic settlement was a gradual process taking two
hundred years to complete. Norse colonists settled the Shetlands and Orkneys around
the year 800 and (according to tradition) Iceland around AD 874. Greenland was
settled from Iceland by Eirik the Red around 985. Vinland was explored from Greenland
and a settlement was attempted by the sons and daughter of Eirik around the year
1000. Island chieftains who (like Eirik) had failed in local power struggles provided the
ships and capital to sponsor further voyages of exploration and
107
settlement. Unsuccessful farmers and dissatisfied younger siblings from successively
filled island ecosystems provided the bulk of the personnel. The first settlers in a new
land had the ritually important right to name the landscape and economically vital right
to claim the best pasture and hunting grounds. As prime grazing is often patchy and
limited in the North Atlantic islands, this initial division of resources set the stage for
increasing economic and social hierarchy in later generations.
During the eleventh and twelfth centuries, the Scandinavian North Atlantic
enjoyed modest prosperity. Island populations seem to have stabilized at low levels;
Iceland’s population was probably between thirty thousand and sixty thousand, and
Greenland’s was six thousand at most. While state formation was taking place in the
Scandinavian homelands, the more distant North Atlantic islands seem to have
maintained a somewhat archaic chiefly oligarchy. Christianity had spread as far as
Greenland by the year 1000, and most Scandinavians were at least nominally Christian
by 1100. Chiefly competition was now conducted through the endowment of churches
and monastic houses as well as by the traditional sheep stealing and house burning. In
Iceland and probably Greenland, sagas and family histories were being composed, and
poets and skalds from the North Atlantic were still in demand in continental courts.
Along with prosperity came the beginnings of decline. Iceland’s chiefly
dominance struggles had thrown up six great families who escalating warfare
increasingly exhausted local resources. Overgrazing in many areas triggered massive
and irreversible soil erosion, turning whole districts into rocky wasteland. After 1250,
volcanic eruptions coupled with the end of the favorable weather of the Little Climatic
108
Optimum added to man-made disaster, and increasing numbers of North Atlantic
farmers slipped from freeholder to tenant status.
After 1264, Iceland and Greenland became part of the Norwegian kingdom just
as that kingdom was about to enter a long period of decline. Their local oceangoing
ships long lost, the settlers of the western Atlantic depended upon continental
merchants to carry their trade. Icelanders bitterly complained that the promised six
ships per year seldom arrived, and it seems to have taken a papal letter five years to
reach Greenland. The eastern Atlantic settlements in the Shetlands and northern
Scotland were luckier, as they were becoming increasingly integrated into the stock fish
trade through the Hanseatic League.
The late thirteenth and the fourteenth centuries saw accelerated decline in the
western North Atlantic. The onset of the Little Ice Age (ca. 1250-1860 in the North
Atlantic) crippled farming, and economic hardship in Norway affected transatlantic
trade. Literature declined, and the populations of Iceland and Greenland became
locked in a struggle for bare survival. By the later Middle Ages, the Norse North Atlantic
was no longer the cutting edge of an expanding European population but a demoralized
and isolated backwater.
Video 3: Experimental Design
Experiments involve introducing a planned intervention (usually referred to as
a treatment) into a situation, with the intent of inferring the association between the
treatment and a resulting change or outcome. Good experimental design facilitates this
inferential process in three ways.
109
First, it translates all aspects of one's hypothesis--the statement of expected
relation of treatment to outcome--into operational terms: subjects, behaviors, situation,
equipment, procedures, and so on. These permit the hypotheses to be tested
empirically.
Second, it rules out those alternative explanations which provide the most
serious challenge to the treatment as the explanation for change. For example,
because of faulty design, an experimental group was tested, exposed to a treatment,
and post-tested. Improvement on the second testing could be attributed to familiarity
with the test, thus providing an alternative explanation.
Third, it facilitates relating the change to other variables, thus permitting better
understanding of the relationship. For example, with proper design, one could tell
whether a treatment was more successful with men than women and with older than
younger subjects, or its relation to any other variable included in the design.
The first step in experimental design is to translate expectations expressed in
one's hypotheses into operational terms. For example, given the hypothesis that
'outlining in advance improves writing,' one must specify what constitutes sufficient prior
organization to be considered outlining and in what aspects of writing one expects to
improve. The accuracy of this translation is crucial. If what passes for outlining in the
study does not accurately reflect what is typically intended by the term, or if the writing
measure is inaccurate or insensitive, then misleading conclusions could result.
Following operationalization, one must create a situation in which the treatment
can occur as intended and changes can be sensed. Sometimes one compares the
status of experimental subject from before and after the intervention. In other instances,
110
experimental subjects may be compared with an comparable untreated group--a control
group. In still other instances, post-treatment condition is compared with estimates of
the untreated state, for instance, test norms or estimates made from previous data from
comparable groups.
By appropriate choice of design, one can rule out whatever alternative
explanations may be important rivals to that intended. For example, if a control group is
used, the groups may not have been equivalent to begin with, or dropouts may make
them nonequivalent at the end. Alternative explanations common to many studies have
been identified (see below) but some may be unique to a study. For example, if subjects
are allowed to complete a test at home, their score may reflect more their ability to seek
help than their own achievement.
Assuming that the data support one's expectations, these steps in the logic
follow: since the results were as predicted; and since there is no reasonable explanation
for the phenomenon other than the treatment (others having been ruled out by one's
design); then the hypotheses escaped disconfirmation. While one cannot test the
hypotheses in every possible type of situation, one infers that similar predictions would
prove accurate in similar instances. With each such confirmation, confidence in the
hypotheses increases. However, even a single disconfirmation, without reasonable
explanation, is sufficient to disprove it.
It is difficult to provide sufficient experimental control to protect against every
possible alternative explanation. Further, one typically buys protection at a price. For
example, a laboratory gives more complete control, but laboratory circumstances are
rarely like those to which one hopes to generalize. Yet, natural circumstances may
111
provide too little control. Psychologist Philip Zimbardo and his colleagues provided an
interesting example of this dilemma and its solution. They hypothesized that the
paranoid behavior frequent in elderly people was due to the gradual unnoticed loss of
hearing common in old age. An expensive longitudinal design following subjects over
time would have been inconclusive because of the subjects' varying social experiences.
In addition, it would involve the unethical behavior of withholding hearing loss
information to see whether the paranoid behavior developed.
The researchers devised a creative experimental design. Post-hypnotic
suggestion produced a temporary unnoticed hearing loss in college student volunteers,
with resulting displays of paranoid behavior. To eliminate rival alternative explanations,
two control groups of similar subjects were established: one received the post-hypnotic
suggestion of a hearing loss of which they would be aware and another received a
neutral post-hypnotic suggestion in order to show that the hypnotic process itself did not
induce paranoid behavior. All subjects were exposed to controlled similar social
experiences following hypnosis. Paranoia was shown to follow only an unnoticed
induced hearing loss. Altogether, this is a clever use of experimental design for an
otherwise difficult problem.
However, using a laboratory-like setting may not be without costs to the validity
of one's inferences. Impressed by the scientific laboratory, subjects may have tried to
please the researcher. In addition, the researchers, knowing which was the
experimental group, may have unintentionally cued subjects to appropriate behavior.
The likeness of the hypnotically induced hearing loss to that which occurs in older
people may be questioned, as may the use of college students.
112
Nearly every design choice involves trade-offs in the use of resources which
have been used to control something else. Part of the art of design is finding a suitable
middle ground, one realistic enough to allow generalization of the results as broadly as
one wishes, but also permitting sufficient control to make valid inferences.
A good design reduces one's uncertainty that the variables are indeed linked in a
relationship and the linkage has generality. Showing that they are linked requires
internal validity (LP) where (LP) stands for 'linking power' --the power of the study to link
the treatment with the outcome. A study has strong internal validity (LP) when the
explanation advanced for the relationship is credible, when the translation of variables
into operational terms is faithful to that originally intended, where a relationship is
demonstrated in the data, where rival explanations for the relationship are eliminated,
and when the results are consistent with previous studies.
Similarly, demonstrating generality requires external validity (GP) where GP
stands for 'generalizing power'--the power of the results to be generalized beyond the
instance in which they were demonstrated. External validity (GP) assures the
applicability of the results to the persons, places and times, and that the generalizability
was not restricted by the conditions of the study. A study has strong external validity
(GP) when the generality implied by the hypotheses, or inferred with it, is consistent with
the choices made in operationalization of the study; that results were appropriately
found throughout the instances of the study as expected; that there were no conditions
of the study that limited generalization; and that the same results would have been
expected were the study operationalized and designed in alternative ways.
113
Good designs accomplish the above with the best use of all available resources,
time, and energy. They fit the design to the problem rather than changing the problem
definition to fit design requirements. They appropriately balance internal and external
validity. They accurately anticipate those alternative explanations to be eliminated that
are most plausible to one's audience. Finally, ethical standards, resource limitations,
institutional and social constraints are observed: altogether, a complex but manageable
set of criteria.
Video 4: Alcohol and Sleep
Ethyl alcohol (ethanol) is a small fat- and water-soluble molecule that is rapidly
and completely absorbed from the whole gastrointestinal tract and is evenly distributed
throughout all body fluids and tissues, including the brain. The rate of absorption is
modified by the concentration of the ethanol beverage (beer at 3 percent to 6 percent
ethanol is slower than whiskey at 40 percent to 45 percent), stomach contents (an
empty stomach facilitates absorption), and rate of consumption. Because ethanol is
distributed by the water content of tissue, a more muscular person will have lower levels
of ethanol in blood than a fat person given the same dose of ethanol based on body
weight. Ethanol is metabolized by the liver into carbon dioxide and water at a constant
rate of about 10 to 15 milligram percent per hour (1 ounce of 80 -proof whiskey, 12
ounces of beer, or 4 ounces of wine is metabolized in an hour).
As with other psychoactive substances, ethanol has profound effects on sleep
and wakefulness. It is considered a sedative, but its effects on waking and sleep are
complex and somewhat paradoxical. The acute bedtime administration of ethanol to
normal nonalcoholic volunteers shortens the latency to sleep onset and, depending on
114
dose, may initially increase the amount of deep slow-wave sleep. Additionally, ethanol
reduces the amount of REM sleep. An ethanol concentration in blood of 50 milligram
percent (100 milligram percent is legal intoxication in most states) or greater is
necessary to observe these sleep effects. Typically, the sleep effects of ethanol are
observed only during the first half of an 8-hour sleep period.
After elimination of ethanol, an apparent compensatory effect on sleep
occurs. During the latter half of the sleep period, an increased amount of REM sleep
and increased wakefulness or light sleep are found. Within three to four nights of
repeated administration of the same dose, the initial effects on sleep are lost
(technically referred to as tolerance), whereas the secondary disruption of sleep during
the latter half of the night remains. REM sleep time and sleep latency return to their
basal levels and effects on slow-wave sleep, if initially present, do not persist. When
nightly administration of ethanol is discontinued, increased amounts of REM sleep
(termed a REM rebound) are found, lasting for several nights. But the finding of a REM
rebound after repeated nightly ethanol administration in healthy, nonalcoholic normals
has not been a particularly consistent result. It has been argued that the presence of a
REM rebound is a characteristic of drugs with a high addictive potential.
When administered to awake nonalcoholic volunteers, ethanol has also been
shown to be sedating. The sedating effect has been clearly demonstrated after blood
ethanol concentration (BEC) has reached a peak of 40 milligram percent or greater. On
repeated tests of the latency of falling asleep during the day, a systematic dose-related
reduction in latency to sleep onset is found. Performance tasks sensitive to sedation
also are disrupted by ethanol. At lower ethanol concentrations and immediately after
115
ethanol consumption when ethanol is still being absorbed, sedative effects are not as
clearly evident. Subjectively, some individuals report increased arousal and euphoria,
although the electroencencephalographic (EEG) studies have found patterns suggestive
of a sedative effect. The effects of ethanol typically have been characterized as
biphasic--at low doses and during absorption, ethanol appears to be arousing, and at
high doses and during elimination, it is sedating. Some data suggest these subjective
arousing effects to ethanol may be individually specific effects associated with genetic
or personality factors.
Given the sedative effects of ethanol and its potential to disrupt performance, it is
not surprising that epidemiological data indicate ethanol is associated with increased
risk of industrial and traffic accidents. The National Highway Traffic Safety
Administration estimated that 49 percent of all traffic fatalities in 1989 were alcohol
related (i.e., a police report indicated that one or more drivers had ethanol
concentrations of 10 milligram percent or more). This is a slight decline from the
percentages of previous years. Assessment of the timing of the alcohol-related
accidents across the 24-hour day showed that alcohol-related accidents were more
prevalent during the nighttime (between midnight and 6 a.m.) than during the
daytime. The age group showing the highest rate of alcohol-related accidents while
legally intoxicated was 20 to 25 year-olds. In both cases, during the nighttime, and in
young adults, laboratory evidence shows increased levels of sleepiness and reduced
alertness. Thus, the epidemiological data regarding the temporal distribution and the
age group distribution of alcohol-related accidents suggest that ethanol and sleepiness
interact to increase the risk of alcohol-related accidents.
116
Recent laboratory studies provide clear evidence of a sleepiness-
ethanol. Reducing bed time increases sleepiness throughout the following day. The
sedative and performance disruptive effects of ethanol are enhanced when sleepiness
is increased in a such a way. Three drinks become the functional equivalent of six
drinks after 5 hours in bed for five nights. Conversely, an extension of bed time
enhances alertness (reduces sleepiness) and reduces the sedative and performance
disruptive effects of ethanol. After six nights of 10 hours in bed, a moderate dose of
ethanol (about four drinks producing a BEC of 50 milligram percent ), which disrupted
performance and increased sleep latency after a usual 8-hour bed time, no longer does
so. Additionally, the same moderate ethanol dose given over the midday, when
sleepiness is enhanced in most individuals, is performance disruptive. However, that
same dose in the early evening, when alertness is at a peak, has no measurable
effect. Finally, after the same 8 hours of bed time, sleepy individuals perform more
poorly and have greater sleepiness when given ethanol than do their alert
counterparts.
The specific mechanism by which ethanol produces sedative effects is not yet
known. Ethanol is known to affect the brain in two major ways. It has long been known
that ethanol (being fat soluble) alters the structure of the neuronal membrane and
thereby can have broad effects on the function of the neuron, altering ion flow across
the membrane and also potentially disturbing neurotransmitter receptor
functions. Ethanol also has been shown to alter the function of nearly all
neurotransmitter systems in various other ways.
117
Two transmitter systems, gamma-aminobutyric acid (GABA) and glutamate, have
received much recent attention because the ethanol effects on these systems are
observed at very low ethanol doses. Importantly, these two systems are implicated in
control of sleep and wakefulness. GABA is a major inhibitory system in the brain, and
ethanol has been shown to facilitate GABA function. Glutamate is a major excitatory
system, and ethanol has been shown to inhibit activation of this system. Thus ethanol
sedation may result from enhancement of GABA inhibition and antagonism of glutamate
excitation.
118
LIST OF REFERENCES
Ackerman, R., Leiser, D., & Shpigelman, M. (2013). Is comprehension of problem solutions resistant to misleading heuristic cues? Acta Psychologica, 143(1), 105-112.
Anderson, M. C., & Thiede, K. W. (2008). Why do delayed summaries improve metacomprehension accuracy? Acta Psychologica, 128(1), 110-118.
Assfalg, J., Bertini, M., Colombo, C., & Del Bimbo, A. (2002). Semantic annotation of sports videos. IEEE Multimedia, (2), 52-60.
Aubert, O., Prié, Y., & Canellas, C. (2014). Leveraging video annotations in video-based e-learning, 6th International Conference on Computer Supported Education (CSEDU), Barcelone: Spain.
Azevedo, R. (2009). Theoretical, conceptual, methodological, and instructional Issues in research on metacognition and self-regulated learning: A discussion. Metacognition and Learning, 4(1), 87-95.
Baddeley, A. (1992). Working memory. Science, 255(5044), 556-559.
Baggaley, J. (2013). The sudden revival of educational video. Paper presented at the Educational Media (ICEM), 2013 IEEE 63rd Annual Conference International Council For, 1-6.
Bassili, J. N., & Joordens, S. (2008). Media player tool use, satisfaction with online lectures and examination performance. International Journal of E-Learning & Distance Education, 22(2). 93-108.
Bossewitch, J., & Preston, M. (2011). Learning through digital media: Experiments in technology and pedagogy. Teaching and Learning with Video Annotations, 175-183.
Butler, D., & Winne, P. (1995). Feedback and self-regulated learning: A theoretical synthesis. Review of Educational Research, 65(3), 245-281.
Caspi, A., Gorsky, P., & Privman, M. (2005). Viewing comprehension: Students’ learning preferences and strategies when studying from video. Instructional Science, 33(1), 31-47.
Cennamo, K. S. (2012). Learning from video. In N. M. Seel, (Ed.), Encyclopedia of the sciences of learning, (pp. 1876-1879). New York: Springer.
Chandler, P., & Sweller, J. (1996). Cognitive load while learning to use a computer program. Applied Cognitive Psychology, 10(2), 151-170.
119
Cheon, J., Crooks, S., & Chung, S. (2014). Does segmenting principle counteract modality principle in instructional animation? British Journal of Educational Technology, 45(1), 56-64.
Cheon, J., Chung, S., Crooks, S. M., Song, J., & Kim, J. (2014). An investigation of the effects of different types of activities during pauses in a segmented instructional animation. Journal of Educational Technology & Society, 17(2), 296-306.
Clark, R. C., & Lyons, C. (2011). Graphics for learning: Proven guidelines for planning, designing, and evaluating visuals in training materials. San Francisco, CA: Pfeiffer.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, N.J: L. Erlbaum Associates.
Cuevas, H., Fiore, S., & Oser, R. (2002). Scaffolding cognitive and metacognitive processes in low verbal ability learners: Use of diagrams in computer-based training environments. Instructional Science, 30(6), 433-464.
Delen, E., Liew, J., & Willson, V. (2014). Effects of interactivity and instructional scaffolding on learning: Self-regulation in online video-based environments. Computers & Education, 78, 312-320.
Doolittle, P., Bryant, L., & Chittum, J. (2014). Effects of degree of segmentation and learner disposition on multimedia learning. British Journal of Educational Technology, 46(6), 1333-1343.
Dunlosky, J., Hertzog, C., Kennedy, M., & Thiede, K. (2005). The self-monitoring approach for effective learning. Cognitive Technology, 4-11.
Dunlosky, J., & Lipko, A. (2007). Metacomprehension A brief history and how to improve its accuracy. Current Directions in Psychological Science, 16(4), 228-232.
Dunlosky, J., & Thiede, K. W. (1998). What makes people study more? An evaluation of factors that affect self-paced study. Acta Psychologica, 98(1), 37-56.
Flavell, J.(1979). Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry. American Psychologist, 34(10), 906-911.
Gough, P. B., & Tunmer, W. E. (1986). Decoding, reading, and reading disability. Remedial and special education, 7(1), 6-10.
Greene, J. A., & Azevedo, R. (2007). A theoretical review of Winne and Hadwin’s model of self-regulated learning: New perspectives and directions. Review of Educational Research, 77(3), 334-372.
120
Graesser, A., Millis, K., & Zwaan, R. (1997). Discourse comprehension. Annual Review of Psychology, 48(1), 163-189.
Griffin, T., Jee, B., & Wiley, J. (2009). The effects of domain knowledge on metacomprehension accuracy. Memory & Cognition, 37(7), 1001-1013.
Griffin, T., Wiley, J., & Salas, C. (2013). Supporting effective self-regulated learning: The critical role of monitoring. In R. Azevedo & V. Aleven, (Eds.), International handbook of metacognition and learning technologies (pp.19-34). New York: Springer.
Griffin, T., Wiley, J., & Thiede, K. (2008). Individual differences, rereading, and self-explanation: Concurrent processing and cue validity as constraints on metacomprehension accuracy. Memory & Cognition, 36(1), 93-103.
Grimes, T. (1990). Audio-video correspondence and its role in attention and memory. Educational Technology Research and Development, 38(3), 15-25.
Guo, P., Kim, J., & Rubin, R. (2014). How video production affects student engagement: An empirical study of MOOC videos. Paper presented at the Proceedings of the First ACM Conference on Learning@ Scale Conference, 41-50.
Hacker, D. J. (1998). Definitions and empirical foundations. Metacognition in educational theory and practice, 14, 1-23.
Hasler, B., Kersten, B., & Sweller, J. (2007). Learner control, cognitive load and instructional animation. Applied Cognitive Psychology, 21(6), 713-729.
Henk, W., & Stahl, N. (1985). A meta-analysis of the effect of notetaking on learning from lecture. Paper presented at the 34th Annual Meeting of the National Reading Conference.
Höffler, T., & Schwartz, R. (2011). Effects of pacing and cognitive style across dynamic and non-dynamic representations. Computers & Education, 57(2), 1716-1726.
Hosack, B. (2010). VideoANT: Extending online video annotation beyond content delivery. TechTrends, 54(3), 45-49.
Jaeger, A., & Wiley, J. (2014). Do illustrations help or harm metacomprehension accuracy? Learning and Instruction, 34, 58-73.
Johnson-Laird, P. N. (1983). Mental models: Towards a cognitive science of language, inference, and consciousness (No. 6). Harvard University Press.
Kalyuga, S., Chandler, P., & Sweller, J. (1999). Managing split-attention and redundancy in multimedia instruction. Applied Cognitive Psychology, 13(4), 351-371.
121
Kaufman, P. B., & Mohan, J. (2009). Video use and higher education: Options for the future. New York: New York University.
Kintsch, W. (1998). Comprehension: A paradigm for cognition. Cambridge: Cambridge University Press.
Kintsch, W., & Van Dijk, T. (1978). Toward a model of text comprehension and production. Psychological Review, 85(5), 363-394.
Kintsch, W., Welsch, D., Schmalhofer, F., & Zimny, S. (1990). Sentence memory: A theoretical analysis. Journal of Memory and language, 29(2), 133-159.
Kobayashi, K. (2005). What limits the encoding effect of note-taking? A meta-analytic examination. Contemporary Educational Psychology, 30(2), 242-262.
Koriat, A. (1997). Monitoring one's own knowledge during study: A cue-utilization approach to judgments of learning. Journal of Experimental Psychology: General, 126(4), 349-370.
Leahy, W., & Sweller, J. (2011). Cognitive load theory, modality of presentation and the transient information effect. Applied Cognitive Psychology, 25(6), 943-951.
Low, R., & Sweller, J. (2005). The modality principle in multimedia learning. In R. E. Mayer, (Ed.), The cambridge handbook of multimedia learning (pp.148-158). Cambridge: Cambridge University Press.
McNamara, D. S., & Magliano, J. (2009). Toward a comprehensive model of comprehension. Psychology of learning and motivation, 51, 297-384.
Magliano, J., Larson, A., Higgs, K., & Loschky, L. (2015). The relative roles of visuospatial and linguistic working memory systems in generating inferences during visual narrative comprehension. Memory & Cognition, 1-13.
Magliano, J. P., Loschky, L. C., Clinton, J. A., & Larson, A. M. (2013). Is reading the same as viewing? In B.Miller, L. Cutting, & P. McCardle, P., (Eds.), Unraveling the behavioral, neurobiological and genetic components of reading comprehension (pp. 78-90). Baltimore: Paul Brookes Publishing.
Magliano, J., Miller, J., & Zwaan, R. (2001). Indexing space and time in film understanding. Applied Cognitive Psychology, 15(5), 533-545.
Maki, R. (1998). Test predictions over text material. Metacognition in Educational Theory and Practice, 14, 117-144.
Maki, R., & Berry, S. (1984). Metacomprehension of text material. Journal of Experimental Psychology: Learning, Memory, and Cognition, 10(4), 663-679.
122
Maki, R., Jonas, D., & Kallod, M. (1994). The relationship between comprehension and metacomprehension ability. Psychonomic Bulletin & Review, 1(1), 126-129.
Maki, R., & Serra, M. (1992). Role of practice tests in the accuracy of test predictions on text material. Journal of Educational Psychology, 84(2), 200-210.
Mayer, R. E. (2005). Principles for managing essential processing in multimedia learning: Segmenting, pretraining, and modality principles. In R. E. Mayer, (Ed.), The Cambridge handbook of multimedia learning (169-182). Cambridge: Cambridge University Press.
Mayer, R. E. (2009). Multimedia learning. Cambridge: Cambridge University Press.
Mayer, R. E. (2014). The Cambridge handbook of multimedia learning. Cambridge: Cambridge University Press.
Mayer, R. E., & Chandler, P. (2001). When learning is just a click away: Does simple user interaction foster deeper understanding of multimedia messages? Journal of Educational Psychology, 93(2), 390-397.
Mayer, R. E., Dow, G., & Mayer, S. (2003). Multimedia learning in an interactive self-explaining environment: What works in the design of agent-based microworlds? Journal of Educational Psychology, 95(4), 806-812.
Mayer, R. E., & Gallini, J. K. (1990). When is an illustration worth ten thousand words? Journal of Educational Psychology, 82(4), 715-726.
Mayer, R. E., & Moreno, R. (1998). A split-attention effect in multimedia learning: Evidence for dual processing systems in working memory. Journal of Educational Psychology, 90(2), 312-320
Mayer, R. E., & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist, 38(1), 43-52.
Mayer, R. E., & Moreno, R. (1998). A split-attention effect in multimedia learning: Evidence for dual processing systems in working memory. Journal of Educational Psychology, 90(2), 312-320.
Merkt, M., & Schwan, S. (2014). How does interactivity in videos affect task performance? Computers in Human Behavior, 31, 172-181.
Metcalfe, J. (2009). Metacognitive judgments and control of study. Current Directions in Psychological Science, 18(3), 159-163.
Middendorf, J., & Kalish, A. (1996). The “change-up” in lectures. National Teaching Learning Forum, 5(2), 1-5.
123
Moreno, R. (2007). Optimizing learning from animations by minimizing cognitive load: Cognitive and affective consequences of signaling and segmentation methods. Applied Cognitive Psychology, 21(6), 765-781.
Mousavi, S., Low, R., & Sweller, J. (1995). Reducing cognitive load by mixing auditory and visual presentation modes. Journal of Educational Psychology, 87(2), 319-334.
Nelson, T., Dunlosky, J., Graf, A., & Narens, L. (1994). Utilization of metacognitive judgments in the allocation of study during multitrial learning. Psychological Science, 5(4), 207-213.
Nelson, T. O., & Narens, L. (1990). Metamemory: A theoretical framework and new findings. Psychology of Learning and Motivation, 26, 125-173.
Ng, H., Kalyuga, S., & Sweller, J. (2013). Reducing transience during animation: A cognitive load perspective. Educational Psychology, 33(7), 755-772.
Paas, F. G., Van Merriënboer, J. J., & Adam, J. J. (1994). Measurement of cognitive load in instructional research. Perceptual and Motor Skills, 79(1), 419-430.
Pardo, A., Mirriahi, N., Dawson, S., Zhao, Y., Zhao, A., & Gaševic, D. (2015). Identifying learning strategies associated with active use of video annotation software. Paper presented at the Proceedings of the Fifth International Conference on Learning Analytics and Knowledge, 255-259.
Petty, L. C., & Rosen, E. F. (1987). Computer-based interactive video systems. Behavior Research Methods, Instruments, & Computers, 19(2), 160-166.
Pilegard, C., & Mayer, R. E. (2015). Within-subject and between-subject conceptions of metacomprehension accuracy. Learning and Individual Differences, 41, 54-61.
Rawson, K., & Dunlosky, J. (2002). Are performance predictions for text based on ease of processing? Journal of Experimental Psychology: Learning, Memory, and Cognition, 28(1), 69-80
Rawson, K., Dunlosky, J., & McDonald, S. (2002). Influences of metamemory on performance predictions for text. The Quarterly Journal of Experimental Psychology: Section A, 55(2), 505-524.
Rawson, K., Dunlosky, J., & Thiede, K. (2000). The rereading effect: Metacomprehension accuracy improves across reading trials. Memory & Cognition, 28(6), 1004-1010.
Sadallah, M., Aubert, O., & Prié, Y. (2014). CHM: An annotation-and component-based hypervideo model for the web. Multimedia Tools and Applications, 70(2), 869-903.
124
Salomon, G. (1984). Television is" easy" and print is" tough": The differential investment of mental effort in learning as a function of perceptions and attributions. Journal of Educational Psychology, 76(4), 647-658.
Scheiter, K., & Gerjets, P. (2007). Learner control in hypermedia environments. Educational Psychology Review, 19(3), 285-307.
Schraw, G. (2009). Measuring metacognitive judgments. In D. J. Hacker, J. Dunlosky, & A. C. Graesser, (Eds.), Handbook of metacognition in education (pp. 415-429). New York: Routledge.
Serra, M., & Dunlosky, J. (2010). Metacomprehension judgments reflect the belief that diagrams improve learning from text. Memory, 18(7), 698-711.
Smith, E. (1987). Interactive video: An examination of use and effectiveness. Journal of Instructional Development, 10(2), 2-10.
Son, L. K., & Schwartz, B. L. (2002). The relation between metacognitive monitoring and control. Applied metacognition, 15-38.
Stiller, K, Freitag, A., Zinnbauer, P., & Freitag, C. (2009). How pacing of multimedia instructions can influence modality effects: A case of superiority of visual texts. Australasian Journal of Educational Technology, 25(2), 184-203.
Spanjers, I. A., Van Gog, T., & van Merriënboer, J. J. (2010). A theoretical analysis of how segmentation of dynamic visualizations optimizes students' learning. Educational Psychology Review, 22(4), 411-423.
Spanjers, I., Gog, T., & Merriënboer, J. (2012). Segmentation of worked examples: Effects on cognitive load and learning. Applied Cognitive Psychology, 26(3), 352-358.
Stiller, K. (2011). Does Segmentation of Complex Instructional Videos in Big Steps Foster Learning? The Segmentation Principle Applied to a Classroom Video. In Proceedings of the Global Learn Asia Pacific 2011- Global Conference on Learning and Technology, 2044-2053.
Szpunar, K. K., Jing, H. G., & Schacter, D. L. (2014). Overcoming overconfidence in learning from video-recorded lectures: Implications of interpolated testing for online education. Journal of Applied Research in Memory and Cognition, 3(3), 161-164.
Szpunar, K. K., Khan, N. Y., & Schacter, D. L. (2013). Interpolated memory tests reduce mind wandering and improve learning of online lectures. Proceedings of the National Academy of Sciences, 110(16), 6313-6317.
Tabbers, H., & de Koeijer, B. (2010). Learner control in animated multimedia instructions. Instructional Science, 38(5), 441-453.
125
Thiede, K. W. (1999). The importance of monitoring and self-regulation during multitrial learning. Psychonomic Bulletin & Review, 6(4), 662-667.
Thiede, K. W., & Anderson, M. (2003). Summarizing can improve metacomprehension accuracy. Contemporary Educational Psychology, 28(2), 129-160.
Thiede, K. W., Anderson, M., & Therriault, D. (2003). Accuracy of metacognitive monitoring affects learning of texts. Journal of Educational Psychology, 95(1), 66-73.
Thiede, K. W., Dunlosky, J., Griffin, T., & Wiley, J. (2005). Understanding the delayed-keyword effect on metacomprehension accuracy. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31(6), 1267-1280.
Thiede, K. W., Griffin, T., Wiley, J., & Anderson, M. (2010). Poor metacomprehension accuracy as a result of inappropriate cue use. Discourse Processes, 47(4), 331-362.
Thiede, K. W., Griffin, T. D., Wiley, J., & Redford, J. S. (2009). Metacognitive monitoring during and after reading. In D. J. Hacker, J. Dunlosky, & A. C. Graesser, (Eds.), Handbook of metacognition in Education (pp.85-106). New York: Routledge.
Thomas, A. O., Antonenko, P. D., & Davis, R. (2016). Understanding metacomprehension accuracy within video annotation systems. Computers in Human Behavior, 58, 269-277.
Weaver, C., & Bryant, D. (1995). Monitoring of comprehension: The role of text difficulty in metamemory for narrative and expository text. Memory & Cognition, 23(1), 12-22.
Wiley, J., Griffin, T., & Thiede, K. (2005). Putting the comprehension in metacomprehension. The Journal of General Psychology, 132(4), 408-428.
Williams, J. R. (1998). Guidelines for the use of multimedia in instruction. Paper presented at the Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 42(20) 1447-1451.
Winne, P. H., & Hadwin, A. F. (1998). Studying as self-regulated learning. Metacognition in Educational Theory and Practice, 93, 27-30.
Wittrock, M. (1989). Generative processes of comprehension. Educational Psychologist, 24(4), 345-376.
Wong, A., Leahy, W., Marcus, N., & Sweller, J. (2012). Cognitive load theory, the transient information effect and e-learning. Learning and Instruction, 22(6), 449-457.
126
Zimmerman, B. J. (1998). Academic studying and the development of personal skill: A self-regulatory perspective. Educational Psychologist, 33(2-3), 73-86.
Zimmerman, B. J., & Schunk, D. H. (2011). Handbook of self-regulation of learning and performance. New York: Routledge.
Zwaan, R, Magliano, J, & Graesser, A. (1995). Dimensions of situation model construction in narrative comprehension. Journal of Experimental Psychology: Learning, Memory, and Cognition, 21(2), 386-397.
127
BIOGRAPHICAL SKETCH
Aaron Thomas earned a Bachelor of Arts degree from St. John’s College in 1995
with a double major in philosophy and the history of mathematics and science, and a
double minor in classics and comparative literature. He studied classics at both the
University of Pennsylvania and the University of Florida from which he earned a Master
of Arts degree in classical studies in 1998 with an emphasis upon both classical Latin
and Greek philology. He has received both Fulbright and NEH fellowships. He is an
experienced secondary and post-secondary teacher of educational technology, Latin,
classical Greek, humanities, mythology, Greco-Roman history, mathematics, and
philosophy. In addition, he was an instructor and subject matter expert at Florida Virtual
School, one of the preeminent K12 virtual schools in the United States.
Dr. Thomas has also worked at the University of Florida as senior instructional
designer and Associate Director of E-learning at the College of Education. Dr. Thomas
is currently the Educational Technology Principal and Learning Architect for the
Counseling and Wellness Center at the University of Florida.