cognitive measurements to design effective learning environments
DESCRIPTION
Cognitive Measurements to Design Effective Learning Environments. Fred Paas 1 & Slava Kalyuga 2. 1 Open University of the Netherlands 2 New York University, USA. 3. I C L E P S WORKSHOP 2005 August 30 . Overview. COGNITIVE MEASUREMENT: GROUP-BASED INSTRUCTION - PowerPoint PPT PresentationTRANSCRIPT
Cognitive Measurements to Design Effective Learning Environments
1 Open University of the Netherlands2 New York University, USA
Fred Paas1 & Slava Kalyuga2
I C L E P S WORKSHOP 2005 August 30 3
Overview
COGNITIVE MEASUREMENT: GROUP-BASED INSTRUCTIONMeasurement of cognitive load Interpretation of performance, mental effort, and combined scores
COGNITIVE MEASUREMENT: PERSONALIZED INSTRUCTIONDiagnostic assessment of organized knowledge structures Applying combined efficiency measures in adaptive training
Basic Assumption
If individuals are to learn effectively in a learning environment, the architecture of their cognitive system, the learning environment, and interactions between both must be
understood, accommodated, and aligned
Cognitive Measurements
Working Memory
Sensory Memory
Long Term Memory
Cognitive architecture
• Perceive incoming information
• Attend to information • Limited capacity• Limited duration• Separate processors for visual and auditory information
• Permanently store all knowledge and skills in a hierarchical network (schemas)
• Unlimited capacity
Schema: • categorizes elements of information according to the manner in which they will be used
• consists of a multi-dimensional web of interconnected nodes of information• can be treated by WM as a single entity, and if the learning process has occurred over a long period of time, it may incorporate a huge amount of information
• can be processed consciously or automatically
Intrinsic
represents the load that performing a particular task imposes on our cognitive system
The Concept Cognitive Load
Extraneous not relevant for learning
Germane relevant for learning
Determined by the number of information elementsand their interactivity
Determined by the manner in which the informationis presented to learners
Intrinsic
Extraneous
Germane
Capacitymax.
Capacitymin.
Cognitive Load TheoryInstructional Techniques
DecreasingExtraneous
Load Goal-free effect Worked-example effect Completion effect Split-attention effect Modality effect Redundancy effect
MinimizingExtraneous
LoadIncreasing
Germane Load Variability effect Self-explanation effect Imagination effect Interactivity effect
MaximizingGermane Load
Sequencing effect Fading support effect
ManagingIntrinsic Load
Measurement of Cognitive Load
Objective measures Task and performance Secondary task Psychophysiological
Subjective measures Rating scales
Goal-Specific Problem Solving
Start
Equationswith goalas only
unknown
Equation withsubgoal(s) and
unknown(s)
Equationwith subgoal
as onlyunknown
Equationwith goal andunknown(s)
Problem solved
Problem statement and
equations in
working memory
Subgoal(s)added to
working memory
yes
yes
yes
yes
no
no
no
no
no
noSolve equations
and add newknown to
working memory
Given: A car that starts from rest and accelerates uniformly at 2 meters/s2 in a straight line has an average velocity of 17 meters/s.
Goal: How far has it traveled?Operators: s = v * t , v = .5V and V = a * t
(V=final velocity, v=average velocity, a=accelaration, t=time, s=distance)
Goal-Free Problem Solving
Start
Stop searchEquation
with only oneunknown
Problem statement and
equations in
working memory
yes
no
Solve equationsand add new
known toworking memory
Given: A car that starts from rest and accelerates uniformly at 2 meters/s2 in a straight line has an average velocity of 17 meters/s.
Calculate the value of as many variables as you can.
Measurement of Cognitive Load
Objective measures Task and performance Secondary task Psychophysiological
Subjective measures Rating scales
Rapid RT Slow RT
Cognitive resources to simple primary task
Cognitive resources to complex primary task
Fixed cognitive capacity Fixed cognitive capacity
Resources to secondary task
Resources to secondary task
Objective measures Task and performance Secondary task Psychophysiological
Subjective measures Rating scales
Measurement of Cognitive Load
Heart rate variability
Eye movementsPupil dilation
very, very low mental
effort
very, very high mental
effort
neither low nor high
mental effort
In solving or studying the preceding problem I invested:
Measurement of Cognitive Load Objective measures
Task and performance Secondary task Psychophysiological
Subjective measures Rating scales
Subjective measures: Rating scales (NASA-TLX)
How to interprete performance and mental effort scores?
Instructional Condition Performance (1-10) Mental Effort (1-10)
A
B
C
D
2
2
8
8
9
2
3
9
Mental Effort
PerformanceM = P
Efficiency = 0high efficiency
low efficiency
Efficiency of Instructional Conditions
Motivation in Instructional Conditions
Mental Effort
Performance
Motivation = 0
high motivation
low motivation
Instructional Efficiency/Motivation
Mental Effort
Performancehigh motivation
low motivation
high efficiency
low efficiency
Find usable objective technique
Intrinsic items1- how easy or difficult do you consider probability theory at this moment?
Extraneous items2- how easy or difficult is it for you to work with learning environment? 3- how easy or difficult is it for you to distinguish important and unimportant information in the learning environment?4- how easy or difficult is it for you to collect all the information that you need in the learning environment?
Germane items5- how easy or difficult was it to understand the solution in the last animation?
Use measures to personalize instruction
Distinguish between different types of load
Measurement of Cognitive Load: Challenges
COGNITIVE MEASUREMENT: PERSONALIZED INSTRUCTION
Expertise reversal effect:
Cognitive load effects depend on levels of learner expertise:instructional designs or procedures that are effective for novices may be ineffective for more proficient (expert) learners.
Instructional implications:
- instructional techniques need to change with alterations in expertise;
- it is critical to have simple rapid measures of learner proficiency (performance and mental effort).
Real time (rapid online) diagnostic assessment of organized knowledge structures.
Why organized knowledge structures (schemas)?
Cognitive studies of expertise:
organized knowledge base in LTM is central to cognitive processing (De Groot, 1946/1965, Chase & Simon, 1973); they affects the way we process information in WM and solve problems (Novices vs Experts).
Solve for x: 5x = - 4
x = - 4/5
Solve for x: 5x = - 4
5x/5 = - 4/5
Solve for x: 5x = - 4
Diagnostic cognitive assessment should be
• sensitive to different cognitive attributes
• sensitive to different levels of proficiency
• practically usable
Typical time scale of cognitive processes: up to several seconds.
Rapid diagnostic approach:
general: What is the highest level of organised knowledge structures (if any) a person is capable of retrieving and applying to the briefly presented material?
first-step method: Presenting learners with a task for a limited time and asking them to indicate their first step towards solution.
Rapid verification diagnostic technique: Presenting learners with a series of possible task
solutions for a limited time and asking them to rapidly verify the suggested solution steps.
Physics (kinematics) vector addition motion problems
A ship is traveling at 7 m/s. A dog runs across the deck at the same speed in a direction of 60° relative to the direction of motion of the ship. What is the velocity of the dog relative to the sea?
R(mental effort rating)
P(performance score)
Pmax
Rmax
Ecr =Pmax
Rmax
E > E cr
E < E cr
P2
P1
R2R1
P1
R1E1=
P2
R2E2=
A
B
E = P/R
Combined efficiency measures
Using rapid diagnostic techniques in adaptive training for dynamic learning task selection (tailoring levels of task complexity and learner support).
Initial Diagnostic Test
STAGE 1
E 29
Diagnostic Test -4x=3
Yes
No
Yes
No
STAGE 2Diagnostic Test 3x+7=3
Yes
No
Yes
No
STAGE 3 Diagnostic Test (3x+2)/2=3
Yes
No
Yes
No
STAGE 4FinalDiagnostic Test
2 fully worked out examples each followed by a problem solving exercise
4 shortened worked examples
2 completion tasks (with 1 step to complete) each followed by a problem solving exercise
END
-3x=7
4x+3=2
(2x+1)/3=2
-7x=2
6x+4=3
(5x+4)/3=2
2 completion tasks (with 2 steps to complete) each followed by a problem solving exercise
4 problem solving exercises
2 worked examples
4 shortened worked examples
4 shortened worked examples
2 worked examples
2 worked examples
29
E 49
49
E 69
E 69
E 29
? E 19
?
E 29
?
E 29
?
E 19
?
E 19
?
Selection algorithm governing the selection of learning tasks with different levels of difficulty (stages 1-4) and support (worked examples, completion tasks, and conventional tasks/problem solving exercises). Adapted from Kalyuga and Sweller (2004)