Dynamically altering the learning trajectories of novices with pedagogical

agents

Carole R. Beal, USCRonald H. Stevens, UCLA

Cognition & Student Learning R305H050052

Project focus: Scientific problem solving

• Learning to do science– hypothesis generation– effective use of resources &

content knowledge– strategy selection &

progress checking• Skills apply across science

domains– “Investigation &

experimentation” in CA standards

• Stevens et al. developed IMMEX to provide problem solving practice opportunities

IMMEX: Online multimedia problem solving

environment providing simulations of case-type

scientific problems

www.immex.ucla.edu

Ron Stevens

IMMEX example: Duck Run

• Domain: Chemistry• Goal: Identify element

dumped in pond• Available resources:

– Experts– Characteristics – Periodic table– Glossary

• Multiple cases

IMMEX: Strategy assessment

• Solution strategies:– searching resources– balancing cost of search

& info. value, time• Students’ actions

classified by ANN– “nodes” represent:

• what problem resources are viewed

• relation with solve rates

8% of casesstudents look at all

resources solve rate only 47%

Stevens et al. (1999). Artificial neural network-based performanceassessments. Computers in Human Behavior, 15, 295-314.

Duck Run

IMMEX: Prior work on problem solving strategies

• Stabilization– approaches tend to persist

over time– case repetition does not

always lead to improved performance

• Student strategies related to teacher implementation– better performance with

teachers who “frame” activity well

• PedAgents goal: Enhance IMMEX to encourage good strategies

“red” teacher’s students using resources well & solving problems

Stevens, R. H., & Thadani, V. (2007). A value-based approach for quantifying scientific problem solving effectiveness. Journal of Technology, Instruction, Cognition & Learning, 5, 325-337.

PedAgents: Strategy intervention

• Integrate strategy guidance into case prologues

– e.g., “Sometimes students just dive in to a problem and try out lots of different things. But it’s better to make sure you know what the problem is asking first. If you don’t, go back and read the prologue again, or ask your teacher for

help.”

• Prologues also address motivational issues– e.g., “Sometimes, students give up too

fast on IMMEX problems. They think the answer should come in just a couple of minutes. But hard problems take a lot of mental effort. Stick with it and you’ll see results!”

• Prologues are not content-specific– delivered randomly across

cases

strategysuggestion delivered

at prologue

PedAgents: Strategy intervention

• Epilogues encourage self-reflection & strategy review:

– e.g., “Look at your notes for this case. Are they organized? Can you see the path you took to get to your answer? If not, can you think of one or two ways of making them more organized? (If you didn’t take notes, take them – it will help!)”

• Epilogues also target motivational issues in problem solving:

– e.g., “Sometimes students get discouraged by this case. If you solved it, that’s great! If you didn’t, don’t be too hard on yourself. You can learn even when you don’t get the right answer. Make failure work for you by figuring out what to do differently next time.” strategy hint also

delivered at caseepilogue

PedAgents: Intervention results

• Improved first-try solve rates: from 64% to 75%

• Reduction in never-solved cases

• Higher IRT scores across five cases

• Decrease in use of “exhaustive” strategies– students who still look at

all resources more likely to solve

– hints may be promoting better integration of information

decrease to 4.5%of cases; solve rate increases

to 61%

Duck Run

PedAgents: Interventions

• Message content is key– Prior study showed generic academic

advice messages were not helpful: • e.g., “Successful students use a planner to

keep track of notes and assignments” • “It’s a good idea to keep up with your

class reading. Don’t let yourself get too far behind!”

• Current study: Role of message delivery– Text messages compared with animated

agents– Human or non-human agent characters

• e.g., teacher or duck

Next steps

• Goal: integrate strategy guidance into complete IMMEX middle school curriculum– earth, life & physical

sciences– match message content

with cases• e.g., link motivation-

oriented messages with more difficult cases

– support for effective implementation by teachers

“Phyto Phiasco”

“Get organized”

“dashboard” forteachers to trackstudent progress