simstudent: building a cognitive tutor by teaching a simulated student noboru matsuda human-computer...

SimStudent:Building a Cognitive Tutor by Teaching a Simulated Student

Noboru Matsuda

Human-Computer Interaction Institute

Carnegie Mellon University

Sim

Stu

den

t Pro

ject

PSLC Summer School 2009 :: SimStudent Demo :: Noboru Matsuda (CMU)2

CTAT: Cognitive Tutor Authoring Tools

• Example-Tracing Tutor with zero programming

– A cognitive model specific to a particular problem

– Some generalization by modifying a behavior graph

• Model-Tracing Tutor requires a cognitive model

– Cognitive task analysis is challenging– Writing production rules is even more challenging

– Performing the task is much easier…

Sim

Stu

den

t Pro

ject


Next Generation AuthoringBuild a tutor GUI

Teaching a solution

SimSt. learning

Rule simplify-LHS:

IF is-equation( Eq ),

is-lhs( Eq, Lhs ),

polynomial( Lhs ),

all-var-terms( Lhs )

Then simplify( Lhs, S-lhs ),

enter( S-lhs )

Production Rules

Rule simplify-LHS:


is-lhs( Eq, Lhs ),

polynomial( Lhs ),



enter( S-lhs )

Rule simplify-LHS:


is-lhs( Eq, Lhs ),

polynomial( Lhs ),



enter( S-lhs )

Sim

Stu

den

t Pro

ject


SimStudent

• Machine learning agent

– Learns problem-solving steps by …

– Observes model solutions / solving problems, and …

– Outputs a set of production rules

• Fundamental technology

– Programming by Demonstration

– Inductive Logic Programming

Lau, T. A., & Weld, D. S. (1998). Programming by demonstration

Blessing, S. B. (1997). A programming by demonstration authoring tool for model-tracing tutors

Sim

Stu

den

t Pro

ject

PSLC Summer School 2009 :: SimStudent Demo :: Noboru Matsuda (CMU)

Authoring Strategies

• Authoring by demonstration– Learning from worked-out examples

– Demonstrate whole solutions

– Learning by generalizing examples (when it can’t “self-explain”)

• Authoring by tutoring– Learning by doing (with tutor feedback)

– Interactively tutor with immediate feedback and hint

– Learning by generalizing hint with taking feedback into account

5

Sim

Stu

den

t Pro

ject


Demo!

6

Sim

Stu

den

t Pro

ject


Learning Production Rules in 3 parts: What-When-How

If

among this and that GUI elements

such and such constraints hold

Then

do actions with the GUI elements

When

What

How

Sim

Stu

den

t Pro

ject


Production Rule in JESS

(defrule trans-lr-lhs

?problem <- (problem (interface-elements ?table1 ? ? ?))?table1 <- (table (columns ?column1))?column1 <- (column (cells $?m1 ?cell0 $?))?cell0 <- (cell (value ?val0&~nil))

?problem <- (problem (interface-elements ? ?table2 ? ?))?table2 <- (table (columns ?column2))?column2 <- (column (cells $?m2 ?cell1 $?))?cell1 <- (cell (value ?val1&~nil))

?column1 <- (column (cells $?m3 ?cell2 $?))?cell2 <- (cell (name ?selection) (value ?input))

(test (consecutive-row ?cell0 ?cell2))(test (same-column ?cell0 ?cell2))(test (distinctive ?cell0 ?cell2))(test (consecutive-row ?cell1 ?cell2))(test (same-column ?cell1 ?cell2))(test (distinctive ?cell1 ?cell2))

(test (polynomial ?val0))(test (not (has-var-term ?val1)))

=>

(bind ?input (first-var-term ?val0))(modify ?cell2 (value ?input)) )

LHS

RHS

WME path

Topologicalconstraints

Featureconstraints

WMEconditions

Working Memory Element (WME)(defrule trans-lr-lhs

?problem <- (problem (interface-elements ?table1 ? ? ?))?table1 <- (table (columns ?column1))?column1 <- (column (cells $?m1 ?cell0 $?))?cell0 <- (cell (value ?val0&~nil))

?problem <- (problem (interface-elements ? ?table2 ? ?))?table2 <- (table (columns ?column2))?column2 <- (column (cells $?m2 ?cell1 $?))?cell1 <- (cell (value ?val1&~nil))

?column1 <- (column (cells $?m3 ?cell2 $?))?cell2 <- (cell (name ?selection) (value ?input))

(test (consecutive-row ?cell0 ?cell2))(test (same-column ?cell0 ?cell2))(test (distinctive ?cell0 ?cell2))(test (consecutive-row ?cell1 ?cell2))(test (same-column ?cell1 ?cell2))(test (distinctive ?cell1 ?cell2))

(test (polynomial ?val0))(test (not (has-var-term ?val1)))

=>

(bind ?input (first-var-term ?val0))(modify ?cell2 (value ?input)) )

LHS

RHS

WME path

Topologicalconstraints

Featureconstraints

WMEconditions

Working Memory Element (WME) GUI elements

Constraints

Actions

Sim

Stu

den

t Pro

ject


Background Knowledge

• Domain concepts to “explain” demonstrations– Operators

– Feature predicates

• External Jess function written in Java

(defrule multi-lhs…?var22140 <- (column (cells ? ? ?var22143 ? ? ? ? ?))?var22143 <- (cell (value ?val0&~nil))

(test (fraction-term ?val0 )) =>

(bind ?val2 (denominator ?val0))(bind ?input (mul-term-by ?val0 ?val2))…

)

Sim

Stu

den

t Pro

ject


Example ： Algebra domain•16 Feature predicates & 28 operators

Feature Predicates for LHS conditions Operators for RHS actions

HasCoefficient

HasConstTerm

HasVarTerm

Homogeneous

IsFractionTerm

IsConstant

IsDenominatorOf

IsNumeratorOf

IsPolynomial

Monomial

NotNull

VarTerm

IsSkillAdd

IsSkillSubtract

IsSkillDivide

IsSkillMultiply

AddTerm

AddTermBy

Coefficient

CopyTerm

Denominator

DivTerm

DivTermBy

EvalArithmetic

FirstTerm

FirstVarTerm

GetOperand

InverseTerm

LastConstTerm

LastTerm

LastVarTerm

MulTerm

MulTermBy

Numerator

ReverseSign

RipCoefficient

SkillAdd

SkillClt

SkillDivide

SkillMultiply

SkillRf

SkillMt

SkillSubtract

VarName

Sim

Stu

den

t Pro

ject


Learning Results

€

StepScore =0, if C = 0

C

C + I

⎧ ⎨ ⎪

⎩ ⎪

# of training tasks

% C

orr

ect

rule

firi

ng

s (

10

test

tasks) Authoring by

tutoring

Authoring by demonstration

Betterthan

Sim

Stu

den

t Pro

ject


Authoring TimeAuthoring by Tutoring

Authoring by Demonstration

• Authoring by tutoring took 86 minutes

• Authoring by demonstration took 238 minutes

• A 2.8x speed-up!

Sim

Stu

den

t Pro

ject


Example: Stoichiometry Tutor

Approximate Time Distribution (hours)

55

30

15

40

Operator generation

Feature predicate generation

Debugging

Testing

Sim

Stu

den

t Pro

ject


Learn more about SimStudents

• Project Web– www.cs.cmu.edu/~mazda/SimStudent

• Download & Tutorial– http://ctat.pact.cs.cmu.edu (linked from project web)

• Contact Noboru– [email protected]

14

Sim

Stu

den

t Pro

ject


SimStudent Projects

• Intelligent Authoring– Building a Cognitive Tutor as a CTAT Plug-in

• Student Modeling and Simulation– Controlled educational studies

– Error formation study

– Prerequisite conceptual knowledge study

• Teachable Peer Learner– Learning by teaching

15

Sim

Stu

den

t Pro

ject


Learning by Teaching SimStudent

Tutor LuckyNext Problem Quiz Lucky

Prepare Lucky for Quiz Level 3 !

Curriculum Browser

Level 1:[+] One-Step Linear Equation

Level 2:[+] Two-Step Linear Equation

Level 3:[-] Equation with Similar Terms

OverviewIn this unit, you will solve equations with integer or decimal coefficients, as well as equations involving more than one variable.

More…

Lucky

Tutor LuckyNext Problem Quiz Lucky

Prepare Lucky for Quiz Level 3 !

Curriculum Browser

Level 1:[+] One-Step Linear Equation

Level 2:[+] Two-Step Linear Equation

Level 3:[-] Equation with Similar Terms

OverviewIn this unit, you will solve equations with integer or decimal coefficients, as well as equations involving more than one variable.

More…

Lucky

simstudent: building a cognitive tutor by teaching a simulated student noboru matsuda human-computer...

Documents

lhs eq

polynomial lhs

slhs rule

simstudent demo

javadefrule multilhs

slhs production rulesrule

noboru matsuda cmudemo

tutor guiteaching