a.m. davisapril 29 th, 2002 intelligent computer-aided instruction: a survey organized around system...

A.M. Davis April 29th, 2002

Intelligent Computer-Aided Instruction: A Survey Organized

Around System Components

Author: Jeff W. Rickel, 1989

Speaker: Amy Davis

CSCE 976 (Advanced AI)

April 29th, 2002


Outline of Presentation

• Why ICAI?

• Overview of main systems and technologies discussed in this paper

• Contributions of seminal systems to various components of ICAI systems


ICAI – better than CAI

• First came CAI– Fully specifies presentations

– All questions and their answers

– Strict flow of control

– “electronic page-turning”

• Need for Intelligence recognized– Rich domain knowledge (and representation)

– Ability to use knowledge in unspecified ways

– Individualize instruction for student


ICAI – Representative of AI

• No commercial ICAI systems exist• ICAI – an active research topic in AI• ICAI employs many AI techniques

– Require reasoning from rich knowledge representation

– Models user

– Needs communication and information structures

– Needs “common sense” reasoning


ICAI systems (I)

• WEST (R. R. Burton and J.S. Brown, 1982)– “Conquer the west” with mathematical

equations that evaluate to the number of spaces you want to move.

• SCHOLAR (Jaime Carbonell, 1970)– Learn geography by holding natural-language

dialog with the computer.


ICAI systems (II)

• WHY (Stevens and Collins, 1977)– Understand rainfall, when and why it happens

by holding a discussion with the computer.

• SOPHIE (Sleeman and Brown, 1982)– Learn by example how to troubleshoot

electronic circuits.


ICAI systems (III)

• STEAMER (Hollan, Hutchins and Weitzman, 1984)– Manipulate controls to a steam propulsion

system to gain an understanding of how each control effects the system.

• RBT: Recovery Boiler Tutor (Woolf, 1986)– Solve problems in real time on a simulated

boiler.


ICAI systems (IV)

• WUMPUS (Goldstein, 1978)– “Hunt the Wumpus” using mathematical and

logical skills

• MYCIN, GUIDON– Find the likely bacterial cause for the symptoms

provided.


ICAI Goals

• More effective computer-based tutors

• More economical computer-based tutors

• Reflect current state of AI research


Components of ICAI systems

1. Learning Scenarios

2. Forms of Knowledge Representation

3. Student modeling

4. Student diagnosis

5. Pedagogical knowledge

6. Discourse management

7. Automatic problem generation

8. User Interfaces


ICAI Learning Scenarios

• Goal: Involve more senses– Retain information longer– Make student an active participant

• Methods–Coaching–Socratic–Mixed-Initiative

Dialogue

–Articulate Expert–Simulation–Discovery Learning


Learning Scenarios: Coaching

• Only give advice when needed– Coach “looks over student’s shoulder”

– Offer timely but unobtrusive advice

– Expose key knowledge when student’s performance plateaus

– Like MS help

• Common in Gaming environment (ex. WEST)– Determine if student is using correct skills

– Determine when student needs guidance


Learning Scenarios: Mixed Initiative Dialog

• Hold conversation with student• Student responds to computer questions

OR• Student initiates a line of questioning and

computer answers• SCHOLAR

– More reactive to student– Allows student initiative


Learning Scenarios: Socratic

“Education can not be attained through passive exercises such as reading or listening, but instead from actual problem solving”

• Ask thought-probing questions – Require use of new knowledge

– Point out gaps in knowledge

– Expose misconceptions

• WHY tutor


Learning Scenarios: Articulate Expert

• SOPHIE• Teach by example

– Solve problems with student watching

– Explain reasons for decisions

– Demonstrate troubleshooting tactics

• Then make student solve problems– Occasionally provide guidance

– Force student to give rationale for choices

• Students should know “Why am I doing this action?”


Learning Scenarios: Interactive, Inspectable Simulation

• Provide a simulation of a domain

• Allow exploration of actions– See the effects of actions– No fear for real-world consequences– Potential to carry into real-life situations

• STEAMER, RBT


Learning Scenarios: Discovery-Based Learning

• Opposite of CAI• Student explores

– Micro-world emulation– Discover rules and knowledge– Full student control – driven by curiosity

• Prepares student for scientific inquiry, real life research, creative thinking

• Outside scope of this paper


Learning Scenarios:Summary

• Determines “Look and Feel” of tutoring system.

• Based on student-tutor balance of control

• Requires support from the Knowledge base of the system


ICAI Domain Knowledge Representation

• CAI: poor knowledge of their domain– Canned presentation– Canned questions– Canned answers

• ICAI: More knowledge fewer limitations– Support understanding– Allow flexibility in teaching

• Knowledge is key to intelligent behavior• Way knowledge is stored dictates its use


Domain Knowledge• No general form suitable for all knowledge• Challenge:

– Determine types of knowledge required– Find suitable representations– Support teaching particular subjects

• Forms examined– Rule Based– Script– Semantic Network– Simulation– Condition Action Rules


Domain Knowledge: Rule-Based KR

• Generally a failure– Miss low-level detail– Miss relations necessary for learning and tutoring– No analogies, multiple views– No levels of explanation– Need to know how rules fit together

• MYCIN, GUIDON• Need knowledge + perspective to communicate

knowledge to student


Domain Knowledge: Scripts

• WHY• Nodes processes, events; • Edges relations between nodes

– X enables Y– X causes Y

• Script partially-ordered sequence of processes and events linked by temporal or causal connections.

• Hierarchy of scripts: lower levels describe causal relationships within higher levels.


Domain Knowledge: Semantic Networks

• Highly structured data base• Stores concepts and facts• Stores connections along many dimensions• Embeds linguistic information• Avoids storing redundant information through use

of many connections• Use data base to generate questions• Common in other disciplines of AI


Domain Knowledge: Simulation

• STEAMER: – Mathematically simulate the steam propulsion

system– Tie graphics to the simulation

• SOPHIE– Propagates constraints to explain why a

behavior is caused


Domain Knowledge:Condition/action rules

• Popular in AI

• Model of human intelligence (?)

• “Recognize a condition, initiate an action”

• Attractive because rules are modular


Domain Knowledge:Summary

• One representation doesn’t work for everything.

• Often need multiple representations within one problem, WHY

• Must be determined by how knowledge is to be used


ICAI Student Modeling

• Goal: Know what the student knows

• CAI: Keep a tally of correct and incorrect answers– Little adaptation to student

• Methods:– Overlay modeling (Goldstein, 1977)– Buggy modeling (R. R. Burton, 1982)


Student Modeling:Overlay

• Represent student knowledge as some function of the teacher’s knowledge.– Allows comparison between what student

knows and what student should know.

• WEST, SCHOLAR, WUMPUS


Student Modeling:Buggy Modeling

• Include both “buggy” and correct rules which the student may be following

• Allows students error to be understood

• May require enumeration of all possible errors!


Student Modeling:Summary

• Student Modeling still very open-ended

• A full discussion is beyond scope of paper

• Allows computer to find reasons behind student errors – student diagnosis.


ICAI Student Diagnosis

• Goal: Allow student to make mistakes, capitalize on them for better learning.

• Methods:– Differential modeling– Direct interpretation– Plan recognition (buggy model)– Error taxonomy


Student Diagnosis:Differential Modeling

• Like overlay modeling: View a student error as a shortcoming that is detected with comparison to the tutor’s knowledge.

• WEST


Student Diagnosis:Direct Interpretation

• Remove constraints on question, until student’s answer becomes valid:

• Example: “What is the capital of Texas?”“Madison”

“Madison is the capital of Wisconsin.”

• Reasons through a semantic net


Student Diagnosis:Plan recognition

• Buggy model: try to find path in the model, (correct or incorrect) leading to student’s answer

• Plan recognition: finding the goals which underlie student actions

• Similar to language parsing


Student Diagnosis:Error Taxonomy

• Classify errors into types

• Example of categories:– Mission information– Lack of concept– Misfiled fact– Overgeneralization

• SCHOLAR


Student Diagnosis:Summary

• Student diagnosis is not goal: teaching is

• Most diagnosis can be made easier by asking a few more questions

• Allowing student to discover own errors is more effective (Socratic)

• “A little meaningful feedback goes a long way”


ICAI Pedagogical Knowledge

• Teachers need to know more than just their subject: they need to know how to teach.

• Main problems– Lesson planning– Dealing with student errors

• Production rules


Pedagogy:Lesson Planning

• Develop strategies for ordering topics

• Decide how to present material

• Decide balance of control between tutor and student


Pedagogy:Dealing with student errors

• Two big decisions:– Decide when to interrupt student– Decide what to say

• Common ideologies:– Trap student into discovering error– Allow student to see consequences of actions– Redirect the student– Affirm correct choices


Pedagogy:Summary

• Just knowing the problem domain isn’t enough

• Effective teachers have teaching “common sense”

• Effective teachers respond to students


ICAI Discourse Management

• Goal: Flexibility in the tutorial discourse• CAI: Hard-code syllabus, sometimes with

alternate paths• Methods:

– Reactive– Incremental knowledge-building– Context dependent– Hierarchical planning


Discourse Management:Reaction

• “Allow responses and misconceptions of student to drive the dialog”

• SCHOLAR, WHY

• Have a few initial goals (WHY), and modify them as session proceeds


Discourse Management:Incremental Building

• Add on to student’s current knowledge– Further develop a strong base– Explore new topics

• WUMPUS


Discourse Management:Context Dependent

• Use context to disambiguate questions, find answers

• Context = Position, progress and current task of student

• Object Oriented Tutoring incorporates this into a subject object


Discourse Management:Hierarchical planning

• PhD dissertation of Beverly Woolf, 1984

• Top-down refinement of goals

• Domain independent


Discourse Management:Summary

• Discourse management requires knowledge

• Knowledge needed not just in subject area

• Authors vary in opinion of how much flexibility is best.


ICAI Problem Generation

• CAI: canned problems, canned answers– Hard for course author

– No adaptation to student

– Limited meaningful feedback

• Generative CAI: programs generate new problems• Methods:

– Problem-generation trees

– Slot filling


Problem Generation:Trees

• Concept tree: – Student is at a level in the tree– Tree determines what to include in question

• Use context-free grammar to form actual question


Problem Generation:Slot filling

• Choose a kind of problem– Example: fill-in-the-blank, multiple choice

• Fill in information to problem from information in semantic net

• Requires rich knowledge base


Problem Generation:Summary

• Tree-like structures are used for generating problems

• Problems that are generated must also be solved


ICAI User Interface

• Tutoring systems should include many senses

• Communication methods:– Graphics– Canned Text– Text generation


User Interface:Graphics

• Graphics allow representation of concepts difficult to explain in words

• Graphics allow user to more fully feel part of the environment

• STEAMER


User Interface:Canned Text

• Most communication in tutoring is in English

• Store text phrases at many levels, select appropriate statements as needed.

• Still more flexible than CAI• Few systems do much else• Also: use canned sentence fragments to

make complete sentences.


User Interface:Text Generation

• SCHOLAR

• Includes knowledge for NLP

• Chooses a style of question, fills in key words from semantic net

• No canned text


User Interface:Summary

• Whole tutoring system is really one big User Interface

• Input of information is more difficult– Most systems use graphics or menus, don’t

mess with parsing natural language.

• Natural Language is “Achilles heel” of tutoring systems.


Summary

• ICAI systems require:– Learning scenario that is appropriate to domain

knowledge

– Student Models, Pedagogical knowledge, and Discourse knowledge are necessary

– Wrap it all in a sensory-stimulating interface

Nature of domain knowledge

Types of misconceptions } Knowledge Representation


Questions and Comments?

a.m. davisapril 29 th, 2002 intelligent computer-aided instruction: a survey organized around system...

Documents

ai icai

student slide

commercial icai systems

icai goals

icai representative

icai learning scenarios

student needs guidance

common sense reasoning