1 usc information sciences institute gil & kim interactive knowledge acquisition tools: a...

1USC INFORMATION SCIENCES INSTITUTE Gil & Kim

Interactive Knowledge Acquisition Tools:

A Tutoring Perspective

Yolanda GilJihie Kim

USC/Information Sciences Institute

August 9, 2002


Motivation: Investigate Synergies between Instructional Systems and Acquisition Tools

Instructional System

AcquisitionTool teaches

teaches

?

?

GoodTutoringPrinciples

GoodLearningPrinciples

SOFTWARE USER


Our Previous Work in Knowledge Acquisition:The EXPECT Project at USC/ISI:

EXPECT architecture for knowledge-based systems exploits highly declarative representations [Swartout & Gil, KAW-95], [Gil & Melz, AAAI-96] [Blythe et al, IUI-

01] http://www.isi.edu/expect

Research focus: interactive knowledge acquisition (KA) tools that help end users to develop knowledge bases Deriving models of knowledge interdependencies to detect

knowledge gaps and errors [Kim & Gil, AAAI-99] [Kim & Gil, IUI-2000] [Kim & Gil, AAAI-2000]

KA dialogue scripts to guide users by following up on effects of complex changes [Gil & Tallis, AAAI-97] [Tallis & Gil, AAAI-99] [Tallis, IJHCS-2001]

Exploiting background theories to understand how new knowledge fits [Blythe, IJCAI-2001] [Blythe, AAAI-02]

http://www.isi.edu/expect




EXPECT: A User-Centered Framework for Developing KBSs

Method instantiator

Knowledge BaseDomain

ontologiesand factual knowledge

Problemsolving methodsDomain

dependentKBS

KBS compiler

Knowledge-BasedSystem

InterdependencyModel (IM)

EXPECT Ontologies and Method libraries

KA toolsEMeD

Plans(PLANET)

Evaluations and Critiques

Evaluation PSMs

Resources(OZONE)

DomainontologiesCYC/Sensus

Upper

NL Editor

Instrumentation

Dialogue plans(KA Scripts)

PSMTool


Brief Overview of Representative KA Tools (I)

CHIMAERA [McGuinness et al 2000] Acquisition of concepts, relations, instances Diagnoses faulty definitions

EXPECT [Blythe et al 2001] Acquisition of problem solving knowledge Exploits dialogue scripts, interdependency models, bg k

INSTRUCTO-SOAR [Huffman & Laird 1995] Acquisition of task models in Soar Situated NL instruction is mapped to PSCM [Newell et al. 1991]

KSSn [Gaines & Shaw 1993] Acquisition of concepts, rules, data Based on personal construct psychology [Kelly 1955]

PROTOS [Bareiss et al 1990] Acquisition and classification of new cases Learning indexes to categories


Brief Overview of Representative KA Tools (II)

SALT [Marcus & McDermott 1989] Acquisition of constraints and fixes for configuration design Exploits Problem Solving Method/ Task (Role-limiting approach)

SEEK2 [Ginsberg et al. 1985] Acquisition of rules Uses verification and validation techniques

SHAKEN [Clark et al. 2001] Acquisition of process models User interaction based on concept maps [Novak 1977]

TAQL [Yost 1993] Acquisition of SOAR rules Editor for high level language for PSCM [Newell et al. 1991]

TEIREISIAS [Davis 1979] Acquires and classifies new cases Learning indexes to categories


Open Challenges in KA Users remain largely responsible for the acquisition

process Decide where, what, when, how, why to enter knowledge System checks errors, may have some short-term acquisition

goals Ideally, KA tools should have student-like skills:

Formulate and pursue learning goals Keep track of lessons and progress Assess how much they are learning and how useful k is If teacher is not so great, still capable of learning


Instructional Systems and Acquisition Tools: What Are the Synergies?

Instructional System

AcquisitionTool teaches

teaches

?

?

GoodTutoringPrinciples

GoodLearningPrinciples

SOFTWARE USER

SupplementStudent’s limitations

SupplementTeacher’s limitations


Teaching/Learning principle Tutoring literature

Start by introducing lesson topics and goals

Atlas-Andes, Meno-Tutor, Human tutorial dialog

Use topics of the lesson as a guide

BE&E, UMFE

Subsumption to existing cognitive structure

Human learning, WHY, Atlas-Andes

Immediate Feedback SOPHIE, Auto-Tutor, Lisp tutor, Human tutorial dialog, human learning

Generate educated guesses Human tutorial dialog, QUADRATIC, PACT

Keep on track GUIDON, SHOLAR, TRAIN-TutorIndicate lack of understanding Human tutorial dialog, WHY

Tutoring and Learning Principles Relevant to KA [Kim & Gil, ITS 02] (I)


Teaching/Learning principle Tutoring literature

Detect and fix “buggy” knowledge SCHOLAR, Meno-Tutor, WHY, Buggy, CIRCSIM

Learn deep model PACT, Atlas-AndesLearn domain language Atlas-Andes, Meno-TutorKeep track of correct answers Atlas-AndesPrioritize learning tasks WHYLimit the nesting of the lesson to a handful

Atlas

Summarize what was learned EXCHECK, TRAIN-Tutor, Meno-Tutor

Provide overall assessment of learning knowledge

WEST, Human tutorial dialog

Tutoring and Learning Principles Relevant to KA [Kim & Gil, ITS 02] (II)


ASSIMILATEINSTRUCTION

TRIGGERGOALS

PROPOSESTRATEGIES

PRIORITIZEGOALS &

STRATEGIES

PRESENTATIONDESIGN

USER INTERFACE

KNOWLEDGE ACQUISITION BACKEND

Five Main Functions of KA Tools

Knowledge Base



TRIGGERGOALS

PROPOSESTRATEGIES

PRIORITIZEGOALS &

STRATEGIES

PRESENTATIONDESIGN

USER INTERFACE


Guidance Exploited by KA Tools Guidance fromKnowledge Base

Problem Solving & Task Knowledge

Domain Knowledge

General Background Knowledge

Example Cases

Guidance fromMeta Knowledge

Knowledge Repres.Model

Diagnosis & Debugging Principles

Tutoring & Learning Principles


Acqu. goals

Acqu. strats

GuessGenerators

InteractionGuidelines

OperationalPrinciples

PrioritySchemes

GeneralTutoring

&Learning

Principles

KnowledgeEditor

Dialogue

- Goals & Strats- State- Suggestions- History

USER INTERFACE


Tutoring and Learning Principles in KA Tools: Basic Conceptual Framework

Knowledge Base


TRIGGERGOALS

PROPOSESTRATEGIES

PRIORITIZEGOALS &

STRATEGIES

PRESENTATIONDESIGN


Tutoring and Learning Principles Implicit in KA tools

KSSnAssess learned knowledge

Summarize what is learned

EXPECTPrioritize learning tasks

SEEK2Keep track of answers

Learn domain language

Learn deep models

EXPECT,CHIMERATAQLDetect and fix “buggy” K

INSTRUCTO-SOAR

INSTRUCTO-SOAR

Indicate lack of understanding

Keep on track

EXPECTTEIREISIASGenerate educated guesses

EXPECTTEIREISIASINSTRUCTO-SOARPROTOSImmediate feedback

PROTOS, SALTTEIREISIASPROTOSSubsumption to existing cog. structure

SALTEXPECTSEEK2SALTUse topics of the lesson as a guide

EXPECT, SEEK2Introduce topics & goals

Design Presentation

PrioritizeGoals & Strats

ProposeStrategies

TriggerGoals

AssimilateInstruction

Tutoring/Learning principle

Limit the nesting of lessons


Tutoring and Learning Principles in KA Tools Observation: Some learning and tutoring principles

are used in some aspects of the dialogue by some tools

Opportunity: Incorporate principles more thoroughly in all aspects of the dialogue

Observation: These principles are implicit in the tool’s code and thus are limited

Opportunity: Exploit declarative representations of learning state, goals, and strategies


USER INTERFACE KB

Proactive Dialogue Window

Active Acquisition

Goals&

Strategies

Awareness Annotations

SLICK Dialogue Manager

KBState

Dial.History

SLICK (Skills for Learning and Interactively Capture Knowledge)


Tutoring&

LearningPrinciples


Conclusions Analysis of existing KA tools shows they use

tutoring/learning principles Sparsely Implicitly

Current capabilities of KA tools can be improved by: Representing tutoring/learning principles declaratively Organizing the dialogue around lesson topics Keeping track of how knowledge improves through dialogue Exposing what knowledge has been assimilated and what

areas need improvement or testing Assessing their competence and confidence on question

answering

1 usc information sciences institute gil & kim interactive knowledge acquisition tools: a...

Documents