reasoning in a tutoring system: transforming knowledge to teaching

July 6, 2004 7th International Protégé Conference

Reasoning in a Tutoring System: Transforming

Knowledge to Teaching.

Olga Medvedeva

Center for Pathology Informatics,University of Pittsburgh


Outline

• Our approach for teaching visual diagnosis

• General system architecture

• Knowledge representation in different tutor modules

• Pluses and minuses of our system


Medical KB Training System Challenge

• Problems – Medical knowledge is complex and dynamic– Errors in KB can cause serious problems– Demands on extendibility and maintenance of

large KBS

• Requirements– Combine knowledge representation and flexible

instructional system– Adaptive for new observables and unique

strategies– Reusable and modular


Intelligent Tutoring SystemsParadigm

ITS strive to replicate a method of teaching and learning exemplified by a one-on-one human tutoring interactionModel Tracing ITS guide user through

problem space, can correct each small intermediate reasoning step Cognitive Tutors based on ACT_R

theory of learning proceduralize declarative knowledge in the rules(step instructions)


Student Module

Pedagogic Knowledge

Interface

Expert Model

•Allow correct steps•Correct errors•Give hints on next step

•Collect data on what student does•Make predictions on what student knows•Provide data for pedagogic decision making

•Canvas for problem solving•Make goals visible

•Case sequence•When to intervene•How to intervene

Intelligent Tutor System Structure


Disadvantages of ITS Paradigm

• Developed for highly procedural domains• Not designed for large complex dynamic

declarative knowledge• Domain specific production rules

knowledge representation• Maintenance is difficult and time

consuming• Knowledge modification alter the rules


SlideTutor Characteristics

SlideTutor – a system to teach visual classification problem solving in Pathology

• Similar to other medical diagnostic tasks– Combination of search, identification, interpretation– Well characterized diagnostic reasoning in medical

domains– Some areas are highly algorithmic, some – not– Both empirical and theoretical work can guide the

development– Combination of heuristic classification and

deductive/inductive reasoning is the best foundation for classification problem-solving.


SlideTutor Approach

• Combine the aspects and methodology of both KBS and ITS to create a general framework for teaching decision-making process for classification problems in Dermopathology using UPML Component Mode approach.– Extract and modularize all expert and pedagogic

declarative knowledge into ontologies => make domain task neutral

– Reuse PSM by tutor procedural rule based system => make system domain neutral

– Preserve all of the major pedagogic components associated with Cognitive Tutors in ontologies and rules => add significant flexibility to pedagogic model


Instructional Layer

Pedagogic TaskStructure

PedagogicTask

DermatologyKnowledge Base

DomainModel

VisualClassification

Task Structure

DomainTask

PedagogicKnowledge Base

PedagogicModel

Case Database

Interface

Expert Model

Student Model

StudentModelState

StudentModelStateStudent Model

Data

SlideRepresentation

Case Data

SlideRepresentation

Case Data

SlideRepresentation

Case Data

Student

Dynamic Solution Graph

Pedagogic Model

DomainBehaviorRefiner

ProblemSolvingMethods

PedagogicBehaviorRefiner

ProblemSolvingMethods

SlideTutor General Architecture


Domain Model

• Set of ontologies that express relationships between evidence and disease concepts

• Uses Motta’s parametric design approach (slightly extended by adding attributes to features)

• Similar disease and evidence representation– Hierarchical structure with multiple inheritance

for diseases– Set of evidences represent set of diseases– Both can occur multiple times in different sets


Feature – Domain KB – Case Relationship


Task Model

Models the abstract structure of the Dynamic Solution Graph (DSG) – a directed acyclic graph

• Represents possible relationships in the domain knowledge that are pertinent for reasoning– Identifying region– Identifying and refining a set of features– Triggering one or more hypothesis– Creating a differential diagnosis– Finding features that distinguish between the hypotheses– Defining that critical feature is absent– Linking supportive features to a particular hypothesis– Accepting some hypotheses as diagnosis

• Direction of DSG is defined by an order of some steps in task

(deftemplate task

(slot type)

(multislot parent)

(slot role)

(slot required)

(slot priority))


JessTab Extensions

• Added UserFunctionsload-jdbc-project - load db projectdisposep - dispose current Protégé

• Modified code– Preserve class hierarchy structure– Multiple inheritance (MAIN::NEUTROPHILS

(is-a NEUTROPHILS) (is-a-name "NEUTROPHILS") (OBJECT <External-

Address:edu.stanford.smi.protege.model.DefaultSimpleInstance>) (has-parents "INFLAMMATORY INFILTRATE")

(feature_name "isolated neutrophils"))


Dynamic Solution Graph

• Generates path through problem state based on combination of Domain, Task and Case models

• Dynamic – no predefined solution – each cycle generates the current problem state and all valid next steps

• Contains a set of abstract PSM that allow to add/delete/update nodes and arcs

• Path through the problem is defined by a consequence of student actions

• Behavior structures encapsulate node type specific response to a triggered event

• Supports forwards and backwards reasoning


DSG Implementation

(deftemplate node

(slot type (type STRING))

(multislot property_name ) ;; e.g “name” “x” “y” “z”

(multislot property_value)

(slot internal_id (type STRING))

(slot state (default " INITIAL ")) ;; INITIAL, IDENTIFIED

(slot input_value) ;; easy match with useraction input slot

(slot external_id (default nil)) ;; id of a corresponding object on user side

(slot is_goal (default FALSE))

(slot is_from_case (default FALSE))) ;; node can not delete if came from case

• Node reflects the semantic meaning of fact• Correct student action must match all of the node properties• State indicates that step was performed by user or not• Interpretation of action is left to the instructional layer• Special node type – Cluster node – expresses integrated

relation between a specific group if nodes and nodes outside it


DSG Cognitive Values

• Enables rapid feedback • Provides a method for stepping forward in

the model to generate next-step hints• Supports intermediate solution and revision• Determines general classes of errors and

allows pedagogic model to remediate them• Provides flexibility in tutor response• Reusable, because domain and

pedagogically independent


Instructional Layer

• Pedagogic Model– Explanation of a particular student error and rich

next-step hints upon student request– Delivered messages contain context-specific text

accompanied by the pointers to the places of interest on the user side

– Determines the most appropriate error from the error list generated by the DSG as a response to incorrect student action based on the state of student model

– Hierarchical hints from general to most specific and directive

• Pedagogic Task – represents the goal of the instructional process


Case-Focused Interface

• Local view of the problem


Knowledge-Focused Interface

• Global view of the problem (use SpaceTree cs.umd.edu)


Conclusion

• Preserved essential characteristics of CT • Utilized KB for modeling knowledge across the

system components• Modular and flexible set of frames and

methods to teach classification problem solving

• Limitation – deterministic approach– No support for probabilistic relationship between

evidence and hypothesis– No attempt to model all evidence combinations or

incomplete evidence– No reasoning under uncertainty


Acknowledgements

• NLM 1 R01 LM007891-01 (Crowley, PI)

• Rebecca Crowley, Pathology Informatics• Eugene Tseytlin, Pathology Informatics • Elizabeth Legowski, Pathology Informatics• Gerish Chavan , Pathology Informatics• Maria Bond , Pathology Informatics


More details at Demo Session

• Integrating Protégé into an Intelligent Medical Training System– Ontologies– Knowledgebase Validation Tool– Case Authoring Protégé plug-in– Dynamic Solution Graph– Protocol Filter Query– SlideTutor– DinoTutor

reasoning in a tutoring system: transforming knowledge to teaching

Documents