rika yoshii, ph.d. and jacquelyn hernandez [email protected] csis department california state...
TRANSCRIPT
Rika Yoshii, Ph.D. and Jacquelyn Hernandez [email protected] DepartmentCalifornia State University, San Marcos
Send us suggestions and requests to use the system
ItsLEADR: INTELLIGENT TUTORING SYSTEM FOR LEARNING ENGLISH
ARTICLES BY DIAGRAMMATIC REASONING
CALICO 2015
Our Goals BackgroundSpeaker’s IntentDiagramsPedagogy
Domain ModelStudent ModelPreliminary
EvaluationSummaryFuture Work
TOPICS
Help ESL students develop reasoning skills in choosing English articles such as “a/an” and “the.”
Avoid the use terms such as “specific” and “definite” which are foreign concepts for students whose native languages do not contain an article system.
Must emphasize the speaker’s intent. “Speaker’s Intent determines the article” (Brown, 1973)(Celce-Murcia 1983)
OUR GOALS
Conducted experiments with ESL students at CSUSM to set our goals.
Started with the DaRT system (CALICO 1998):1. Incorporated Diagrammatic Reasoning2. Showed the intent of the speaker
Students practiced understanding the reasons behind article usage by looking at diagrams.
BACKGROUND
send comments to [email protected]
1. We refined the speaker’s intent types.2. We simplified the intent depicting
diagrams, based on feedback from students at CSUSM.
3. We incorporated the pedagogical model, the domain model and the student model to provide individualized tutoring and to enforce mastery learning.
4. We continuously improved the graphical user interface based on feedback from test users.
STEPS TO THE CURRENT INTELLIGENT SYSTEM
send comments to [email protected]
7 intent types for “What is the speaker trying to communicate
through the use of an article?” :
1. In Current Focus 2. Introduction 3. Asking Awareness 4. General Statement 5. Category is Important 6. Entailment for Whole 7. Entailment for Part
SPEAKERS INTENT TYPES
In Current Focus– The speaker is communicating that the referent is in current focus on the discussion between the speaker and the listener.
E.g. “And the teacher walked out.”Entailment for Whole - The speaker is
communicating that he is referring to a complete part of another object.
E.g. “Please change the tires.”
SPEAKER’S INTENT – EXPLAINED (SINGULAR “THE”, PLURAL
“THE”)
send comments to [email protected]
Introduction– the referent is being introduced by the speaker to the listener.
E.g. “I saw a cute dog today.”Asking Awareness– the speaker is asking if the listener
has the knowledge of the referent. E.g. “Did you see a man running that way?”
General Statement – the speaker is making a general statement about a category.
E.g. “Tigers are dangerous.”Category is Important – the speaker wants to emphasize
the category more than a particular instance of it. E.g. “Hand me a pen.”
Entailment for Part – the speaker is referring to an incomplete part of another object.
E.g. “Please change a tire.”
SPEAKER’S INTENT – EXPLAINED (SINGULAR “A”, PLURAL NONE)
send comments to [email protected]
Is the referent a category, a group, or its member(s)?
Is the referent currently known by the speaker alone or by both the speaker and the listener?
Is the referent entailed by the referent of another noun phrase?
SPEAKER’S INTENT - COMPONENTS
send comments to [email protected]
Intent components are depicted by:The shape of the noun nodeThe image of the link nodeThe location of the noun node in the Venn
Diagram
DIAGRAMS
send comments to [email protected]
A concrete element A concrete groupA conceptual category
NOUN NODES
send comments to [email protected]
LINKS
•IS-link is used when a noun phrase refers to the noun node itself.•IS-IN-link is used when a noun phrase refers to some member(s) of the noun node.•IS-ALL-OF-link is used when a noun phrase refers to all members of a noun node.•IS-REP-link is used when a noun phrase refers to a representative of a noun node.
3 phases: Introduction – introduces intent types, diagrams and their components and asks review questions to make sure the student understands them.
Diagram selection - an intent type and a sentence are given, and the student is asked to choose the correct diagram.
Article selection - an intent type and a diagram are given, and the student is asked to fill in a blank space in the exercise sentence with an article.
If the Help button is clicked, then a pop up window reviewing the diagrams and their components will appear.
PEDAGOGY
The student cannot move on to the next intent type until the mastery for that type is demonstrated by repeatedly answering correctly.
The student cannot move onto the next phase until the mastery of that phase is demonstrated..
MATERY LEARNING
Expert knowledge of the subjectUsed to check student answer againstUsed to generate exercises
Represented as a semantic net that contains the intent types and corresponding diagram components.
DOMAIN MODEL
Used to achieve individualized tutoring.
Components:Student performance record –
perturbedversion of the domain model.Predictability model – probabilistic inference for predicting the student answers based on performance.
Hint table - composed of the question type, the correct answer, the expected student answer, and corresponding hints addressing the student’s misconception.
STUDENT MODELING
If the expected student answer is an incorrect answer, the system will retrieve a hint from the hint table to present along with the question to the student.
If the student answers incorrectly, the system will retrieve a hint from the hint table to help the student.
Performance record is always updated.
Once the student has successfully mastered a phase, the system computes a student performance summary.
HOW THEY ARE USED
To help us identify areas of improvement in terms of the diagrams as well as the system features.
Nine ESL/EFL students at CSUSM (Chinese, Japanese and Korean)
1. Baseline survey2. Use ItsLEADR for one hour3. Feedback survey4. Interview
PRELIMINARY EVALUATION
send comments to [email protected]
Average of 3.44 (5 being the best) when rating their belief that the ItsLEADR system is useful.
Commented that the diagrams help to give context where as memorization was the main mechanism in classrooms.
Found the links to be diffi cult to understand and remember.
Most participants did not read the introductory information and relied on the Help button.
EVALUATION RESULTS
Revise the links and/or link definitions and images to make them easy to understand and remember.
Create a training manual written in English.
Add online help features such as balloons and visual walkthrough.
SUGGESTED BY EVALUATORS
Enhanced DaRT from a CALL system to an intelligent tutoring system.
Refined the intent types.Improved the diagrams.Individualized tutoring by predicting
student answers.Developed in C++ with Qt for GUI.Performed preliminary evaluation.
SUMMARY
Enhance the prediction of student answers by determining the root cause of errors and use this information in giving hints and selecting remedial exercises.
A larger scale formative evaluation followed by further improvements.
A summative evaluation with experimental groups and a control group, comparing improvements and retention over several months.
Please send suggestions and requests to use the system to [email protected]
FUTURE WORK