rika yoshii, ph.d. and jacquelyn hernandez ryoshii@csusm.edu csis department california state...

Rika Yoshii, Ph.D. and Jacquelyn Hernandez ryoshii@csusm.eduCSIS DepartmentCalifornia State University, San Marcos

Send us suggestions and requests to use the system

ItsLEADR: INTELLIGENT TUTORING SYSTEM FOR LEARNING ENGLISH

ARTICLES BY DIAGRAMMATIC REASONING

CALICO 2015

Our Goals BackgroundSpeaker’s IntentDiagramsPedagogy

Domain ModelStudent ModelPreliminary

EvaluationSummaryFuture Work

TOPICS

Help ESL students develop reasoning skills in choosing English articles such as “a/an” and “the.”

Avoid the use terms such as “specific” and “definite” which are foreign concepts for students whose native languages do not contain an article system.

Must emphasize the speaker’s intent. “Speaker’s Intent determines the article” (Brown, 1973)(Celce-Murcia 1983)

OUR GOALS

Conducted experiments with ESL students at CSUSM to set our goals.

Started with the DaRT system (CALICO 1998):1. Incorporated Diagrammatic Reasoning2. Showed the intent of the speaker

Students practiced understanding the reasons behind article usage by looking at diagrams.

BACKGROUND

send comments to ryoshii@csusm.edu

1. We refined the speaker’s intent types.2. We simplified the intent depicting

diagrams, based on feedback from students at CSUSM.

3. We incorporated the pedagogical model, the domain model and the student model to provide individualized tutoring and to enforce mastery learning.

4. We continuously improved the graphical user interface based on feedback from test users.

STEPS TO THE CURRENT INTELLIGENT SYSTEM

send comments to ryoshii@csusm.edu

7 intent types for “What is the speaker trying to communicate

through the use of an article?” :

1. In Current Focus 2. Introduction 3. Asking Awareness 4. General Statement 5. Category is Important 6. Entailment for Whole 7. Entailment for Part

SPEAKERS INTENT TYPES

In Current Focus– The speaker is communicating that the referent is in current focus on the discussion between the speaker and the listener.

E.g. “And the teacher walked out.”Entailment for Whole - The speaker is

communicating that he is referring to a complete part of another object.

E.g. “Please change the tires.”

SPEAKER’S INTENT – EXPLAINED (SINGULAR “THE”, PLURAL

“THE”)

send comments to ryoshii@csusm.edu

Introduction– the referent is being introduced by the speaker to the listener.

E.g. “I saw a cute dog today.”Asking Awareness– the speaker is asking if the listener

has the knowledge of the referent. E.g. “Did you see a man running that way?”

General Statement – the speaker is making a general statement about a category.

E.g. “Tigers are dangerous.”Category is Important – the speaker wants to emphasize

the category more than a particular instance of it. E.g. “Hand me a pen.”

Entailment for Part – the speaker is referring to an incomplete part of another object.

E.g. “Please change a tire.”

SPEAKER’S INTENT – EXPLAINED (SINGULAR “A”, PLURAL NONE)

send comments to ryoshii@csusm.edu

Is the referent a category, a group, or its member(s)?

Is the referent currently known by the speaker alone or by both the speaker and the listener?

Is the referent entailed by the referent of another noun phrase?

SPEAKER’S INTENT - COMPONENTS

send comments to ryoshii@csusm.edu

Intent components are depicted by:The shape of the noun nodeThe image of the link nodeThe location of the noun node in the Venn

Diagram

DIAGRAMS

send comments to ryoshii@csusm.edu

A concrete element A concrete groupA conceptual category

NOUN NODES

send comments to ryoshii@csusm.edu

LINKS

•IS-link is used when a noun phrase refers to the noun node itself.•IS-IN-link is used when a noun phrase refers to some member(s) of the noun node.•IS-ALL-OF-link is used when a noun phrase refers to all members of a noun node.•IS-REP-link is used when a noun phrase refers to a representative of a noun node.

send comments to ryoshii@csusm.edu

GENERAL STATEMENT DIAGRAM IS-LINK - CATEGORY - SHARED

send comments to ryoshii@csusm.edu

GENERAL STATEMENT DIAGRAM IS-REP-LINK - CATEGORY - SHARED

send comments to ryoshii@csusm.edu

IN CURRENT FOCUS DIAGRAM IS-LINK – ELEMENT - SHARED

send comments to ryoshii@csusm.edu

INTRODUCTION DIAGRAM IS-LINK –ELEMENT - SHARED

send comments to ryoshii@csusm.edu

CATEGORY IMPORTANT DIAGRAMIS-IN-LINK –CATEGORY - SHARED

send comments to ryoshii@csusm.edu

ENTAILMENT DIAGRAMIS-ALL-OF – ENTAILED GROUP -

SHARED

3 phases: Introduction – introduces intent types, diagrams and their components and asks review questions to make sure the student understands them.

Diagram selection - an intent type and a sentence are given, and the student is asked to choose the correct diagram.

Article selection - an intent type and a diagram are given, and the student is asked to fill in a blank space in the exercise sentence with an article.

If the Help button is clicked, then a pop up window reviewing the diagrams and their components will appear.

PEDAGOGY

The student cannot move on to the next intent type until the mastery for that type is demonstrated by repeatedly answering correctly.

The student cannot move onto the next phase until the mastery of that phase is demonstrated..

MATERY LEARNING

send comments to ryoshii@csusm.edu

1. Domain Model2. Student Model

INTELLIGENT TUTOR COMPONENTS

Expert knowledge of the subjectUsed to check student answer againstUsed to generate exercises

Represented as a semantic net that contains the intent types and corresponding diagram components.

DOMAIN MODEL

send comments to ryoshii@csusm.edu

PART OF DOMAIN MODEL

Used to achieve individualized tutoring.

Components:Student performance record –

perturbedversion of the domain model.Predictability model – probabilistic inference for predicting the student answers based on performance.

Hint table - composed of the question type, the correct answer, the expected student answer, and corresponding hints addressing the student’s misconception.

STUDENT MODELING

If the expected student answer is an incorrect answer, the system will retrieve a hint from the hint table to present along with the question to the student.

If the student answers incorrectly, the system will retrieve a hint from the hint table to help the student.

Performance record is always updated.

Once the student has successfully mastered a phase, the system computes a student performance summary.

HOW THEY ARE USED

send comments to ryoshii@csusm.edu

EXAMPLE PERFORMANCE SUMMARY

To help us identify areas of improvement in terms of the diagrams as well as the system features.

Nine ESL/EFL students at CSUSM (Chinese, Japanese and Korean)

1. Baseline survey2. Use ItsLEADR for one hour3. Feedback survey4. Interview

PRELIMINARY EVALUATION

send comments to ryoshii@csusm.edu

Average of 3.44 (5 being the best) when rating their belief that the ItsLEADR system is useful.

Commented that the diagrams help to give context where as memorization was the main mechanism in classrooms.

Found the links to be diffi cult to understand and remember.

Most participants did not read the introductory information and relied on the Help button.

EVALUATION RESULTS

Revise the links and/or link definitions and images to make them easy to understand and remember.

Create a training manual written in English.

Add online help features such as balloons and visual walkthrough.

SUGGESTED BY EVALUATORS

Enhanced DaRT from a CALL system to an intelligent tutoring system.

Refined the intent types.Improved the diagrams.Individualized tutoring by predicting

student answers.Developed in C++ with Qt for GUI.Performed preliminary evaluation.

SUMMARY

Enhance the prediction of student answers by determining the root cause of errors and use this information in giving hints and selecting remedial exercises.

A larger scale formative evaluation followed by further improvements.

A summative evaluation with experimental groups and a control group, comparing improvements and retention over several months.

Please send suggestions and requests to use the system to ryoshii@csusm.edu

FUTURE WORK