Compurqrs E&c. Vol. 20, No. I, pp. 141-146, 1993 Prirted in Great Britain. All rights reserved

0360- I3 I5193 $6.00 + 0.00 Copyright 0 1993 Pergamon Press Ltd

GEOMETRY PROBLEM SOLVING WITH MENTONIEZH

DOMINIQUE PY

IRISA, Campus de Beaulieu, 35042 Rennes Cedex, France

Abstract-The Mentoniezh project is an ITS in Euclidian plane geometry for children aged 13-14. The system guides and corrects a student during problem solving. Its main original feature is a plan recognition method which deduces, from the student’s inputs and the search space of the problem, the student’s intended proof. That enables the student to be provided with context-sensitive help. The different parts of the tutor, the expert model, the student model and the errors analysis method are described. Finally, the evaluation results and a comparison with other systems are presented.

INTRODUCTION

A goal of Intelligent Tutoring Systems (ITS) is to provide teachers with sophisticated pedagogi- cal tools in open disciplines such as programming or problem-solving. The definition of such systems encounters difficulties: usually there are several solutions to a given problem and the successive steps of resolution cannot be analysed separately. For example, in geometry, the correctness of one proof step depends on the previous ones.

The Mentoniezh project is an ITS in Euclidian plane geometry, for children aged 13-14. The system guides and corrects a student during the two stages of problem solving: figure drawing and proof writing[l,2]. In this paper, only proof writing is addressed.

The most natural proof, according to a pupil, is not necessarily the straightest one, neither the one chosen by a teacher. So, an interesting design for a tutor is one which accepts a correct though non-optimal proof. That is the reason why we need to recognize the stu- dent’s intended proof, in order to provide well-adapted assistance, while avoiding too much directiveness. That proof can be formalized as plans, the actions of which are applications of theorems. A plan recognition method is presented, which deduces, from the search space of the problem and the student’s inputs, the underlying intention and can also detect any shift in intention.

Apart from that, the system relies on the classical architecture of an ITS. The expert in the domain, a theorem prover using about 60 rules, can solve exercises in different ways and distinguish the student’s erroneous or useless inferences. The student model consists of the history of recognized plans. The misconceptions model, a library of common mistakes formal- ized as transformation meta-rules, is able to recognize basic and combined errors in the student’s deductions. The pedagogical model is used to conduct the interaction between the student and the system, to provide explanations whenever the student applies an erroneous reasoning, and to provide clues when in deadlock. Lastly, experiments with the system are described and compared with related work.

GEOMETRY PROBLEM-SOLVING

Informally,. a problem in geometry takes the form of a statement in natural language, including a set of hypotheses considered as true, and a conclusion or goal to be demonstrated. Consider the following example, where the first two sentences are the hypotheses and the third one is the required conclusion:

Let ABC be an isosceles triangle. The line parallel to AB passing through C meets the line parallel to AC passing through B at, &he point K. Prove that ABKC is a rhombus.

I41

142

The

DOMINIQUE PC

ABCD isosceles AB/ JCK AC//BK

$ I ABKC rhombus

Fig. 1. The search graph.

tools are theorems and definitions, for example:

Definition of an isosceles triangle: an isosceles triangle is a triangle with two equal sides.

These definitions and theorems apply to proved properties, to produce new properties. In a formal manner, the search space of a given problem is viewed as an and/or graph, the nodes

of which are properties at odd levels and theorems at even levels. Then a proof is a solution tree in this graph. The leaves of the tree are the problem’s hypotheses, and the root is the problem’s conclusion. The example presented in Fig. 1, although quite simple, can serve as a basis for comment.

On the one hand, there may exist several proofs for one problem, that is, several proof trees in one graph. For example the proof formed of theorems 1, 2 and 3 in Fig. 1 is the shortest one, but the proof obtained when replacing 3 by 4 and 5, although one step longer, is correct too.

On the other hand, a proof can be realized in different ways:

-From hypotheses to conclusion (forward chaining). Theorems are applied to proved facts to produce other facts. For example: ABC is an isosceles triangle, so AB = AC.

-From conclusion to hypotheses (backward chaining). Theorems are applied to goals, in order to produce subgoals. For example: to prove that ABKC is a rhombus, it suffices to prove that ABKC is a parallelogram and that AB = AC.

-By combination of these two methods (mixed chaining). Some steps are realized in forward chaining, others in backward chaining.

Finally, some deductions, even geometrically sound ones, are not useful in the proof. On the example, it can be proved that ABKC is a trapezium, but this property cannot be used in a proof.

These remarks point to the great freedom left to the pupil in writing a proof. A simple diagnosis “right” or “wrong” at every step is not enough if we want to parse the student’s work. That is why a global analysis of the proof is essential. With this end in view, we use a plan recognition method, which tries to detect as soon as possible the user’s strategy, from previous observed steps. This method is the most original feature of the tutoring system.

THE AUTOMATED THEOREM-PROVER

An automated theorem prover, in a tutoring system, does not look like a classical one. It does not aim at finding an optimal proof, but rather at approaching the human way of reasoning, to understand the student’s performance and to provide suited explanations. So it must use the same rules as the pupil, and discover alternative solutions for non-trivial exercises. In particular, if students are to choose their own strategies, it is important to recognize than an inference is valid, even if it has little interest.

Geometry problem solving 143

Description language: the description language is a logical one, quite close to natural language, called HDL for Hypotheses Description Language. It can express complex properties in a straightforward manner. For example, the sentence “ABC is an isosceles triangle” is translated “isosceles(A,B,C)” rather than “equal(AB,AC)“, because the latter would be to perform one proof step in place of the student.

Example: the statement:

Let ABC be a right triangle. D is the midpoint of segment BC. Line E is parallel to AB, passing through point C.

is represented by:

right-triangle(A,B,C), midpoint(D,B,C), pass(C,E), parallel(E,AB)

Automated theorem prover: the deduction rules of the theorem prover fit the theorems of the french curriculum. Every rule is expressed by a logical formula.

Example: definition of a square:

VW, x, y, z, square(w, x, y, z) 0 rhombus(w, x,y, z) A rectangZe(w, x,y, z)

To reduce combinatorial explosion due to the numerous symmetries of geometry, any object is given a canonical name, so triangle ABC is the same as triangles ACB, BAC, CBA . . . and any line is given an internal name, so the line passing through points M, N, P can be called either MN or NM or NP . . . .

The theorem prover works in two phases. First, it produces all the properties deducible from the hypotheses by a systematic application of the theorems. This result is represented as a graph, the search space of the problem. Then, it enumerates the proof trees in this graph, i.e. the trees the roots of which are the conclusion, and the leaves of which are the hypotheses. When the search space is too large, and the trees too numerous, the prover only produces a limited number of solutions: those which are, at the most, n steps longer than the shortest one. In practice, n = 3 appeared to be quite enough, because pupils never engage in abnormally complex proofs.

PLAN RECOGNITION

Student modelling is defined by Wenger as “pedagogical activities aiming at collecting and inferring information about the student or his actions”[3]. Here, because of the teaching domain, a modelling method based upon plan recognition was used. Before presenting the main features of this method, it is useful to define the vocabulary.

A plan is a sequence of actions, aiming to reach a given goal. The performance of any action depends on the consequences of the previous ones. Reasoning about actions and plans is an important concern in artificial intelligence. Plan synthesis consists of determining the sequence of actions which will produce the intended result. Plan recognition is the reciprocal process. It consists of discovering from observed actions the plan being performed by the subject.

How can plan recognition apply to student modelling? In some domains, the learner’s activity can be viewed as the synthesis and the execution of a plan, That is the case in geometry problem solving. The task consists of elaborating a plan (the proof) and executing its actions (the proof steps). Then, recognizing the student’s plan enables one to follow progress. For example, errors in planning can be distinguished from miscalculations, and help can focus on the student’s idea rather than on a predefined “optimal” solution.

Most plan recognition methods are specific to a given domain, but the theory of Kautz is general and fits the purpose[4]. He defines a formal theory of plan recognition based upon a model of non-monotonic reasoning named circumscription and describes several algorithms which model different styles of recognition. Only one is described here, the “sticky covers” algorithm. The method is based upon the knowledge of actions likely to be observed, of plans likely to be executed, and of the links between these actions and plans. The leading principle is to minimize the number of possible explanations for each observation. When the subject performs a sequence of actions, one tries first to explain these actions by one plan, rather than connecting every action to a different plan. So, we assume that the subject is coherent, and acts with some logic.

144 DOMINIQUE PY

Usually, one plan (proof) suffices to reach the goal (prove the conclusion). But the student may change and begin a new plan, or resume an old plan. These attempts must be recognized and taken into account. When all the observed actions cannot be explained by a single plan, one among the recognized plans is a current plan which explains the most recent actions. The others are waiting plans.

Consider the previous example. There are two plans, Pl and P2. Pl comprises the steps SI, S2 and S3. P2 comprises the steps Sl, S2, S4 and S5.

First observation: the student performs Sl forward. That does not suffice to pick out the plan being executed, but indicates the strategy and the probable next action. Since Sl belongs to Pl and P2, one concludes that the current plan is PI v P2, and that the strategy is forward or mixed. According to the logical order of the actions inside the plans, the most likely continuation is S2.

Second observation: the student changes strategy, but not plan, and performs S5 backward. Since this action belongs to plan P2, and only to it, one can conclude that plan P2 is being executed. If the two actions were not linked, one could consider as well that Sl is explained by Pl and S5 by P2, or that Sl and S5 are both explained by P2. But the minimization principle discards the first explanation, and retains the second one, since it explains the whole sequence by only one plan. It follows that plan P2 is performed in mixed chaining. The most likey continuation becomes S2 (forward) or S4 (backward).

Third observation: the student changes to a new plan and performs S3 backward. This action is explained by PI. So it cannot be linked to the previous one. The system detects the beginning of a new plan, and tries successively to connect the current action with an older one, from the most recent to the less recent. This attempt succeeds with the first action. Then the system revises its conclusions, and assumes that the current plan is P2 (gathering the first and third observations). The expected next action is S2.

The results of plan recognition can be used in different ways. First, they enrich the student model with the strategy of the pupil: does the student follow a unique plan, or hesitate between two plans, or become completely lost? Then it enables one to provide the pupil with a clue concerning the next proof step, according to the recognized plan. This clue is either the next property to be proved, or the next theorem to be applied. It is delivered when the pupil asks for help. If no plan is recognized with enough precision, the tutor encourages the backward strategy, and proposes several theorems, the conclusion of which unifies with the current goal.

MISCONCEPTIONS ANALYSIS

The student who tries to write a proof can make different kinds of errors, from typing errors to deep errors such as semantic mistakes or planning errors. These errors have to be analysed, and their origin recognized, in order to provide the student with a suited explanation.

A taxonomy of errors has been established on the basis of tests and evaluations[5] in collaboration with the GRECO didactique. (Group de REcherches COordonnees en didactique des mathematiques, a national research project on the didactic of mathematics gathering teachers and researchers). There are four kinds of errors:

Typing or inattention errors: for example, using an object (point, line, circle) which does not exist in the problem statement.

Logic errors: for example, the inversion of a premise and a consequence, or the confusion between a theorem and its converse.

Semantic errors: a confusion between words, e.g. “parallel” and “equal”, which entails a confusion between theorems.

Planning errors: the student tries a theorem in forward chaining, but did not prove its premises.

Typing errors are detected at the interface level and immediately corrected. Planning errors which correspond to correct although premature inferences are detected by the plan recognition method. Logic and semantic errors which concern the handling of geometrical objects are described in a bug library consulted by the analysis process. Any error is viewed as an incorrect application of a correct rule. Instead of considering the erroneous rules in the same way as the correct ones, they are defined as tranformational rules, that is meta-rules which produce an actual erroneous rule

Geometry problem solving 145

when applied to correct rules. Due to this formalism, the treatment of errors is more general and reduces the number of rules in the model. When the student tries a wrong inference, the system applies the converse of these meta-rules until it can produce a right inference, supposed to be the student’s intention. Then, the explanation associated with the successful meta-rule is displayed. If a single error cannot be recognized, the process looks for a combined error, for example a logical error together with a planning error.

Example

The pupil enters the following proof step S: Premise: ABC is an isosceles triangle Definition: if a triangle has two equal sides, then it is an isosceles triangle. Result: AB = AC

This inference does not belong to the valid set provided by the theorem prover. To detect the origin of error, the system applies the meta-rules on S. For example, meta-rule MR, “Replace the definition by its converse” produces the proof step MR,(S):

Premise: ABC is an isosceles triangle Definition: If a triangle is isosceles, then it has two equal sides. Result: AB = AC

As this new inference is right, it is assumed that it is what the child intended to do. So, the message associated with MR, is displayed: “Confusion between the definition and its converse”.

EVALUATION

Experimentations: a prototype, written in PrologII/MALI, was developed on a SUN station. A version also exists for IBM/PC and compatibles.

The theorem-prover has been tested on about 40 problems from school manuals, concerning the three parts of the curriculum: lines and circles, quadrilaterals, triangles. During 199&1991 the tutor has been used in a second school classroom of 30 pupils, during 20 sessions of 1 h. This experience contributed to the improvement of the interface and the interaction, the inventory of the pupils’ errors, and the validation of the plan recognition method. It pointed to the lack of advisory capabilities at the strategy level (how to imagine a plan?), and to the difficulty in designing such capabilities.

A second evaluation has taken place in 1991-1992. Its purpose was to estimate the pedagogical relevance of the tutor. The results are still being analysed, but some conclusions may be drawn. At the end of the experience, the pupils appeared to master better the structure of the proof, and the structure of the proof step (they distinguished more easily the premises from the conclusions). They asked the tutor less often for help, and tried to correct their errors by themselves, after reading carefully the error message. Moreover, they declared that working with the tutor was interesting, particularly because they could verify their assertions, and that they better understood geometry.

Comparison with other works: realizations of ITS in geometry are not numerous. The most famous is certainly Anderson’s Geometry Tutor[6]. It contains a set of rules (ideal and buggy rules) dealing with the domain, a communication interface for managing the interaction with the student and a tutor which supervises the whole process and guides the student.

Like Mentoniezh, Geometry Tutor allows a flexible proof (forward or backward chaining) and gives the student on-line help. Moreover, it explicitly sets out the proof tree, which is permanently on the screen, and which constitutes a strong means of reification for the pupil.

In contrast, Geometry Tutor is more directive. The theorem prover generates a single solution to an exercise: if the student uses a correct but unexpected strategy, the tutor will reject it. Furthermore, if the student tries three erroneous consecutive inferences, the tutor gives the next step; thus the student may reach the goal without understanding anything. The weakest point of Geometry Tutor is the model-tracing approach, which entails a local vision of the student model. In a domain like theorem-proving, the steps cannot be considered separately, and a global analysis of the student’s work is essential. Finally, some pedagogical aspects of Geometry Tutor give rise to sharp critiques from didacticians [7].

146 DOMINIQUE PY

A tutoring system for geometry problem-solving with a plan recognition method can perform student modelling. The result is a global analysis of the student’s performance, and explanations or advice more suited to the work.

The current prototype does not include all the features of an actual ITS. In particular, the student’s intended proof is only one facet of a student model, and the pedagogical model is mainly implicit, but it constitutes, for teachers and didacticians, an interesting tool for experimentation and analysis. For example, analysis of the successive plans a student tries during a session may reveal very rich information.

1.

2.

3. 4.

5.

6. I.

REFERENCES

Nicolas P.. Construction et verification de figures geometriques dans le systeme Mentoniezh These, LJniversite de Rennes I (19X9). Py D., Reconnaissance de plan pour I’atde a la demonstration dans un Tuteur Intelligent de la geometric. These. Gniversite de Rennes I (1990). Wenger E.. Arrificinl fn/e//igence md 7irroring Sb’.~I~m.s. Morgan Kaufmann. Calif. (1987). Kautr H. A.. A formal theory of plan recognition. Ph.D. thesis, Department of Computer Science. University of . Rochester (1987). Didactique et acquisition des connaissances scientiliques. Scienttfic report of GRECO didactique, CNRS (Autumn 1991). Anderson J. R., The geometry tutor. Prtx~. I.J.C.A.I.. Los Angeles, pp. I 7 (1985). Guin D. CI al.. Modelisation de la demonstration geometrique dans Geometry Tutor. In Annu1e.v dr Diducriyue ct de Sciewes Cognirircv, Vol. 3. pp. I 20. IREM de Strasbourg (1991).