a conceptual model for representing verbal …d.kettani/pub/verbalexpression.pdf · descriptions...

LANGUAGE AND SPACE, PATRICK OLIVIER EDITIONS, ENGLAND, 1998.

1

A Conceptual Model For Representing Verbal Expressions Used In Route Descriptions

AGNES GRYL1,2, BERNARD MOULIN1,3, DRISS KETTANI1,3

1 Centre de Recherche en Géomatique Pavillon Casault, Université Laval Québec, G1K 7P4, Canada

[email protected]

2 LIMSI-CNRS, Université Paris-Sud, BP 133, F-91403 Orsay Cédex, France

3 Département d’Informatique Pavillon Pouliot, Université Laval Québec, G1K 7P4,Canada {kettani / moulin}@ift.ulaval.ca

KEY-WORDS : route descriptions, representation of space, verbal expressions, spatial conceptual map ABSTRACT : In this chapter, we propose a new conceptual representation for handling spatial information. This representation takes into account the linguistic and cognitive constraints that we found during a study of natural language route descriptions. We concentrate on one of the main components of route descriptions: verbal expressions. First, we present a categorization of verbal expressions. Then, we introduce several concepts which are used to model the semantic content of verbal expressions found in route descriptions, especially the notions of object’s influence area and of displacement. We show how these concepts can be used to define a subset of the verbal expressions found in route descriptions. Finally, we present several issues related to the implementation of this approach for generating route descriptions using spatial conceptual maps and a simulation of a virtual pedestrian’s movements in these maps.

1. INTRODUCTION

During the past two decades, researchers have proposed models to represent and manipulate spatial information. We are interested in a specific application focusing on how people communicate about spatial information : route descriptions. A route description is essentially a narrative text in which sentences are prescriptions given by the speaker to an addressee: they describe a succession of actions that the addressee will have to carry out when s/he follows the route in the described environment. Our work aims at proposing tools for handling different kinds of information (spatial, linguistic and cognitive) found in route descriptions. Usually systems dealing with route descriptions fall into one of the following categories : systems acquiring spatial knowledge and systems dedicated to natural language descriptions of spatial scenes. For example, the Tour system [Kuipers, 1978] acquires spatial information by a simulated sequence of sensor inputs. It aims at simulating the execution of movements. This system mainly deals within a network structured environment. To follow a route consists in using a predetermined route or in concatenating parts of a predetermined route. There is no possibility to construct a new route, to optimize recorded routes or to use new environments. Traveller [Leiser & Zilbershatz, 1989] is based on a set of nodes connected by edges. A route is represented as a sequence of edges and nodes. The edges


2

represent the actions to be performed in order to go from one node to another. The nodes represent different points of the environment. The nodes and the edges which are the most used obtain a proeminent status and are memorized. Each time a new route is to be built, a width-first search is applied to the network. Here too, as in the Tour system, new routes are constructed on the basis of already known routes. The system does not have deductive capabilities allowing it to reorganize the memorized routes. Navigator [Gopal et al., 1989] introduces the notions of landmark and path. A network of two kinds of nodes represents the environment. High-level nodes represent decision points (points where navigation decisions are required) and edges represent the spatial information needed to move between two decision points. Low-level nodes represent the objects located at the decision points. The objects of the environment are described by a set of valued features. A route is composed of a sequence of decision points and is created in two steps. During the first step, relevant objects are recognized. During the second step, those objects are associated with the decisions points. This system is based on an environment simulated by a grid of streets. KEY-WORDS : route descriptions, representation of space, verbal expressions, spatial conceptual map ABSTRACT : In this chapter, we propose a new conceptual representation for handling spatial information. This representation takes into account the linguistic and cognitive constraints that we found during a study of natural language route descriptions. We concentrate on one of the main components of route descriptions: verbal expressions. First, we present a categorization of verbal expressions. Then, we introduce several concepts which are used to model the semantic content of verbal expressions found in route descriptions, especially the notions of object’s influence area and of displacement. We show how these concepts can be used to define a subset of the verbal expressions found in route descriptions. Finally, we present several issues related to the implementation of this approach for generating route descriptions using spatial conceptual maps and a simulation of a virtual pedestrian’s movements in these maps.

2. INTRODUCTION

During the past two decades, researchers have proposed models to represent and manipulate spatial information. We are interested in a specific application focusing on how people communicate about spatial information : route descriptions. A route description is essentially a narrative text in which sentences are prescriptions given by the speaker to an addressee: they describe a succession of actions that the addressee will have to carry out when s/he follows the route in the described environment. Our work aims at proposing tools for handling different kinds of information (spatial, linguistic and cognitive) found in route descriptions. Usually systems dealing with route descriptions fall into one of the following categories : systems acquiring spatial knowledge and systems dedicated to natural language descriptions of spatial scenes. For example, the Tour system [Kuipers, 1978] acquires spatial information by a simulated sequence of sensor inputs. It aims at simulating the execution of movements. This system mainly deals within a network structured environment. To follow a route consists in using a predetermined route or in concatenating parts of a predetermined route. There is no possibility to construct a new route, to optimize recorded routes or to use new environments. Traveller [Leiser & Zilbershatz, 1989] is based on a set of nodes connected by edges. A route is represented as a sequence of edges and nodes. The edges represent the actions to be performed in order to go from one node to another. The nodes represent


3

different points of the environment. The nodes and the edges which are the most used obtain a proeminent status and are memorized. Each time a new route is to be built, a width-first search is applied to the network. Here too, as in the Tour system, new routes are constructed on the basis of already known routes. The system does not have deductive capabilities allowing it to reorganize the memorized routes. Navigator [Gopal et al., 1989] introduces the notions of landmark and path. A network of two kinds of nodes represents the environment. High-level nodes represent decision points (points where navigation decisions are required) and edges represent the spatial information needed to move between two decision points. Low-level nodes represent the objects located at the decision points. The objects of the environment are described by a set of valued features. A route is composed of a sequence of decision points and is created in two steps. During the first step, relevant objects are recognized. During the second step, those objects are associated with the decisions points. This system is based on an environment simulated by a grid of streets. Finally, Plan [Chown et al., 1995] is based on an incremental process for building spatial knowledge. First, objects of the environment are recognized. Some of them are considered as landmarks when they satisfy the criteria of perceptual and functional distinctiveness. Second, a directional structure is defined. This structure provides the relative change of orientation needed to go from one landmark to another. Finally, a higher level representation integrates the information built during the first two stages. Thus, a hierarchical structure allows to take into account different levels of spatial information acquisition. Nevertheless, all these systems do not take into account linguistic aspects of route descriptions. This issue is partly addressed by systems which generate descriptions. The Vitra project [AndrÈ et al., 1987] aims at describing in natural language spatial relationships between objects within sequences of scenes. The choice of the spatial relation to be expressed relies on different criteria such as “ the degrees of applicability, compatibility, uniqueness and facility to be memorized ” [Blocher & Stopp, 1995]. The route description part of the project is inspired from Habel’s model and from the KOPW project. Habel [Habel, 1987] proposed an approach for route descriptions composed of three steps. The first step builds a path between the starting point and the ending point. The second step segments the path and extracts the relevant information for the route description. The choice of the route is totally disjoint from the natural language generation. The KOPW project [Hoeppner et al., 1987] mainly focus on the link between the route itself and the medium to express it easily. Thus, the authors proposed a model in which the object’s characteristics and the elements needed for their linguistic description share the same knowledge structures. These systems only deal with certain aspects of route descriptions. They do not consider route description as a complete task. Such a view would require the specification of a new integrative structure based on the study of person interactions. To this end, our approach exploits the results of the analysis of natural language corpora of route descriptions in order to define tools for handling spatial information used in route descriptions. In the GRAAD project1 we aim at developping a knowledge-based system which manipulates spatial and temporal knowledge while simulating the kind of behaviour that people adopt when describing a route.

1 GRAAD is a shuffle of the first letters of the following title: Artificial Agent for Generation and Description of Routes.


4

In Section 2 we present the linguistic data resulting from the analysis of our corpora and we emphasize the main features of route descriptions relevant to our study: verbal expressions. In Section 3 we introduce several concepts which are used to model the semantic content of verbal expressions found in route descriptions, especially the notions of object’s influence area and of displacement. In Section 4 we show how these concepts can be used to define a subset of the verbal expressions found in route descriptions. In Section 5 we present several issues related to the implementation of this model using the notion of spatial conceptual map.

3. ROUTE DESCRIPTIONS

A route description provides a set of instructions that a user can follow in order to reach a destination point. A route description system should generate natural language descriptions in a cognitively plausible way. Thus, the instructions given by the system might be close to those given by persons when describing routes. In order to identify the elements used by a group of subjects who described routes, Gryl [1995] collected and analyzed two corpora of pedestrian route descriptions written in French. Our present research aims at providing a general framework for handling spatial information and for generating route descriptions. Hence, we are interested in the following part of Gryl’s study. The author showed that natural language route descriptions are composed of two main components : landmarks which are elements of the considered environment and actions which are the instructions given to the pedestrian. These two components are expressed in natural language using nominal expressions - proper or common names or nominal propositions - and verbal expressions - complex or simple verbs or verbal propositions. In this paper, we focus on verbal expressions. Gryl proposed a categorization of the verbal expressions used in the corpora. This categorization is based on two levels of properties. The first level identifies four categories which are used to distinguish between the different kinds of instructions that the pedestrian has to follow at a specific point of the environment. The second level, which identifies sub-categories, characterizes the spatial features of the instruction. For example, the verbal expressions “ to walk by x ” and “ to go to x ” where x is a landmark of the environment are in the same category (first level) but in two distinct sub-categories (second level) because the verbal expressions evoke two different spatial relations between the pedestian and object x The first category of verbal expressions is onward move. Verbal expressions belonging to this category are used to express the continuation of a movement already initiated by the pedestrian in a given direction. The second category of verbal expressions is change of orientation. Verbal expressions belonging to this category are used to express a modification of the pedestrian previous direction. The third category of verbal expressions is individual localization. Verbal expressions belonging to this category are used to express the pedestrian’s position at a specific point of the environment. The last category of verbal expressions is referent localization and indicates the position of an object in the environment. Several properties characterize the sub-categories. These properties provide information about the spatial content of the verbal expressions. They characterize the spatial relations linking the objects (objects of the environment and/or pedestrian). While the categories specify the kind of instruction to follow, the


5

properties defining the sub-categories indicate the spatial relation in which the pedestrian has to be when following the instruction. In the following table (Table 1), we use examples2 to illustrate categories and sub-categories. category sub-category example onward move frontality to go straight ahead continuer tout droit goal to go to x aller jusqu’‡ x laterality to walk by x longer x passing to pass x dÈpasser x simplicity to keep going continuer change of orientation direction to turn to the left

to turn to the right tourner ‡ gauche tourner ‡ droite

medium to turn on x prendre x individual localization direction to be in front of x Ítre devant x goal to reach x arriver ‡ x simplicity to find x trouver x referent localization simplicity that is x c’est x

Table 1 - Instanciated properties of verbal expressions

The definitions below explain the specificity of each of these properties. direction: it corresponds to the expression of an action whose direction is detailed. This direction may be expressed relatively to a reference frame characterized by the axes left / right / in front of / behind. frontality: it corresponds to the expression of an action which is specified by using the front axis of the given reference frame. goal: it corresponds to the expression of an action which involves a referent (landmark) as a goal. This goal may be explicitly or implicitly specified and the referent can be reached or not at the end of the action. laterality: it corresponds to the expression of an action which is specified by mentionning a referent situated on one of the main axis (left/right). medium: it corresponds to the expression of an action which is specified by indicating the place where the action will take place. passing: it corresponds to the expression of an action in which the intrinsic reference frame of a referent is used. This orientation *** quelle orientation? *** is expressed in the action (in front of/behind/between).

2 All the examples are translations of French verbal expressions.


6

simplicity: it corresponds to the expression of an action without providing any additional spatial property. transversality: it corresponds to the expression of an action which is described relative to a referent specified by its extremities. In the following section, we present the concepts we propose in order to represent the instructions contained in the verbal expressions.

4. BASIC CONCEPTS

Route descriptions are composed of qualitative spatial relations. When they give instructions, human subjects use qualitative spatial relations of orientation, topology and distance such as “ to turn left ”, “ to be on x ”, “ to be near ”. In order to be able to handle these kinds of relations, we present a model for qualitative spatial representations. Consider an object O, the space surrounding this object (whatever the object’s shape is) is partitioned into what we call influence areas (IA) [Moulin, Gryl, Kettani 1997]. Given an object O of any shape, an influence area IA of O is a portion of space surrounding O such that: IA has two borders (an interior border and an exterior border); IA’s borders have the same shape as O’s border; if from any point Oi located on O’s border BO we draw a perpendicular line, this line crosses IA’s interior border at point IAIBi and IA’s exterior border at point IAEBi such as (∀ Oi ∈ BO) (dist(Oi,IAIBi) = c1 and dist(Oi,IAEBi) = c2 and c1>c2). The distance dist(IAIBi,IAEBi) is called the width of the influence area.

In Figure 1a and 1c we present examples of influence areas for a rectangle and an ellipse. In order to illustrate the preceding definition, we also show in Figure 1a the intersection points of a perpendicular line originating from the border of object O1 with the internal and external borders of the influence area denoted Next-to-O1.

Influence areas can be used to express a degree of proximity to an object. For example, in Figure 1 object 1 is represented with two kinds of influence areas: A “nearness” influence area named “next-to” (denoted NTO) and a “closeness” influence area “close-to” (denoted CTO).

Thus, we can define neighborhood relations, given the position Pos(Q) of a point Q in the spatial conceptual map:

A point Q is close to object O iff Pos(Q) ∈ CTO . A point Q is near an object O iff Pos(Q) ∈ NTO .

The size of influence areas depends on several factors such as the size of the object and the perception of the closeness relation that is possessed by an individual. As a simplification which is sufficient for our current purposes, we consider that all the objects in the SCM have influence areas which have the same proportion. How influence areas are cognitively established is a research topic that we are currently investigating.

Certain objects such as buildings possess an intrinsic reference frame, which is used to define specific orientations relative to the object; usually the front, back, right and left directions. Figure 1b illustrates


7

the basic orientations of a punctual object. These intrinsic orientations are used to express orientation relations between objects. For example “the backery is to the left of the city hall” or “VP is in front of the court building”. In the case of a non-punctual object, the intrinsic orientations can be used to partition the influence areas associated with this object. We have an example in Figure 1c. Object2’s intrinsic front orientation is symbolized by the bold arrow. The doted lines originate from the object’s faces and mark off the portions of space that can be defined by its intrinsic orientations (named front, back, left and right directions). Those lines partition the closeness and nearness influence areas into sub-areas3 that can be named as it is done in Figure 1c. Given an object O, an intrinsic orientation r associated with O creates a sub-area IAO,r within an influence area IA of O.

For example, in Figure 1, the closeness influence area of object O2 is partitionned into sub-areas denoted CTO2, Center-Front (named ‘Close Center Front’ in Figure 1), CTO2, Center-Back (named ‘Close Center Back’), CTO2, Front-Right (named ‘Close F-R’), etc. Similarly the nearness influence area of object O2 is partitionned into sub-areas denoted NTO2, k such as NTO2, Center-Front (named ‘Near Center Front’ in Figure 1c), etc.

Hence, we can express orientation relations in terms of topological relations involving influence areas. Given the position Pos (Q) of a point Q in the spatial conceptual map,

- A point Q is close to object O and in front of O

iff Pos (Q) ∈ CTO, front or if Pos (Q) ∈ CTO, Center-Front ∪ CTO, Front-Right ∪ CTO, Front-Left

- A point Q is near object O and in front of O

iff Pos (Q) ∈ NTO, front or if Pos (Q) ∈ NTO, Center-Front ∪ NTO, Front-Right ∪ NTO, Front-Left

- and by extension, a point Q is in front of O iff

(Pos (Q) ∈ CTO, Front) OR ( Pos (Q) ∈ NTO, Front)

Similar equations apply to the other influence sub-areas of an object O in relation to its intrinsic orientations. In the following sections we will consider a general notion of influence area IA of an object O denoted IA(O) = ∪x IAO,x where x ∈ {front, front-right, right, back-right, back, back-left, left, front-left}.

3 Let us comment upon how sub-areas are named. The first part of the name is the influence area’s name. For any intrinsic orientation we have three sub-areas: “center” which is between the lines delimiting a face of the object and two sub-areas which are at the intersection of two adjacent orientations. For instance, we have the sub-areas “Near-center-back” and “Near B-R” and “Near B-L”. We denote “Near-Back” the union of these sub-areas.


8

In order to handle route descriptions, we need to introduce different kinds of objects on the basis of this representation. A route description involves a user, the pedestrian who follows the instructions and that we call Virtual Pedestrian (VP). We do not want to exactly model the trajectory s/he is following but the successive actions s/he is supposed to do when following the instructions. From a spatial point of view, VP is considered as a punctual object with an intrinsic reference frame (the usual orientations of human beings). This frame of reference defines IAs denoted IA x(VP) where x ∈ {front, front-right, right, back-right, back, back-left, left, front-left}.VP moves in the environment. At a specific instant i, VP occupies a position denoted Posi(VP). This position is defined as a point belonging to a specific part of the space. VP may have other characteristics such as a name that we do no consider here. When following the instructions, VP moves along specific objects of the environment. These objects are defined by Lynch [1960] as “ ...the channels along which the observer customarily, occasionally, or


9

potentially moves. They may be streets, walkways, transit lines, canals, railroads. ”. We call them medium-objects(W). When describing a route, subjects refer to objects of the environment such as buildings, shops, etc. We call these objects Landmark-objects(L). Landmark-objects are usually located along medium-objects. They have sometime an intrinsic orientation which allows to attach to them a reference frame and to define IAs. The location of a landmark-object O1 may be defined with respect to another landmark-object O2 or to VP. Then, O1 occupies a specific position Pos(O1) into O2’s or VP’s reference frame. Medium-objects compose a network along which landmark-objects are distributed. By considering the intersections of different medium-objects as well as the intersections between IAs of landmark-objects and medium-objects4, we segment the medium-objects into way portions. Given a medium-object Wi, the way portions of Wi are denoted Sn(Wi) where n for n = 1 to p. Figure 2 illustrates two main cases of how we segment medium-objects5: a medium-object intersects an IA of a landmark-object; two medium-objects intersect.

Figure 1 - Segments of medium-objects

Each instruction corresponds to a part of the route. A verbal expression is used to describe an instruction and defines a movement of VP which takes place on a medium-object and may refer to landmark-objects. We define for each instruction a specific notion called displacement(D). D represents the part of the route along which VP has to move. D is defined by four elements: VP’s position at the beginning of the instruction; VP’s position after that the completion of the instruction; the medium-object W on which the instruction occurs; and the direction Dir in which VP moves (usually Dir corresponds to the front orientation of VP). D is denoted as follows : Di [Posj-1(VP), Posj (VP), Dirk, Wl, VP)] In order to handle the different objects that we have just defined, we need to introduce two functions : F-ORIENTx(O) and F-DIR(Pos(O1), O2). We specify the orientation of an object using the orientation function F-ORIENTx(O) such as x ∈ {front, front-right, right, back-right, back, back-left, left, front-left}. This function determines the orientation of an object as for example F-ORIENTleft(VP) which

4 An intersection is denoted INT(Wi, Wj) when it applies to medium-objects Wi and Wj and INT(O, Wi) when it applies to a landmark-object O and a medium-object Wi. 5 Other cases have to be taken into account when defining a complete representation of an environment. The complete definition of these cases is out of the scope of this paper. But see [Moulin, Gryl, Kettani 1997].

Wi

Wj

O

portions of Wi

INT(O, Wi) INT(Wi, Wj)

portions of Wj


10

returns the IA corresponding to the left of VP. This function can also be used to assign a specific orientation to an object as for example F-ORIENTleft(VP) = var where var is an external orientation. The direction function F-DIR(Pos(O1), O2) is a complex function which gives the direction of object O2 with respect to object O1 positioned within the current medium-object. For example, this function may be used to define the direction in which VP has to move on the current medium-object in order to move toward a landmark-object. We also introduce a specific denotation for the axes of intrinsic reference frames. Thus considering an object O with an intrinsic reference frame, the axes of this reference frame are denoted Left(O), Right(O), Front(O) and Back(O). These basic concepts allow us to propose a represention of verbal expressions used in route descriptions. We show in the following section how we foresee to use these representations to generate natural language route descriptions in a similar way as human beings do.

5. REPRESENTATION OF VERBAL EXPRESSIONS

The verbal expressions which are used to express instructions in route descriptions, contain much more than the simple statement of a spatial relation. They express a displacement, thus a change of VP’s position which occurs when certain conditions are satisfied. In this section, we propose a conceptual representation of the displacement itself and of the associated conditions. Three elements compose the conceptual representation of verbal expressions : prerequisites, implications, constraints. Prerequisites correspond to conditions that characterize an object’s movement or position just before applying the verbal expression. Implications correspond to the meaning of the verbal expression. That is the conceptual representation of the verbal expression. Constraints correspond to the conditions derived from the meaning of the verbal expression, that is, the conditions to satisfy in order to be able to apply the constraints. Below we present the conceptual representation used to represent the verbal expressions listed in section 2. For each verbal expression, we explain the content of the verbal expression and propose a representation based on the basic concepts. ONWARD MOVE frontality - to go straight ahead This verbal expression indicates that VP is on a medium-object and must go on moving on this medium-object. Thus, VP already moved from a previous position Posj-1(VP) to the current position Posj (VP) on medium object Wl

6 in the direction Dirk. The representation of the verbal expression “ to go straight

6 When VP occupies a position Posj (VP) on medium object Wl, Posj (VP) ∈ Sn(Wl).


11

ahead ” indicates the creation of a new displacement from the current position Posj (VP) to a new position Posj+1(VP) on the same medium-object Wl and in VP’s front direction. ⟨ prerequisites : Di [Posj-1(VP), Posj (VP), Dirk, Wl, VP)] ⟨ implications : Di+1 [PosjVP), Posj+1(VP), Dir k+1, Wl, VP)] ⟨ constraints : Posj+1(VP) ∈ Sn+1(Wl), Sn+1(Wl) <> Sn(Wl) and Dir k+1 = Front(VP). Remarks : Posj+1(VP) may be determined by a specific configuration of the environment, a specific segment of the medium-object such as a crossing between two medium-objects or a decision-point (see Figure 2 and [Gryl, 1995]). The exact direction of the considered medium-object is not important. It may be curved even if, as simplification, we draw it as a straight line. We present in Figure 3 the pictorial representation for this verbal expression. *** Agnes, dans toutes tes figures il faut que tu changes PV par VP Il faut aussi que tu changes les numéros des figures pour “to go straight” et “to go until”: elles devraient êre numérotées 3 et 4 ***

Figure 2 - “ to go straight ahead ” pictorial representation

goal - to go to x This verbal expression indicates that VP is on a medium-object and has to go on moving on this medium-object until reaching object O (O is the landmark-object designated by x in natural language).. Thus, VP already moved from a previous position Posj-1(VP) to the current position Posj (VP) on medium object Wl in the direction Dirk. The representation of the verbal expression “ to go to x ” indicates the creation of a new displacement from the current position Posj (VP) to a new position Posj+1(VP). Posj+1(VP) is in the area which intersects one influence area of the object O and the considered medium-object Wl. The displacement occurs in the direction of O (O is the landmark-object designated by x in natural language). ⟨ prerequisites : Di [Posj-1(VP), Posj (VP), Dirk, Wl, VP)]. ⟨ implications : Dirk+1 = F-DIR(Posj (VP), O) Di+1 [Posj(VP), Posj+1(VP), Dirk+1, Wl, VP)] Posj+1(VP) ∈ IA(O) ∩ Wl ⟨ constraints : x designates the landmark-object O, Posj+1(VP) ∈ Sn+1(Wl), Sn+1(Wl) <> Sn(Wl),

PV’s intrinsic reference frame front

right

Posj-1(PV) Posj(PV) Posj+1(PV) Wl Dir

Dirk

Di Di+1


12

the intersection of one of the IAs of O and Wl exists. Remark : the object’s reference frame is not relevant to represent this category of verbal expression. We present the pictorial representation of this verbal expression in Figure 4.


13

Figure 3 - “ to go until x ” pictorial representation

laterality - to walk by x This verbal expression indicates that VP is on a medium-object and must go on moving and leave object O (O is the landmark-object designated by x in natural language) on his/her left or right. Thus, VP already moved from a previous position Posj-1(VP) to the current position Posj (VP) on medium object Wl in the direction Dirk. The representation of the verbal expression “ to walk by x ” indicates the creation of a new displacement from the current position Posj (VP) to a new position Posj+1(VP). Posj+1(VP) is in the area which intersects one influence area of the object O and the considered medium-object Wl. prerequisites : Di [Posj-1(VP), Posj (VP), Dirk, Wl, VP)]. implications : Dirk+1 = F-DIR(Posj (VP), O) Di+1 [Posj(VP), Posj+1(VP), Dirk+1, Wl, VP)] Posj+1(VP) ∈ IA(O) ∩ Wl constraints : x designates a landmark-object of the environment, Posj+1(VP) ∈ Sn+1(Wl), Sn+1(Wl) <> Sn(Wl), the intersection of one of the IAs of O and the considered medium-object exists. Pos(O) ∈ IArignt(VP) or Pos(O) ∈ IAleft(VP) Remark : the object’s reference frame is not relevant to represent this category of verbal expression. simplicity - to keep going This verbal expression indicates that VP is on a medium-object and must go on moving on this medium-object. Thus, VP already moved from a previous position Posj-1(VP) to the current position Posj (VP) on medium object Wl in the direction Dirk. The representation of the verbal expression “ to keep going ” indicates the creation of a new displacement from the current position Posj (VP) to a new position Posj+1(VP) on the same medium-object Wl and in the same direction

7.

7 We distinguish the verbal expressions “ to go straight ahead ” and “ to keep going ” by the explicit mention in the first one of the displacement direction.

front

right

Posj1(PV) Posj(PV) Posj+1(PV) Wl Dir

Dirk+1

Di Di+1

PV’s intrinsic reference frame

O


14

prerequisites : Di [Posj-1(VP), Posj (VP), Dirk, Wl, VP)]. implications : Di+1 [Posj(VP), Posj+1(VP), Dirk, Wl, VP)] constraints : Posj+1(VP) ∈ Sn+1(Wl), Sn+1(Wl) <> Sn(Wl), CHANGE OF ORIENTATION direction - to turn to the left This verbal expression indicates that VP is on a medium-object Wl

8 and must go to the first intersection of the medium-object Wl and another medium-object Wm situated on the left of VP. This suggests the creation of new displacements Di and Di+1 from Posj-1(VP) to a new position Posj+1(VP). The first displacement Di occurs on the medium-object Wl until this medium-object meets Wm. Then, VP is reoriented and a second displacement Di+1 is created on Wm. prerequisites : Posj+1(VP) ∈ Sn(Wl) implications : Di [Posj-1(VP), Posj(VP), Dirk, Wl, VP)] Dirk+1 = F-ORIENTleft(VP) Front(VP) = Dir k+1 Di+1 [Posj(VP), Posj+1(VP), Dirk+1, Wm, VP)] constraints : the medium object Wm should exist, Pos(Wm) ∈ IAleft(VP) when VP is in Posj.

Figure 5- “ to turn to the left ” pictorial representation

8 This prerequisite allows to characterize the change of orientation verbal expressions with respect to the onward move verbal expressions. In the latter, the prerequisites apply to the displacement.

front

right

Posj-1(PV) Posji(PV)

Posi+1(PV)

Wl Dir

Dirk+1

Di

Di+1 PV’s intrinsic reference frame

Wm

front right

new PV’s intrinsic reference frame


15

Remark : “ to turn to the right ” implies the same definition with a change in Pos(Wi+1) ∈ IAright(VP) before VP moves on Wm. medium - to go down x This verbal expression indicates that VP is on a medium-object Wl and must move until meeting a new medium-object Wm designated by x in the verbal expression by x. This suggests the creation of new displacements Di and Di+1 from Posj-1(VP) to a new position Posj+1(VP). The first displacement Di. occurs on the medium-object Wl until this medium-object meets medium-object Wm situated in one of the VP’s IAs9. Then, VP is reoriented and a second displacement Di+1 is created on Wm. prerequisites : Posj+1(VP) ∈ Sn(Wl) implications : Di [Posj-1(VP), Posj(VP), Dirk, Wl, VP)] Front(VP) = F-DIR(Posj (VP), Wm) Di+1 [Posj(VP), Posj+1(VP), Dirk+1, Wm, VP)] constraints : the medium object Wm should exist, Pos(Wm) ∈ IA(VP) when VP is in Posj. INDIVIDUAL LOCALIZATION direction - to be in front of x This verbal expression indicates that VP is on the medium-object Wl and that VP’s position must be in IAfront(O) (if O is the object designated by x) and on the medium-object Wl. prerequisites : Posj(VP) ∈ Sn(Wl) implications : Posj(VP) ∈ IAfront (O) ∩ Wl constraints :O should have an intrinsic reference frame

Figure 6- “ to be in front of x ” pictorial representation

goal - to reach x This verbal expression indicates that VP is somewhere on the medium-object Wl and VP’s position must be in one of the object’s IAs designated by x and on the medium-object Wl.

9 We distinguish the verbal expressions “ to turn to the left ” and “ to go down x” because in the first expression the position of the medium-object (on VP’s left) is explicitly mentioned.

Wl

O right

front

Posji(PV)


16

prerequisites : Posj-1(VP) ∈ Sn(Wl) implications : Posj(VP) ∈ IA(O) ∩ Wl constraints : to be defined if necessary simplicity - to find x This verbal expression indicates that VP is somewhere on the medium-object Wl and the object position must be in one of the VP’s IAs. prerequisites : Posj(VP). ∈ S(Wl) implications : Pos(O) ∈ IA(VP) ∩ Wl constraints : to be defined if necessary In the proposed approach, the transformations applied on displacements and object’s positions are typical of the category of the corresponding verbal expression. Thus onward-move verbal expressions imply a process creating a displacement in the same direction, change of orientation verbal expressions imply a process creating two displacements in two distinct directions, localization verbal expressions imply a process testing an object’s position. Moreover, based on the notion of displacement, we can represent a complete route. The sequence of displacements defined by the sequence of VP’s positions represents the route itself. As the displacement represents an instruction expressed by the way of a verbal expression, it is possible to find and to describe the sequence of instructions to follow and consequently to generate a route description.

6. IMPLEMENTATION ISSUES

In order to implement these notions we introduced the notion of spatial conceptual map [Moulin, Gryl, Kettani 1997]. A spatial conceptual map (SCM) is an analogical model that consists of a network of medium objects and a collection of landmark objects which are spatially localized on the map. A SCM can be thought of as a simulation of spatial cognitive maps used by people10.

In the GRAAD system a graphical editor enables a user to draw on the screen the spatial layout of a SCM, including medium and landmark objects. The SCM manager is a piece of software providing a function that tests whether a query point Q is contained in any area A of the SCM. Taking advantage of that analogical property of the GRAAD system, we defined functions that simulate spatial relations between objects (neighborhood, topological and orientation relations), thanks to the notion of influence area (Section 3).

A SCM contains the elements necessary and sufficient to generate and describe useful routes for human users. It is used by the GRAAD system in the same way as a spatial mental model is used by a human user in order to carry out qualitative spatial reasoning rather than precise quantitative spatial calculus.

10 Several studies [Lynch 1960; Tversky 1993; Timpf et al. 1992] showed that most people use some kind of mental model of a region or city part in order to generate and describe a route: they mentally visualize the salient elements characterizing the way that they want to describe.


17

Landmark objects and medium objects are positioned in the SCM in a way that respects the layout of the corresponding geographical map: the relative positions of objects are preserved, but distances may not be completely accurate. This is cognitively sound since human beings are better at reasoning qualitatively on spatial information. In addition to the relative spatial positions of landmark and medium objects, a SCM contains the influence areas of these different objects as well as specific information such as allowed traffic directions on ways and front orientations of landmark objects. In an associated data base we record complementary non spatial information such as shapes and colors of objects, social usage and any other information relevant for a route description.

The SCM manager provides different kinds of functions that apply on the elements contained in the SCM. A function determines the area which is the intersection of any set of areas contained in the SCM. Another function determines which areas contain a given point. A virtual pedestrian VP is represented by a point associated with a front orientation. The SCM manager also provides functions to control VP’s movements and changes of orientation in the SCM. Hence, the SCM provides a dynamic model that can be used to simulate the movements of the virtual pedestrian in a simplified urban environment and to generate a route description accordingly.

As we mentioned earlier, route descriptions generated by human subjects are essentially composed of local descriptions and path descriptions. A route from a point A to a point B is a path composed of a succession of way segments. In this paper we do not address the problem of route generation, but see [Moulin, Kettani 1998]. We assume that a user or a specialized module of the GRAAD system has already determined a cognitively plausible route that can be expressed in terms of local views and path descriptions. Since VP’s movements follow the way segments composing a route, it is natural to try to characterize the portions of ways (called “way elementary areas” or WEAs) to which we can apply the expressions found in human route descriptions: expressions of onward moves, orientation changes, description of VP’s localization and local descriptions. In fact, most of these descriptions match with specific way portions in a SCM.

We identified several cases that we summarize here (for more detail see [Moulin, Gryl, Kettani 1997]). A WEA is a portion of a way which has one of the following properties: 1) it is the intersection of an object influence area with the way; 2) it is an intersection between two ways; 3) it is a complex intersection between several objects’ influence areas that overlap on the way; 4) it is a straight unremarkable segment on the way and it is adjacent to WEAs that are defined using one of the preceding 3 cases.

In order to model VP’s movements, we need to specify the temporal and spatial characteristics of its trajectory. In a route description this trajectory is composed of a succession of points which are deemed necessary by the route generator with respect to cognitive constraints (easiness for the generator to explain the characteristics of the route, easiness for the addressee to understand those characteristics). Hence, a route RA,B is associated with a sequence of relevant instants {t1, t2, ..., tc-1, tc, tc+1, ..., tnab} at which a local or path description is provided. Obviously, the time intervals [ti, tj] need not be equal.

VP’s position in the spatial conceptual map is time dependent and is denoted Pos(VP, tc ) , where tc is a time stamp identifying the time at which the position has been plotted on the trajectory. The virtual pedestrian is also associated with an intrinsic reference frame providing its front orientation which is denoted Orient (VP, tc ), where tc is a time stamp identifying the time at which the corresponding position and orientation have been plotted on the trajectory.


18

Based on the analysis of verbal expressions in terms of displacements (Section 4), we identified the equations of VP’s movements in the spatial conceptual map. We show here how some of these verbal expressions can be specified in terms of the functions Pos(VP, tc ) and Orient (VP, tc ).

Given a sequence of relevant instants {t1, ..., tnab} used to describe relevant portions of a route RA,B which is composed of a succession of route segments RA,B[k] for k =1 to p, we can specify VP’s movements using verbal expressions. The following table11 presents the equations of VP’s position and orientation for some of these expressions.

11 Note that we use the following conventions. ORIENT (Wi[x],k) represents the orientation of route portion Wi[x] in the direction k of way Wi. A route can be associated intrinsically with two opposite directions: we assume that the succession Wi[x], Wi[x+1], ..., Wi[x+n] defines the direction denoted Orient(Wi[x],+1) and that the succession Wi[x], Wi[x-1], ..., Wi[x-n] defines the direction denoted Orient(Wi[x],-1). CTOc denotes the closeness influence area of object Oc. CTOc,z denotes a sub-area of the closeness influence area of object Oc which characterizes orientation z in the intrinsic reference frame associated with object Oc. INCO denotes the interior area of a crossable object CO.

nb Verbal expression VP’s positions VP’s orientations

1 to keep going previous: Pos(VP, tc-1) ∈ Wi[x]

current: Pos(VP, tc) ∈ Wi[x+n]

n is a positive or negative or null integer

previous: Orient(VP,tc-1) = ORIENT(Wi[x],k)

current: Orient(VP, tc) = ORIENT (Wi[x+n],k)

n is a positive or negative or null integer

to go to Oc current: Pos(VP, tc) ∈ Wi[x ]

future: (∃ n) (∃ j) (∃ y)

CTOc ∩ W j[y] ≠ ∅ AND

Pos(VP,tc+n) ∈ (CTOc ∩ Wj[y])

current:

Orient(VP, tc) = ORIENT (W i[x], k1)

future:

Orient(VP, tc+n) = ORIENT (W j[y], k2)

3 to cross way Wj previous: Pos(VP, tc-1) ∈ Wi[x-e]

with e = +1 OR e = -1

current:

Pos(VP, tc) ∈ Wi[x] ∩ Wj[y]

next: Pos(VP, tc+1) ∈ W i[x + e]

previous: Orient(VP, tc-1) =

ORIENT (W i[x-e], k) with e = +1 OR e = -1

current: Orient(VP, tc) = ORIENT (W i[x], k)

next:

Orient(VP, tc+1) = ORIENT (W i[x + e], k)

4 to turn on way Wj current:

VP is at intersection of ways i and j

Pos(VP, tc) ∈ Wi[x] ∩ Wj[y]

next: Pos(VP, tc+1) ∈ W j[y + e]

with e = +1 OR e = -1

current: Orient(VP, tc) = ORIENT (Wi[x], k1)

next:

Orient(VP, tc+1) = ORIENT (Wj[y + e], k2)

with e = +1 OR e = -1


19

5 to reach Oc current: Pos(VP, tc) ∈ CTOc current: Orient(VP, tc) = Orient(VP, tc-1)

6 to be in front of Oc

current: Pos(VP, tc) ∈ CTOc,front

current: Orient(VP, tc) unspecified

7 to be on CO current: Pos(VP, tc) ∈ INCO current: Orient(VP, tc) unspecified


20

Note that this table only gives a sample of the various verbal expressions that can be used to specify VP’s movements. Let us comment upon them briefly. Case 1 is an example of an onward move: in the previous position (at tc-1) VP is on way segment Wi[x] with the orientation Orient(Wi[x], k) and in the current position (at tc) VP is on a subsequent segment of Wi in the same direction k: if n= 0, VP is on the same Wi[x] ; if n> 0, VP is on a segment Wi[x+n] in direction k= +1; if n< 0, VP is on a segment Wi[x-n] in direction k= -1. Case 2 corresponds to an onward move with the goal of reaching a landmark object Oc: there exists a future position (at tc+n) where VP will be at the intersection of a way portion Wj[y] and CTOc, the closeness influence area of Oc. In case 3 VP crosses the intersection between way Wi and way Wj without changing its orientation: Orient(VP, tc) = Orient(W i[x], k): crossing the intersection is indicated VP’s position changes from Wi[x - e] to Wi[x ] and Wi[x + e], with e = +1 or e = -1. In case 4 we have an orientation change where VP is at the intersection of ways Wi and Wj and changes its orientation in order to follow Wj on its portion denoted Wj[y + e], with e = +1 or e = -1. In case 5 we have an individual localization where VP’s current position (at tc) is in the closeness influence area of landmark object Oc, with the same orientation it had previously (at tc-1). In case 6 VP’s current position (at tc) is in the “front” sub-area of the closeness influence area of landmark object Oc with the same orientation it had previously (at tc-1). In case 7 VP is in the interior area of a crossable object CO.

As an illustration, let us consider a portion of the city of Paris in which Gryl [1995] conducted the experiments of route generation with human subjects. Figure 7a presents a portion of Paris’ spatial conceptual map which emphasizes Danton street (Rue Danton) and its intersections with other streets (rue des Poitevins, rue Serpente, rue Mignon, Boulevard St Germain) with a crossable object (Place St André des Arts) and with the closeness influence area of a landmark object (Centre Henri Piéron). Figure 7b displays the nouns of the various portions W Danton [k] of Danton Street. For instance, INT (Danton, Piéron, Serpente) denotes the intersection of Danton Street with Serpente Street and the closeness influence area of Centre Henri Piéron. The name of intermediate unremarquable segments is built from the name of the street and the first letters of the intersecting ways. For instance, Danton-PS stands for Danton-Poitevins-Serpente.

Figure 8 is the same portion of SCM as in Figure 7b where the little circles materialize the virtual pedestrian’s trajectory along several positions which have been identified by the time stamps ti.

Here are the position and orientation formulae for each of these points and the corresponding verbal expressions in natural language.

at t0 : Pos(VP, t0) ∈ INPlace-St-André Orient(VP, t0) You are on St André Square

at t1 : Pos(VP, t1) ∈ WDanton [AP]

Orient(VP, t1) = ORIENT(WDanton [AP] , 1) Follow Danton Street

at t2 : Pos(VP, t2) ∈ WDanton [PO] ∩ WPoitevins [DA]

Orient(VP, t2) = ORIENT (WDanton [PO] , 1) Cross Poitevins Street


21

at t3 : Pos(VP, t3) ∈ WDanton [SE] ∩ WSerpente [DA]

Orient(VP, t3) = ORIENT (WDanton [SE], 1) Cross Serpente Street

at t4 : Pos(VP, t4) ∈ CTCentre-Henri-Piéron,front

Orient(VP, t3) = ORIENT (WDanton [SE] , 1) You are in front of Centre Henri Piéron

at t5 : Pos(VP, t5) ∈ WDanton [MI] ∩ WMignon [DA]

Orient(VP, t5) = Orient(WMignon [DA] , 1) Turn on Mignon Street

at t6 : Pos(VP, t6) ∈ WMignon [DS]

Orient(VP, t1) = ORIENT (WMignon [DS] , 1) Follow Mignon Street


22

Boulevard St Germain

Rue Danton

Centre Henri Piéron

Rue Serpente

Rue des Poitevins

M

Crossable object: Place St André des Arts

M Rue Danton

Magasin Gibert

Fontaine St Michel

Landmark object Closeness influence area

Intersection of ways or of way and crossable object Subway station's closeness influence area

M

Figure 7: Danton street and its intersections in a portion of Paris conceptual map

a

Legend

INT (Danton, St André)

INT(Danton, Serpente)

INT(Danton, Poitevins)INT(Danton, St Germain)

INT(Danton, Piéron)

INT(Danton, Piéron, Mignon)

INT(Danton, Mignon)

INT(Danton, Piéron, Serpente)

Danton-APDanton-PS Danton-MS

b

Place St André des Arts

Rue Mignon


23

7. CONCLUSION

We proposed a set of simple concepts to represent and to handle spatial information in a cognitively plausible manner. Based on a study of the verbal expressions contained in natural language route descriptions, we proposed basic concepts to represent and to implement the elements used by human beings when communicating about space. This representation is based on the notion of influence area. Influence areas seem to provide a very convenient representation for handling qualitative spatial relations, representing the elementary components of route descriptions and natural language expressions. We identified a specific representation for each category of verbal expression. We showed how to use this representation to simulate the movements of a virtual pedestrian in a spatial conceptual map based on a partition of the ways on which it can move. Each way is partitioned in elementary areas that are either the intersection of that way with other ways, or the intersection with landmark objects’ influence areas, or even unremarkable way segments. We showed that each of these categories of way elementary areas can be described using a verbal expression taken from the collection of expressions used by human subjects. Hence, the simulation of VP’s movements in the SCM provide a sound basis for generating route descriptions in a cognitive plausible way. The GRAAD system is under development. The SCM manager is fully implemented. The simulation of VP’s movements is used to generate route descriptions similar to the example of Section 5. We currently work on the refinement of these natural language descriptions which would enable the GRAAD Sytem to generate more sophisticated descriptions such as:

“You are on St André Square. Follow Danton street. You will cross Poitevins street and Serpente street. At that time, you will be in front of Henri Piéron Center. Turn on Mignon street”.

We are also currently working on the implementation of a route generator [Moulin, Kettani 1998]. When all these modules will be completed and integrated we will conduct experiments with human subjects in order to compare route descriptions generated by people and by the GRAAD System. The goal is to evaluate the quality of route descriptions generated by the GRAAD System and consequently the cognitive plausibility of our approach.


24

Acknowledgments This research is supported by the Canadian Natural Sciences and Engineering Research Council. This project has also been funded under the auspices of a joint France-QuÈbec grant in the framework of the program “ Autoroute de l’information ”. A. Gryl was funded under a Bourse d’Excellence offered by the QuÈbec Ministry of Education. REFERENCES [AndrÈ et al., 1987] ANDR… E., BOSCH G., HERZOG G., RIST T., 1987, “ Coping with the Intrinsic and Deictic Uses of Spatial Prepositions ”, in Proceedings of the Second International Conference on Artificial Intelligence, Varna, Bulgarie, pp 375-382 [Blocher & Stopp, 1995] BLOCHER A., STOPP E., 1995, “ Time-dependent Generation of Minimal Sets of Spatial Descriptions ” in Proceedings of Workshop on Representation and Processing of Spatial Expressions, IJCAI’95, MontrÈal, Canada, pp 1-7 [Chown et al., 1995] CHOWN E., KAPLAN S. KORTENKAMP D., 1995, “ Prototypes, Location and Associative Networks (PLAN) : Towards a Unified Theory of Cognitive Mapping ”, in Cognitive Science 19, pp 1-51 [Frank et al. 1992] FRANK A. U. , CAMPARI I. , FORMENTINI U. (edts.), 1992, Theories of Spatio-Temporal Reasoning in Geographic Space, Berlin: Springer Verlag

[Gopal et al., 1989] GOPAL S., KLATZKY R.L., SMITH T.R., 1989, “ Navigator: a psychologically based model of environmental learning through navigation ”, Journal of Environmental Psychology, 9, pp 309-331 [Gryl, 1995].GRYL A., 1995, “ Analyse et modÈlisation des processus discursifs mis en úuvre dans la description d’itinÈraires ”, PhD Thesis, University Paris XI-Orsay, Notes et Documents LIMSI 95-30. [Gryl et al., 1996] GRYL A., LIGOZAT G., EDWARDS G., 1996, “ Spatial and temporal elements of route descriptions ” in Proceedings of American Association for Artificial Intelligence, Workshop on Spatial and Temporal reasoning (AAAI-96 - Portland), pp33-38 [Habel, 1987] HABEL C., 1987, “ Prozedurale Aspekte der Wegplannung und Wegbeschreibung ”, in LILOG-Report 17 also published in Sprache in Mensch und Computer, Schnelle H. & Rickheit G. (Eds), Westdeutscher Verlag, 1988, pp 107-133 [Hoeppner et al., 1987] HOEPPNER W., CARSTENSEN M. RHEIN U., 1987, “ Wegausk¸nfte : Die Interdependenz von Such- und Beschreibungprozessen ”, in Repr‰sentation und Verarbeitung r‰umlichen Wissens, C.Freksa & C. Habel (Eds.), Springer Verlag, pp 221-234


25

[Kuipers, 1978] KUIPERS B. 1978 “ Modeling spatial knowledge ” in Cognitive Science 2, pp 129-153 [Leiser & Zilbershatz, 1989] LEISER D., ZILBERSHATZ A., 1989, “ THE TRAVELLER : A Computational Model Of Spatial Network Learning ”, in Environment and Behavior, 21 (4), pp 435-463 [Lynch, 1960] LYNCH K., 1960, “ The Image of The City, MIT Press. [Moulin, Gryl, Kettani 1997] MOULIN B, GRYL A., KETTANI D., 1997, “Route descriptions based on the notions of spatial conceptual map and object’s influence areas”, in proceddings of the AAAI Workshop on Temporal and Spatial Reasoning, Providence, Rhode Island, July 1997, 95-104 [Moulin, Kettani 1998] MOULIN B, KETTANI D., 1998, “Combining a logical and an analogical framework for route generation and description”, Proceedings of the 5th International Symposium on Artificial Intelligence and Mathematics (available on line at http:/rutcor.rutgers.edu/~amai/ ), Fort Lauderdale, Florida, January 1998.

[Timpf et al. 1992] TIMPF S., VOLTA G. S., POLLOCK D. W., EGENHOFER M. J., 1992, “A conceptual model of wayfinding using multiple levels of abstraction”, In (Frank et al. 1992) : 349-367

[Tversky 1993] TVERSKY B., 1993, “Cognitive maps, cognitive collages and spatial mental models”, In A. U. Frank and I. Campari (edts.), Spatial Information Theory: A Theoretical Basis for GIS, Springer Verlag: 14-24