exception handling system for autonomous robots based on pes

Robotics and Autonomous Systems 7 (1991) 197-209 197 North-Holland

Exception handling system for autonomous robots based on PES

G.R. Meijer and L.O. Hertzberger Computer Science Department (FWI), University of Amsterdam, 1098 SJ Amsterdam, The Netherlands

T.L. Mai, E. Gaussens and F. Arlabosse D~partement de Recherche & D$velopement, FRAMENTEC S.A., Tour Fiat, 92084 Paris La D~fense C$dex 16, France

Abstract

Meijer, G.R., L.O. Hertzberger, T.L. Mal, E. Gaussens, and F. Arlabosse, Exception handling system for autonomous robots based on PES, Robotics and Autonomous Systems 7 (1991) 197-209.

To reach autonomy of a robot system, the robot needs to be able to adapt itself to the changing environment. One of the key capabilities required for such a system is exception handling. We realized exception handling mechanisms for autonomous robots with a Procedural Expert System (PES). The work described in this paper is based on the Exception Handling Model (EHM), a model describing the handling of exceptions within the framework of an a priori plan. Exception handling provides the robot with capabilities to monitor its operations. Upon the detection of an exception, a diagnosis can be performed which leads to a classification of the exception at hand. Task rescheduling and recovery planning is performed based on heuristic knowledge of the application. Modelling of a robot assembly application and the realization of EHM with PES are described in the paper.

Keywords: Exception handling; Autonomous robots; Reactive systems; Procedural expert system; Planning; Knowledge-based control.

1. Introduct ion

In this paper the realization of a system for

diagnosis and hand l ing of exceptions is presented. One of the key capabili t ies required by an autonomous system is the abil i ty to manoeuvre in a

complex and changing envi ronment . For this purpose techniques are needed to moni to r the robot activities and to offer mechanisms for failure re-

covery [6,11,14]. It is our view that these moni to r and recovery techniques have to be an integral par t of the robot control process, thus inter leaving p l ann ing and p lan execution [7,12,17].

In general, the robot control system operates by using an in terna l representa t ion of the robot en- v i ronment . This in te rna l model is of ten specified dur ing an off-line p l ann ing and p rogramming

phase. A n impor t an t prerequisi te for recovery

Geleyn R. Meijer recieved his masters degree Physics from the University of Amsterdam in 1985. From 1985 to March 1990 he was research engineer and local project manager of Esprit I project 623. Currently he is working on planning tools for industrial CIM systems and is local project manager of Esprit II project 2202 CIM-PLATO. Since 1988 he is involved in industrial contract negotiations and member of the project control committee of Esprit II project 2003 MARIE. During 1989

he was member of the organizing committee of the IAS-2 conference. His interest is the development of robot control systems capable of handling exceptions in industrial environments. The research work comprises the development of sensor systems and control architectures using knowledge based techniques. He is author/coantber of various international pub- .lications on exception handling and robot control systems and is a member of the CIM Europ interest group on manufacturing systems. He is preparing a thesis on exception handling for autonomous robots.

0921-8830/91/$03.50 © 1991 - Elsevier Science Publishers B.V. (North-Holland)

198 G.R. Meijer, L.O. Hertzberger, T.L. Mai, E. Gaussens, F. Arlabosse

planning is to update the internal model representation of the robot environment after the detection of an exception. This diagnosis process tries to infer the cause of the exception and determines the consequences for the internal environment model [19].

Following an adequate diagnosis of the exception at hand, a recovery is attempted. The recovery process plans actions for the robot to execute in order to restore the environment conditions to such an extend that execution of the initial plan can be continued. In this case only a partial recovery plan is needed. If the recovery planner is not able to restore the environment, the initial plan is omitted. A new plan for realizing the task of the robot is generated, based on the new environment conditions.

To realize these recovery schemes two difficul-

Louis O. Hertzberger was born in 1941 in Hilversum, The Netherlands. He received his masters degree in experi- mental physics in 1969 and his PhD in 1975, both from the University of Amsterdam. From 1969 till 1983 he was a staff member in the High En- ergy Physics group, later the NIKHEF-H (Dutch Institute for Nuclear and High Energy Physics). He was the leader of a group building driftchambers and of a group building a multiprocessor system FAMP. He

was as an instrumental physicist involved in a number of bubblechamber and counter experiments mostly at CERN, Geneva but partly at DESY, Hamburg and at Brookhaven. The last counter experiment he was involved in was UA1 headed by Prof. C. Rubbia. From 1980 onwards he was involved in establishing a Computer Science department at the University of Amsterdam. First as a member of a Computer Science group in physics, later, after establishing the Computer Science department, as a member of that group. In 1983 he was appointed as professor in Computer Science. His current research interests are in the field of parallel computing, intelligent autonomous robotics and their application in industrial automation. His researchgroup is collaborating in various projects mostly with industry, either in connection with ESPRIT or national programmes like SPIN or NWO. He has organized a number of international workshops and conferences in the field of robotics, parallel computing as well as real-time computing. Moreover, he has acted as program committee member of various international conferences in the fields of his research interest.

Lan Mai received her master degree in computer science from Pierre et Marie Curie University (Paris VI) in 1988. She joined Framentec in 1989 as research engineer. Member of the Re- seach and Development team, she par- ticipes to the conception and the re- alisation of applications in the ESPRIT2 project MARIE, in particu- lax, PES and the control architecture. Her current research interests include the treatment of uncertainty in per- ception and the exception handling

mechanism.

ties have to be dealt with. In the first place, the recovery planner needs to identify the extend of the partial recovery plan within the initial plan. This information is needed to find the restarting point after the recovery is completed. On the other hand, if a partial recovery is not possible, the recovery planner is faced with the task to generate a new plan from scratch. Research efforts have not yet provided a satisfactory solution in applying general problem solvers (GPS) to carry out this task.

We argue that recovery planning for a task with any complexity is not feasible without the use of application-driven heuristics. To solve the problems described above, we describe a model for exception handling that makes use of pre-defined

Erick Gaussens is Assistant Director of the Research and Development De- partment of Framentec (he joined this company in Autumn 1985). Within this context, he participated in the management of the KRITIC project and took an active part in defining the control mechanisms and the associ-

-,, ated architecture as well as the specification of the applications. Otherwise, he was involved in several negotiations of the department concerning the "Defense" fields (DRET contract,

consultant) or the robotic, for instance. Before joining Framentec, he was working at the Commission of Atomic Power (Institut for Nuclear Protection and Safety), at IBM (speech recognition) and at 'Saint Gobain Research' (numeric modelisation of industrial processes). Erick Gaussens is Doctor (Docteur d'Etat) of Mathematics (team of Pr. Ivar Ekeland at the University Paris IX Dauphine). His main research topics are: temporal planning, 'real time' and 'on line' systems, and the establishment of links between the mathematical theories of decision or information and the Artificial Intelligence con- cepts.

Fram;ois Arlabosse is Director of the Reseiach and Development Depart- ment of Framentec. While on second- ment at Teknowledge (Palo Alto) for AI training, he participated in the founding of Framentec in January 1984. Within the R&D department framework, he negotiated and ob- tained several ESPRIT projects (P256, P387-KRITIC, P820-QUIC, P1592- TAO, P2043-MARIE, P2152-VIEWS, P2256-ARCHON), of which he en- sures the management, and some con-

tracts in the 'Defense' field. He is chairman of the Scientific Commitee of CRIM (the Reseach Centre of Montpellier in Computer Science). Graduated as an engineer from ENSEM of Nancy, he is also holder of a B.SC. and a masters in mathematics. Francois Arlabosse was a research assistant in mathematics at the University of Rennes, a research engineer in the Central Telecommunication Laboratory (LCT) (mobile target interpretation from radar signals), a consultant to Framatome (cost control system), for whom he insured the creation of the 'Information Management' Department. His current works concern 'non-standard' logics, and the application of AI techniques to the control of dynamic systems.

Exception handling system for autonomous robots 199

planning strategies in combination with a collec- tion of general purpose planning functions. The planning strategies represent a scenario to handle a specific type of exception and consist of a sequence of planning functions. The recovery planner executes the planning functions to build the recovery plan.

In Section 2 of this paper, the model for exception handling is presented. First an outline is given in 2.1. In 2.2 we describe how the Exception Handling Model is applied to the control of an autonomous robot. Section 3 deals with the realization. The design criteria are given in 3.1 and in 3.2 the selected system is described. In the following sections the realization of an exception handling system for assembly applications is described. Section 4 presents the evaluation of the realization in respect to the criteria of Section 3.1. The paper is finalized by conclusions and a status report.

2. Diagnosis and exception handling

2.1. Exception Handling Model

In this section we describe a model which forms the frame work for the handling of exceptions of an autonomous system. This is called the Excep- tion Handling Model (EHM). EHM describes how autonomy can be reached within the scope of an a priori plan. Exception handling involves planning and execution of operation of a robot system which were not foreseen in advance. To cope with these situations, a planner needs a complete un- derstanding of all the consequences of an exception. Without knowing the specific details of the application, the number of these consequences and their complexity can be very large. As a result, the planner is not always able to react adequately to exceptions in a real environment. EHM addresses this problem of general purpose planners.

The underlying assumption of EHM is that the problem of exception handling planning for robotic systems can be reduced by combining application-driven heuristics with general purpose task and path planning functions. This allows on the one hand to overcome the problems of applying general purpose planners to specific application-driven domains, because the heuristic infor-

mation guides the planning functions. On the other hand, effective planning functions can be still freely used by the user of the system. First, we introduce the basic terminology used in EHM.

In EHM the task of a robot is represented by a task decomposition structure, called the task tree. The task tree is an ordered graph in which the root node represents the task of the robot. By decomposing the task into subtasks, addition~/1 nodes are created. At the lowest level the subtasks are no further decomposed, but directly expressed in the elementary operations of the robot. Each elementary operation (EO) of the robot represents a certain behaviour of the robot which is described by the control mechanism and the sensor feedback that is used. The sensor feedback is called nominal feedback if the value lies within the corresponding working range of the elementary operation. Any feedback outside the working range is non-nominal feedback. The set of all working ranges are the operating conditions of the elementary operation.

The environment of the robot is represented by a set of environment parameters. These parameters describe the state of the robot, transport systems, tools, parts and obstacles. The feedback of the robot sensors can be expressed as the current value of the environment parameters. Thus the operating conditions of an elementary operation are expressed as constraints on the allowable value of the environment parameters. The robot sensors do not measure all possible environment parameters, but are limited to a selection of the parameters. These observed environment parameters of an elementary operation are called the monitor conditions.

Task knowledge plays an important role in EHM. In the first place, EHM uses the task tree to keep track of the context information during the execution of the robot operations [11]. A stack structure is used to store the information concerning the tasks the robot is performing. At any moment the context of the current robot operation can be retrieved by looking into the stack structure. In the second place, the task knowledge is used to determine the scope of the affected a priori plan resulting from the exception. Hence the planning horizon is limited as well.

The second important concept used by EHM is that of recovery strategies. A recovery strategy is a framework of planning instructions which can be executed to produce a recovery plan for the robot.

200 G.R. Meijer, L.O. Hertzberger, T.L. MaLE. Gaussens, F Arlabosse

The strategies can be regarded as blue prints for a certain behaviour of the robot, without the specific details of how this behaviour is carried out in detail. The strategies in EHM are used to encode the heuristic information needed to guide the exception handling activity. In this way the planning complexity is reduced.

Next we take a closer look on how the mecha- riisms described by EHM are used for adaptive control of autonomous robots. To benefit from the capabilities of EHM, we must identify the degrees of freedom the robot system has within the context of a given task or mission. For this purpose we focus on the domain of robot assembly systems, although a similar analysis can be applied to mobile robots. Assembly tasks are often under- constrained and EHM aims at exploiting the in- herent degrees of freedom to allow exception handling. The degrees of freedom of a robot assembly task are: - freedom resulting from the assembly itself, - freedom associated with the operations of the

robot system. Each of these degrees of freedom can be exploited by applying a rescheduling, respectively recovery planning technique.

consequences for a possible alternative sequence have to be taken into account. As these aspects are very much driven by the nature of the application and the use of the resources, a general purpose rescheduling mechanism needs models of the effect of the exception on the assembly components and the resources. Without knowing the constraints of a particular assembly system, the generation of these models is very complex. This in turn results in time-consuming and complex scheduling mechanisms which do not guarantee to find a good solution within the given application environment.

EHM describes a mechanisms for the rescheduling of the assembly tasks which embeds the specific application dependent constraints by means of heuristics. These heuristics are specified in so-called task-rescheduling strategies. Each strategy refers to a distinct scheduling criterion. In EHM, the rescheduling mechanism is controlled by a limited number of pre-defined strategies. To apply the proper strategy at the correct moment, the application needs to be modelled in terms of the strategies to follow in case a certain exception occurs. This exception mechanism is described in detail in [12,20].

2.1.1. Rescheduling Any assembly contains parts and possibly sub-

assemblies. The order in which the assembly task can be performed is generally not uniquely defined. Different sequences of assembly are possible, all resulting in the required assembly. In state of the art robot systems, a specific order of assembly is chosen based on some optimization criteria. These criteria can be either associated with the availability of the parts or with the required resources for performing the assembly. Once a particular order is chosen, the robot is instructed to perform the assembly.

During the execution of the assembly task an exception can interrupt the chosen order of assembly. Although a required part of the assembly might be broken down or not be available, this does not necessarily mean that no further assembly steps can be taken. The freedom in the assembly order can be used to reschedule the assembly sequence, taking into account that a certain assembly step can not be performed. The scheduling criterion is now determined by the exception that occurred. The cause of the exception and the

2.1.2. Recovery planning For the assembly of a part the robot has to

plan a sequence of operations to perform. A typi- cal scenario for an assembly task is 'grasp and assemble' where 'assembly' can be 'place, insert, or any other task'. This scenario is decomposed into instructions which can be executed by the robot. These instructions are the elementary operations. For each elementary operation a geometrical planning has to be performed as well. The result of these planning activities is that a plan is built for the robot to perform the required assembly task. An exception during the execution of one of the operations aborts the execution of all other planned operations. If the robot is still in an operational state, the planning capabilities of the robot can be used to generate a recovery plan.

As pointed out, the occurrence of the exception can have complex consequences for the state of the assembly, the parts and the resources needed for the assembly. The recovery plan, however, has to take into account these consequences and use an appropriate scenario for the handling of the exception. Just as in the case of the rescheduling


mechanisms, in EHM the scenarios are specified as strategies. Each recovery planning strategy consists of a sequence of planning instructions, which together form the recovery plan. Because the deft- nition of the elementary operations of a robot are only depending on its capabilities and not on the task or mission, EHM provides a selection of the available recovery planning strategies for each exception that might occur. This so-called kernel already gives the system the capabilities to select proper recovery planning scenarios.

To bring in the heuristic knowledge of a specific application, the kernel of recovery planning strategies can be tailored to the needs of the application. Simply by changing the association between an exception and a strategy, or by alter- ing the order of the strategies, the recovery planning behaviour is adapted. Of course new strategies can be specified as well. This mechanism for recovery planning was introduced in [13] and was selected for the realization described in this paper.

EHM makes use of heuristics to effectively exploit the degrees of freedom in an assembly task. On the other hand, EHM uses general functions to transform the heuristics into detailed action plans. For the exploitation of the freedom in assembly order, scheduling functions are used. The order constraints of the assembly are represented in an adequate formalisms and scheduling functions are used to sort the possible assembly orders, given criteria corresponding to the required changes of the robot positions and tools.

For recovery planning EHM uses functions for task decomposition, path planning, fine motion planning, and grasp planning. These functions are general purpose and different variants can be used. Depending on their computational complexity appropriate functions can be selected by the recovery planner.

In the remainder of this paper we will deal only with the recovery planning aspects of EHM and in the following section the application to an autonomous robot is presented.

2.2. Exception handling for an autonomous robot

Planning functions to deal with unexpected events, can be realistically applied if the information concerning the application environment of the autonomous robot is available. In other words, models axe needed which describe the application

domain of the robot. These models are used by the exception handling functions of the robot control system. In this section we describe the exception handling functions and the corresponding models. In Section 3.3 the models for a specific application are presented.

Three functions are added to the control system of the autonomous robot to provide the exception handling behaviour. These are a monitor function, a diagnosis function and finally a function for recovery planning.

The robot of the assembly system must be capable of detecting deviations of environment parameters from their expected value during the execution of an elementary operation. This activity is performed by the monitor. For each elementary operation the monitor conditions consist of a status vector of relevant environment parameters and the value of each environment parameter is delivered by a corresponding sensor primitive. The monitoring conditions are the operating conditions of the elementary operation selected for monitoring.

The monitor performs its activity in 3 distinct steps. The rationale behind this is that the consequences of a failure or error message of the monitor are very much depending on whether the failure occurred before, during, or after the execution of the elementary operation.

After detection of an exception, the diagnosis function gains additional information on the nature of the exception and updates the internal model of the robot environment. In general, it is a problem to decide which environment parameters need updating and which do not. We are using default reasoning in combination with fault tree structures to guide the diagnosis activity [16,19]. The input for the diagnosis is the list of sensor values which exceeded their boundary values (as detected by the monitor). For each entry in this list, a corresponding entry is given for the fault trees. The fault tree for a specific sensor fault entry consists of a number of arcs and nodes. Each node represents a query to the sensory system to measure the value of an environment parameter. Based on the result of the query, a new arc is followed which either leads to another sensor request, or ends in a leave of the tree. Only those environment parameters are updated for which the corresponding sensor query is traversed in the fault tree. The leaves of the fault trees represent

202 G.R. Meijer, L.O. Hertzberger, T.L. MaLE. Gaussens, F. Arlabosse

the possible outcome of the diagnosis which is a classification of the exception at hand.

The third function is the recovery planner. For each elementary operation and the possible exceptions a set of strategies is defined. Each strategy of this set represents a scenario to handle the corresponding exception. The recovery planner selects the first available strategy and tries to execute the planning instructions which form the body of the strategy. If the recovery plan can be built, the recovery planner passes over the control to the plan executor of the robot. If a recovery plan can not be built, either because the planning functions do not provide a solution or the physical capabilities of the robot are inadequate, the recovery planner selects the second strategy of the strategies set. If no recovery plan can be built and the strategies list is exhausted, the robot can not achieve the current task and rescheduling of tasks is needed.

Summarizing, for the application of EHM, monitoring, diagnosis and recovery planning functions are added to the control system of the autonomous robot. To reach autonomy in a given application domain, modelling of the following information is needed by these functions: - elementary operations of the robot, - formalization of task instructions, - available sensor primitives, - monitor conditions for each elementary oper-

ation, - diagnosis trees, - formalization of exceptions, - description of available planning functions, - recovery planning strategies for each elemen-

tary operation-exception pair. In Section 3.3 the modelling for an assembly application is further discussed.

3 . R e ~ a f i o n

3.1. System design

Several considerations were taken into account before a selection of an appropriate realization environment was made. The mechanisms described by EHM are an example of adaptive control for autonomous robots. In such a system a continuous feedback based on the observations of the environment is necessary. This demands that

Control Program

Domain specific procedures

General planning functions

Fig. 1. Structure of the Exception Handling Model.

given an initial task or mission, the robot post- pones final decisions on which route to follow or which action to take. Only when the robot has to take action, the actual environment data has to be used to plan these actions. Even during the execution of an action, the robot has to be sensitive to unexpected events which require adaptation of the plan. This allows the robot behaviour not to be strongly constrained in a strictly predefined plan of actions. The plans become more and more precise as the robot task evolves from a given intention to a physical robotic operation. Deci- sions can be taken as late as possible which pre- vents the planner from making too many assump- tions on the environment while building a plan.

In EHM this adaptive behaviour is realized by combining general planning functions with domain-specific procedures. The combination of these two sources of information is performed by a control program (Fig. 1 ).

The control program selects the proper procedures which are encoded as strategies, and uses the general planning functions to build a recovery plan. The recovery plan allows the robot to adapt itself to the new environment conditions. As such, the control program is provided by the interpreter mechanisms of an expert system. The expert knowledge is then represented by the general planning functions and the domain-specific procedures.

Based on these considerations, we listed the following requirements for the realization: - To access the functionality of the exception

handling mechanisms, an environment is needed which reflects the functional division between the general planning functions and the domain-specific procedures.

- To apply the exception handling mechanisms to various application domains, the domain- specific knowledge should be easily adaptable.

- Simulation facilities are needed to test the exception handling mechanisms.

- The exception handling system has to operate in a real robot environment, The realization


therefore needs adequate interfaces to control a robot system and to react on sensor data.

Several systems were analysed on their functionality like a temporal planner [1] and Black Board Systems [3]. Because of the need to postpone actions of the robot and react to new environment data, we selected a system called Procedural Ex- pert System [2,4,9,10].

3.2. Procedural Expert System for task planning

In this section we describe the use of Proce- dural Expert System (PES) for task planning. PES is a partial and hierarchical planner. Partial, because a robot needs only a partly elaborated plan to start acting as more precisions about this plan will only be specified just when the robot is about to use them, and hierarchical, because we have several abstraction levels in plan generation.

PES provides the robot with goals and intentions which dictate the robot behaviour. The idea is to associate each goal, or task, of the robot with a particular context, defined by both the current state of the environment and the remaining task the robot has to achieve. Resulting from this association, are the intentions of the robot which dictate how the robot will reach its goal. In other words, each intention of the robot is only valid, in a given configuration of the robot and the environment. Any unexpected change may thus make the current intention invalid and the modified world will then justify another intention. In this way, the robot is able to interrupt immediately whatever action was active and modify its plan in order to tackle the arising problem.

The underlying assumption of PES is that most of the information needed for robot control has a procedural nature. As such, PES provides facilities to manipulate the procedural knowledge which is represented by so-called Knowledge Areas (the KA's). At any given time the state of the robot and the environment is represented by a Knowl- edge Base (KB). An interpreter mechanism links the procedural information in the KA's to the current goals and beliefs (or state of the environment) in the KB to select the proper procedure to follow.

1. Knowledge Base (KB) This base contains both the data describing the internal state of the system such as the current goals and the knowledge the system has of the external world or the state of the world. It can be seen as the set of the robot beliefs at a distinct moment in time and may include sensor data as well as information on the structure of the robot or its environment.

2. Knowledge Areas (KA's) This knowledge base contains the procedural information. A KA is composed of two parts: (a) The invocation conditions: a declarative part

describing the conditions under which this KA will be chosen to be executed. A condition can be: - (GOAL (!P)) - the execution of this KA

will make the fact P true, i.e., present in the KB. These KA's are called goal- driven KA's,

- (FACT (?P)) - if the fact P is present in KB, execute this KA. If the invocation conditions contain only fact clauses, the KA is said to be fact driven,

- or any combination of goals and facts. (b) The body: the procedural part which can be

either a rigidly coded procedure, or a Re- cursive Transitive Network (RTN). An RTN is a graph having one starting node and several ending nodes. Nodes have no particular meaning and each edge is labelled with an action which may be: - (!P) - achieve P, i.e., make P true, - (?P) - query P, i.e., is P true?, - ( # P) - with the constraint P, i.e., main-

tain P true, - or any combination of goals, facts and

constraints. !, ? and # are defined by Georgeff as temporal operators. The procedure is executed by crossing down the RTN from the starting node until an ending node in a depth-first left-to-right way is reached. To cross an edge, the action that labels it must have been achieved, which may be done by recursively calling another KA. For this reason KA's can be seen as partially elaborated plans of the robot.

3.2.1. Knowledge bases PES works with two information structures

named Knowledge Base and Knowledge Areas:

3.2.2. The interpreter The PES interpreter deals with the KA's much

in the same way as a classical inference system

204 G.R. Meijer, L.O. Hertzberger, T.L Mai, E. Gaussens, F Arlabosse

does with rules. At any given moment, several goals are active in the system and certain beliefs are held in the database. PES selects the applica- ble KA's by matching their invocation conditions with the current goals and beliefs, chooses one and executes it. PES will repeat this process each time a new goal is activated or a new fact is acquired.

The way PES selects one KA among several candidates depends on the programmer of a PES application. However, to allow the system to react rapidly as new beliefs are acquired in the data base, fact-driven KA's, whose invocation conditions are only fact clauses, have higher priority than others. With this it is possible to realize interrupt driven actions.

The stack of current active goals dictates the robot behaviour. The outstanding KA's represent its intentions and PES will first try to fulfil them. However, the system remains sensitive to changes in the external world and can be interrupted whatever pending action, to deal with an unexpected event. This can be done by using fact-driven KA's or by directly dealing with the goals' stack. In particular, the second way allows the system to completely switch its focus and pursue new goals as situation may demand it.

In a similar approach taken by Brooks [7] the robot is controlled by a set of behaviours which can subsume the role of others as the environment demands so. In our view the subsumption architecture is limited by the inability to systematically incorporate a priori plan information in the behaviour of the robot. In PES this information is coded as partial plans represented by the goal- driven KA's.

We used the fact-driven capacities of PES to realize a reactive planning and exception handling system to control a robot performing an assembly task. This application is described in the next section.

3.3. Assembly application

3. 3.1. Modelling of the assembly application To apply the techniques for exception handling

described in the Section 2 with PES, we modelled the knowledge needed for monitoring, diagnosis and recovery planning of a robot assembly application according to the model requirements presented in Section 2.2 [22]. The robot system we

Table 1 Sensor primitives

Sensor primitive Description

Check Motion (CM)

Robot Forces (RF) Robot Free (RFR) Plan Position (PP) Object Available

(AO) Object Orientation

(oo) Find Object (LO) Identify Object

0o)

Measure movement of robot Measure forces on the robot Measure contact with objects Measure current position

Measure avalibility of object in gripper

Measure orientation of object in gripper Find location and orientation of object

Identify object, using data-base

studied consists of an assembly robot equipped with a wrist force sensor and with a camera placed on the gripper. Also an overhead camera is available. The sensor primitives are listed in Table 1. A presentation of exception handling within the area of sensor data processing can be found in [21].

In Table 2 the elementary operations of the robot and the corresponding monitor conditions are given. This list instructs the monitoring mod- ule which sensor primitive should be checked when the corresponding elementary operation is executed. The required value of the sensor primitive is derived from the parameters of the corresponding elementary operation. For instance, the required status of the gripper is determined by using the value of the 'Part ' parameter of the elementary operation. If this parameter is nil, no part is expected in the gripper. If the parameter contains the name of a part, the width of the gripper is determined from the geometrical description of the part. The monitor uses this table to select the sensor primitives it has to check before (pre),

Table 2 Elementary operations and monitor conditions

Elementary pre during post operation

Transfer Grasp Detach Insert Approach Depart FindObject IdentifyObject -

OA, OO CM, RF, AO, OO PP, AO, OO RF PP, AO, OO OA, OO PP, RF RF, AO, OO RF, OA, OO PP AO, OO RF, OA, OO PP RF, AO, OO RF, OA, OO PP


Table 3 Exceptions

COLLISION - Collision with unknown object (C) -Coll is ion with unknown object at position P

(CP) - Collision with object O at position P (COP)

OBJECT ERROR - Object lost at unknown position (O) -Objec t lost at position P (OP)

NOT REACHED GOAL - not reached planned goal, now at position P (CE)

OBSTRUCTION BY FORCE - Obstruction with unknown object (F)

-Obs t ruc t ion with unknown object at position P (FP)

- Obstruction with object O at position P (FOP)

GRIPPER ERROR - Object moved in gripper (GE)

used 10 strategies to deal with the exceptions of Table 3. An example of a strategy for handling a lost object is given by the following planning instructions:

Strategy: PlanPickUpPart sp_findobject(CurrentPart), pLanrecovery (GetPart(CurrentPart)).

The first planning instruction invokes sensor primitive to indentify and locate the current part. The recovery planner uses the task stack to instantiate the CurrentPart variable. The second instruction invokes the task decomposition and planning of a 'GetPart' operation.

during or after (post) the execution of the elementary operation.

The structure of the task decomposition of the robot is specified by:

Task:

Subtask:

Subtask:

Subtask:

assemble_part(Partname) :=get_part(Partname)

move_part(Partname) place_part(Partname)

get_part(Partname) :=eo_move(Partlocation)

eo_grip(Partname)

move_part(Partname) :=eo_move(Partgoal)

place_part(Partname) :=eo_ungrip( )

For each of the elementary operations a fault tree is modelled according to the description of 2.2. The diagnosis activity results in the classification of the exception. In Table 3 the possible exceptions are listed. Each type of exception is further divided into several variants, depending on the results of the diagnosis.

Finally we specified strategies for each combination of elementary operation and exception. Each pair elementary operation/exception is associated with a list of handling strategies. The list contains at least one strategy. Each strategy itself is a sequence of planning instructions. In total we

3.3.2. Realization of PES for exception handling In this section we will describe the representa-

tion of the models and control structure for the exception handling functions in PES. There are many ways to project the exception handling mechanisms of EHM on the program structures of PES. We choose a projection such that the PES structures correspond closely with the architecture of EHM. The global design consideration is that the planning elements like task decomposition and planning functions, can be interleaved by using goal-directs KA's. On the other hand, the reactive functions for monitoring and exception handling which correspond to the influence of the external environment, are represented by fact-driven KA's.

The task decomposition is represented by a set of goal-driven KA's. Each KA invokes another KA by querying the invocation condition of this KA. Also the general functions for path planning are represented by goal-driven KA's and invoked only then when a path for the robot is needed. In Fig. 2 the interactions between a part of the goal-driven KA's are graphically represented. The nodes 'Identify Part' and 'Locate Part' represent sensor primitive instructions and the nodes 'Plan' and 'Local Plan' stand for the general path planning functions. The other nodes are part of the task decomposition structure.

The reactive behaviour of the PES system is used to represent the monitor and the exception handling functions. The monitor conditions are treated as follows. For each KA of the task decomposition the monitor conditions are represented as context labels of the KA. An example is

206 G.R. Meijer, L O. Hertzberger, T.L. Mai, E. Gaussens, 1:.. Arlabosse

I ~en~y Paa ] I Assemble Parl I ('P' LP)

(RFR) (CM,RFR) (PP)

Fig. 2. Graphical representation of a partial task planning for the assembly robot in PES.

given in Fig. 3 which displays a part of a KA corresponding to the 'get-part' task. The 'ARC' element of the KA refers to the RTN representation of the body of the KA. In this node the action (! (the POSITION-PART of $PART is IN- GRIPPER)) invokes the KA which represents the grip operation. The context label is expressed as the parameter of a 'with' statement.

Logically the 'with' statement is equal to a fact-driven KA. Upon execution of the KA by the interpreter mechanism, the context label is added in the KB.

If a subsequent KA invalidates the label from

*******************************************

(PES:INSTANTIATE ARC2-5 'PES:ARK

:LABEL "! grip"

:CHILDREN '(ARC2-6)

:ACTION '(! (the POSITION-PART of

$PART is INGRIPPER)

with

(the MESSAGE of monitorOA is OA))) *******************************************

Fig. 3. A segment of a goal-driven KA for a ' Grip' task.

the KB, the interpreter backtracks to the point where the assertion of the label was made. In other words, the context information can be inter- preted as the internal model representation of the state of the robot system and the environment. If a part of the model is invalidated, the system backtracks to the point with a valid state model. This method is called constraint-based backtracking. Using this interpretation of the context labels, we incorporated the diagnostics functions into the task decomposition.

To simulate the use of sensor primitives, we use a set of fact driven KA's. Each KA of this set corresponds to an elementary operation in the pre, during or post phase. The KA queries the user of the system for the values of the sensor primitives corresponding to the context labels. The labels in the KB are updated which can result in the backtracking behaviour described above. Fig. 4 gives a fact-driven KA which checks the monitor condi-

***************************************************************************************

(PES:INSTANTIATE KA6 'PES:KA

:LABEL "monitor eo grip, pre"

:INVOCATION CONDITION '(AND (FACT

:EFFECT '((AMBIGUOUS))

:FIRSTARKS ' (ARC6-0))

(? (the EO of ROBOTSTATUS is GRIP)))

(FACT (? (the PHASE of ROBOTSTATUS is PRE)))

(FACT (? (the MESSAGE of monitorOA is NOA)))


:LABEL "instantiate SPART "

:CHILDREN '(ARC6-1)

:ACTION '(? (the TASK of ROBOTSTATUS is SPART)


:LABEL "get part "

:CHILDREN ' (ARC6-2)

:ACTION ' (! (the CONTENT of GRIPPER is $PART)) ***************************************************************************************

Fig. 4. A fact-driven KA for a monitor function.


tion during the execution of a grip operation in its pre-phase.

The exception handling function is split in two. First, the strategies themselves are represented by goal driven KA's. Each node of the KA representing an exception handling strategy invokes other goaldriven KA's. These other KA's are noth- ing else then the already ,existing KA's for task decomposition, sensor primitives functions and planning functions. Thus, existing capabilities of the system are configured in such a way that they handle a certain exception.

Secondly, the selection of the proper strategy is represented by fact-driven KA's. For each elementary operation and exception pair there are different strategies. A fact-driven KA is used to boot the proper exception handling strategies given the current state of the environment. The body of these KA's contain the invocation condition of the KA representing the relevant strategy. If alternative strategies are available to handle an exception, these are represented as 'OR' nodes.

During execution of the system the KB contains the status of the robot, the model of the parts and their location. It also represents the planned paths for the robot. With each KA invoked by the interpreter, a new entry is made in the stack of intentions. As these intentions follow logically from the task decomposition of the KA's, they represent the task structure of the robot activities and can be used by the exception handling function to identify the context information of a failed elementary operation.

4. Ev~uafion

An evaluation of the realization was made based on the considerations of Section 3.1. The control part of the EHM deals with the manipulation of status information of the robot, provides the selection mechanisms of the exception handling strategies and controls the building of a recovery plan. The domain-specific information is described by the models of the application domain.

In PES the functional division between the general planning functions and the domain-specific procedures is reflected by the concept of the Knowledge Area (KA). The interaction of the KA's is determined by the invocation conditions of the KA. Therefore the invocation conditions

reflect the way the control program makes use of the KA. On the other hand, the body of a KA contains the procedural knowledge which corresponds to the models describing the application domain. For example, the task decomposition and the exception handling strategies are represented by the Recursive Transitive Network (RTN) of the KA's. So the general planning functions and the domain-specific procedures are separated by using different KA's to represent them.

Thus, the system structure of PES can reflect the architecture of the EHM mechanisms and the functionality of exception handling mechanisms can be directly accessed by observing the behaviour of the PES system. However, as explained earlier, there exists several solutions for the projection of the EHM architecture on the KA's of PES. For example disjunction ('OR') can be represented in two ways. The first solution is to use two KA's, both with the same invocation condition. The alternative is to use only one KA and to explicitly model the disjunction in the RTN of the KA. Both solutions result in the same system behaviour, but behave differently under constraint-based backtracking using the context labels.

The additional advantage of the distinction between control and application domain information is that the specification of the application can be performed independent of the control structures using the models. By changing the contents or body of a KA, the control of the system is not affected. Therefore PES allows effective modelling of other application domains without the burden of dealing with control details.

The realized system is tested in an interactive manner. The PES user interface provides different windows to display the status of the KB, the goal stack and the KA which is currently executed. Additional windows are available for user input and sensor simulation requests. The system can either run automatically, only stopping when user input is required, or run by hand. In the last case various tracing options are possible.

Editing of the KA's can be done with the use of a text editor or by using a graphical editor. To provide adequate test facilities, work is in progress to run the interpreter together with the KA editor. This will shorten the test loop.

Applying PES to the control of a real robot equipped with sensors, requires reactive behaviour. The interpreter mechanisms provide this

208 G.R. Meijer, L.O. Hertzberger, T.L. Mai, E. Gaussens, F. Arlabosse

behaviour by assigning a higher priority to the fact-driven KA's. We used this possibility to incorporate sensor input. This means that when new sensor data is available, direct action can be taken to bring the robot in a save state. Only when a stable and save condition of the robot is reached, is it justified to spend time on strategy selection and planning.

5. Status of the project and conclusions

We presented the realization of exception handling mechanisms for autonomous robots with the tool PES. Exception handling as described by EHM, provides the robot with capabilities to monitor its operations. Upon the detection of an exception, a diagnosis can be performed which leads to a classification of the exception at hand. Task rescheduling and recovery planning is performed based on heuristic knowledge of the application. This knowledge is encoded as strategies.

The PES environment is well suited to realize the mechanisms of EHM for its close correspon- dence with the architecture of EHM. We presented an implementation of EHM with PES for an assembly robot application. Modelling and implementation are described. The current implementation runs standalone and the user provides the answers to sensor requests.

The reactive behaviour of PES makes it also suitable to realize adaptive control of a real robot. The interfaces between PES and an assembly robot equipped with a combination of force and vision sensors, is currently realized [15] and we will in- vestigate closed loop control with PES.

To improve the modularity and reactiveness of a system realized with PES, several PES instances can be coupled. Sensor selection and planning can be split from task decomposition and control functions. Each area can be realized by a PES instance. To maintain consistency in the beliefs and intentions of cooperating PES instances and to handle conflict resolution, a tool called 'A Truth Maintenance System' (ATMS) can be applied. Work on this field was started and results will be reported elsewhere.

The PES system used is written in Com- monLisp and runs on a MicroExplorer. The origi- nal version of the system runs under KEE on a TI-Explorer. We recently started using a version

of PES for S U N / U n i x workstations running under CommonLisp and X-Windows.

Acknowledgements

The work on exception handling was carried out in Esprit I project 623: 'Operational control for robot system integration into CIM'. Started in February 1985, this project resulted in methods and tools for production planning as well as robot programming systems [5,18]. The University of Amsterdam participated in this area with the development of the Exception Handling Model and tools to generate monitor and recovery procedures [8,22]. Currently this work is continued in Esprit II project 2202: 'CIM Systems Planning Toolbox (CIM-PLATO)' by the same consortium.

The realization of the mechanisms of EHM using PES is supported by Esprit II project 2043: 'Mobile Autonomous Robot in Industrial En- vironment (MARIE)' . In the project numerical and symbolical techniques are studied and devel- oped for sensing and control of an autonomous robot. The diagnosis and exception handling system will be applied both on an inspection robot arm and on the control of a mobile platform. The use of PES in planning has been tested by Framentec among others in Esprit project 820 and in a French space project called SATEXPERT.

References

[1] F. Arlabosse, V. Duong and E. Gaussens, An event based planner for a real scale process control application, Proc. IEEE 1988 Workshop on AI in Process Engineering (August 1988).

[2] F. Arlabosse, B. Jean-Bart, N. Porte and B. deRavinel, An efficient problem solving architecture using ATMS, J. of AI Communications 1 (4) (1988) 6-15.

[3] F. Arlabosse, V. Duong, P. Lepage and E. Gaussens, Blackboard and alternative architecture design for two real scale KBS industrial applications in: J. Jagannathen, R. Dodhiawala and L. Baum, eds., Blackboard Architec- tures and Applications (August 1989).

[4] F. Arlabosse, Esprit project 820; Toolkit specification, Project internal documentation, available at Framentec.

[5] R. Bernhardt and K. Tierney, Operational Control for Robot System Integration into CIM (Chapmann & Hall, 1991).

[6] R.A. Brooks, Symbolic error analysis and robot planning, The Int. J. of Robotics Research 1 (4) (1982) 29-68.

[7] R.A. Brooks, A robust layered control system for a mobile


robot, IEEE J. of Robotics and Automation RA-2 (1) (1986).

[8] L. Camarinha, U. Negretto and G.R. Meijer, Information integration for assembly cell programming and monitoring in CIM, Proc. of 21th International Symposium on Automotive Technology and Automation, Wiesbaden (6-10 Nov. 1989).

[9] M.P. Georgeff and A.L. Lansky, Procedural knowledge, Proc. of IEEE Special issue on Knowledge representation 74 (1986) 1383.

[10] M.P. Georgeff and A.L. Lansky, Reactive reasoning and planning, Proceedings of AAAI (1987) 677.

[11] M. Gini, Symbolic and qualitative reasoning for error recovery in robot programs, in: U. Rembold and K. H~Srmarm, eds., Proc. of the NA TO International Advanced Research Workshop on Languages for Sensor Based Control in Robotics, Italy (Sept. 1986).

[12] G.R. Meijer and L.O. Hertzberger, Exception handling for robot manufacturing process control, Proc. of CIM Europe Conference, Madrid (May 18-20 1988) (IFS publications).

[13] G.R. Meijer and L.O. Hertzberger, Off-line programming of exception handling strategies, Proc. o f lFAC/SYROCO Conference, Karlsruhe (October 1988) (VDE/VDI publications).

[14] G.R. Meijer, G.A. Weller, F.C.A. Groen and L.O. Hertz- berger, Sensor based control for autonomous robots, Proc. of IEEE International Conference on Control and Applica- tions, Jeruzalem (April 1989) WP-3-4.

[15] G.R. Meijer, T.L. Mai, E. Gaussens, L.O. Hertzberger and F. Arlabosse, Robot control with procedural expert sys-

tern, in: T. Jordanides and B.J. Torby, eds., Proc. of NATO Advanced Study Institute on Expert Systems and Robotics, Corfu (15-27 July 1990) (NATO-ASI Series F, 1991).

[16] R. Milne, Strategies for diagnosis, IEEE Transactions on Systems, Man, and Cybernetics SMC-17 (3) (May/June 1987).

[17] M.J. Schoppers, Universal plans for reactive robots in unpredictable environments, Proc. of 11th IJCAI, Milano (1987) 1039-1046.

[18] Spur, et al., Planning and programming of robot in- tegrated production cells, in: Esprit '87 Achievements and Impacts (North-Holland, Amsterdam, 1987).

[19] S. Srinivas, Error recovery in robots through failure reason analyis, AFIP Proc. of National Computer Conference, Anaheim, CA (1978) 275-828.

[20] F. Tuijnman, G.R. Meijer and L.O. Hertzberger, Data modelling for a robot query language, Proc. of Conference on Intelligent Autonomous Systems 2 (IAS-2), Amsterdam (11-14 Dec. 1989) (North-Holland, Amsterdam).

[21] G.A. Weller, F.C.A. Groen and L.O. Hertzberger, A sensor processing model incorporating error detection and recovery, Proc. of NATO Advanced Research Workshop on Traditional and Non-Traditional Sensors, Maratea (28 August-1 September 1989) (NATO ASI series).

[22] G.M. van Wonderen, AI and high level robot control and a preliminary implementation of a High Level Interpreter, Technical report, Faculty of Mathematics&Computer Science, Department of Computer Systems, July 1988.

exception handling system for autonomous robots based on pes

Documents