1

Robotics and Artificial Intelligence:a Perspective on Deliberation Functions

Felix Ingrand, Malik GhallabLAAS–CNRS, Universite de Toulouse7, Av. Colonel Roche, 31077 Toulouse, FranceE-mail: {felix,malik}@laas.fr

Abstract: Despite a very strong synergy betweenRobotics and AI at their early beginning, the two fieldsprogressed widely apart in the following decades. How-ever, we are witnessing a revival of interest in the fer-tile domain of embodied machine intelligence, whichis due in particular to the dissemination of more ma-ture techniques from both areas and more accessiblerobot platforms with advanced sensory motor capabil-ities, and to a better understanding of the scientificchallenges of the AI–Robotics intersection.The ambition of this paper is to contribute to thisrevival. It proposes an overview of problems and ap-proaches to autonomous deliberate action in robotics.The paper advocates for a broad understanding of de-liberation functions. It presents a synthetic perspectiveon planning, acting, perceiving, monitoring, goal rea-soning and their integrative architectures, which is il-lustrated through several contributions that addresseddeliberation from the AI–Robotics point of view.

1. Introduction

Robotics is an interdisciplinary integrative field,at the confluence of several areas, ranging from me-chanical and electrical engineering to control the-ory and computer science, with recent extensionstoward material physics, bioengineering or cogni-tive sciences. The AI–Robotics intersection is veryrich. It covers issues such as:• deliberate action, planning, acting, monitoring

and goal reasoning,

• perceiving, modeling and understanding open en-vironments,

• interacting with human and other robots,

• learning models required by the above functions,

• integrating these functions in an adaptable andresilient architecture.

Robotics has always been a fertile inspirationparadigm for AI research, frequently referred toin its literature, in particular in the above top-ics. The early days of AI are rich in pioneeringprojects fostering a strong AI research agenda onrobotics platforms. Typical examples are Shakeyat SRI [85] and the Stanford Cart in the sixties,or Hilare at LAAS [36] and the CMU Rover [70]in the seventies. However, in the following decadesthe two fields developed in diverging directions;robotics expanded mostly outside of AI laborato-ries. Hopefully, a revival of the synergy betweenthe two fields is currently being witnessed. Thisrevival is due in particular to more mature tech-niques in robotics and AI, to the development ofinexpensive robot platforms with more advancedsensing and control capabilities, to a number ofpopular competitions, and to a better understand-ing of the scientific challenges of machine intelli-gence, to which we would like to contribute here.

This revival is particularly strong in Europewhere a large number of groups is actively con-tributing to the AI–Robotics interactions. For ex-ample, out of the 260 members of the Euron net-work,1 about a third investigate robotics decisionand cognitive functions. A similar ratio holds forthe robotics projects in FP6 and FP7 (around ahundred). Many other european groups not withinEuron and projects outside of EU programs areequally relevant to the AI and Robotics synergy.This focused perspective on deliberative capabili-ties in robotics cannot pay a fair tribute to all eu-ropean actors of this synergy. It illustrates howeverseveral contributions from a few groups through-out Europe.2 Its ambition is not to cover a com-prehensive survey of deliberation issues, and evenless of the AI–Robotics intersection. In the lim-

1http://www.euron.org/2e.g., from Barcelona, Bremen, Freiburg, Grenoble, Karl-

sruhe, London, Lille, Linkoping, Munich, Orebro, Os-

nabruck, Oxford, Rennes, Roma and Toulouse

AI Communications

ISSN 0921-7126, IOS Press. All rights reserved

2

ited scope of this special issue, we propose a syn-thetic view of deliberation functions. We discussthe main problems involved in their developmentand exemplify a few approaches that addressedthese problems. This “tour d’horizon” allows us toadvocate for a broad and integrative view of de-liberation, where problems are beyond search inplanning, and beyond the open-loop triggering ofcommands in acting. We hope through this per-spective to strengthen the AI–Robotics synergies.

The outline of the paper is the following: fivedeliberation functions are introduced in the nextsection; these are successively addressed throughillustrative contributions; section 8 is devoted toarchitecture problems, followed by a conclusion.

2. Deliberation functions in robotics

Deliberation refers to purposeful, chosen orplanned actions, carried out in order to achievesome objectives. Many robotics applications donot require deliberation capabilities, e.g., fixedrobots in manufacturing and other well-modeledenvironments; vacuum cleaning and other deviceslimited to a single task; surgical and other tele-operated robots. Deliberation is a critical function-ality for an autonomous robot facing a variety ofenvironments and a diversity of tasks.

Monitoring

Goal Reasoning

Acting Perceiving

Models, data, and knowledge bases

Environment

User

Planning

Robot’s Platform

Fig. 1. Schematic view of deliberation functions.

Several functions can be required for acting de-liberately. The frontiers between these functionsmay depend on specific implementations and ar-

chitectures, but it is clarifying to distinguish thefollowing five deliberation functions, schematicallydepicted in figure 1:

• Planning : combines prediction and search to syn-thesize a trajectory in an abstract action space,using predictive models of feasible actions and ofthe environment.

• Acting : implements on-line close-loop feedbackfunctions that process streams of sensors stimu-lus to actuators commands in order to refine andcontrol the achievement of planned actions.

• Perceiving : extracts environment features to iden-tify states, events, and situations relevant for thetask. It combines bottom-up sensing, from sen-sors to meaningful data, with top-down focusmechanisms, sensing actions and planning for in-formation gathering.

• Monitoring : compares and detects discrepanciesbetween predictions and observations, performsdiagnosis and triggers recovery actions.

• Goal reasoning : keeps current commitments andgoals into perspective, assessing their relevancegiven observed evolutions, opportunities, con-straints or failures, deciding about commitmentsto be abandoned, and goals to be updated.

These deliberation functions interact within acomplex architecture (not depicted in Fig. 1)that will be discussed later. They are interfacedwith the environment through the robot’s plat-form functions, i.e., devices offering sensing andactuating capabilities, including signal processingand low-level control functions. The frontier be-tween sensory-motor functions and deliberationfunctions depends on how variable are the environ-ments and the tasks. For example, motion controlalong a predefined path is usually a platform func-tion, but navigation to some destination requiresone or several deliberation skills, integrating pathplanning, localization, collision avoidance, etc.

Learning capabilities change this frontier, e.g.,in a familiar environment a navigation skill iscompiled down into a low-level control with pre-cached parameters. A metareasoning function isalso needed for trading off deliberation time foraction time: critical tasks require careful delibera-tion, while less important or more urgent ones maynot need, or allow for, more than fast approximatesolutions, at least for a first reaction.3

3Learning as well as metareasoning are certainly neededfor deliberation; they are not covered here to keep the ar-

gument focused.

3

3. Planning

Over the past decades, the field of automatedplanning achieved tremendous progress such asa speed up of few orders of magnitude in theperformance of Strips-like classical planning, aswell as numerous extensions in representations andimprovements in algorithms for probabilistic andother non-classical planning [35]. Robotics stressesparticular issues in automated planning, such ashandling time and resources, or dealing with un-certainty, partial knowledge and open domains.Robots facing a variety of tasks need domain spe-cific as well as domain independent task planners,whose correct integration remains a challengingproblem.

Motion and manipulation planning are key ca-pabilities for a robot, requiring specific represen-tations for geometry, kinematics and dynamics.Probabilistic Roadmaps and Rapid Random Treesare well developed and mature techniques for mo-tion planners that scale up efficiently and allow fornumerous extensions [61]. The basic idea is to ran-domly sample the configuration space of the robot(i.e., the vector space of its kinematics parameters)into a graph where each vertex is a free configura-tion (away from obstacles) and each edge a directlink in the free space between two configurations.Initial and goal configurations are added to thisgraph, between which a path is computed. Thispath is then transformed into a smooth trajectory.Manipulation planning requires finding feasible se-quences of grasping positions, each of which is apartial constraint on the robot configuration thatchanges its kinematics [87]. Many other open prob-lems remain in motion and manipulation planning,such as dynamics and stability constraints, e.g. fora humanoid robot [46], or visibility constraints toallow for visual servoing [14].

Task planning and motion/manipulation plan-ning have been brought together in several work.The Asymov planner [12] combines a state-spaceplanner with a search in the motion configurationspace. It defines places which are both states, aswell as sets of free configurations. Places definebridges between the two search spaces. The state-space search prunes a state whose correspond-ing set of free configurations does not meet cur-rent reachability conditions. Asymov has been ex-tended to manipulation planning and to multi-

robot planning of collaborative tasks, such as tworobots assembling a table.

The integration of motion and task planningis also explored in [96] with Angelic HierarchicalPlanning (AHP). AHP plans over sets of stateswith the notion of reachable set of states. Thesesets are not computed exactly, but bounded, e.g.,by a subset and a superset, or by an upper and alower bound cost function. A high-level action hasseveral possible decompositions into primitives. Aplan of high-level actions can be refined into theproduct of all feasible decompositions of its ac-tions. A plan is acceptable if it has at least one fea-sible decomposition. Given such a plan, the robotchooses opportunistically a feasible decomposingfor each high-level action (AHP refers to the an-gelic semantics of nondeterminism). The boundsused to characterize reachable sets of states areobtained by simulation of the primitives, includ-ing through motion and manipulation planning,for random values of the state variables.

A different coupling of a hierarchical task plan-ner to fast geometric suggesters is developedin [45]. These suggesters are triggered when thesearch in the decomposition tree requires geomet-ric information. They do not solve completely thegeometric problem, but they provide informationthat allows the search to continue down to leaves ofthe tree. The system alternates between planningphases and execution of primitives, including mo-tion and manipulation actions. Online planning al-lows to run motion or manipulation planners (notsuggesters) in fully known states. The approachassumes that the geometric preconditions of theabstract actions can be computed quickly and ef-ficiently by the suggesters, and that the sub-goalsresulting from actions decomposition are executedin sequence (no parallelism). The resulting systemis not complete. Failed actions should be reversibleat a reasonable cost. For problems where these as-sumptions are met, the system is able to quicklyproduce correct plans.

4. Acting

In contrast to planning that can easily be spec-ified as an offline predictive function, decoupledfrom the intricacies of the executing platform, act-ing is more difficult to define as a deliberation func-tion. The frequent reference to execution control is

4

often reductive: there is more to it than just trig-gering actions prescribed by a plan. Acting has tohandle noisy sensors and imperfect models. It re-quires non-deterministic, partially observable anddynamic environment models, dealt with throughclosed-loop commands.

To integrate these requirements with those ofpredictive planning models, different forms of hier-archization are usually explored. For example (fig-ure 2):• planning deals with abstract preconditions-effects

actions;

• acting refines opportunistically each action intoskills and a skill further down into commands.This refinement mechanism may also use someplanning techniques but with distinct state spaceand action space than those of the planner.

Planning techniques in action refinement

30

Acting

Mission 〈 ..., action, ... 〉 〈 ..., skill, ... 〉

〈 ..., command, ... 〉

Robot’s platform

Planning

Fig. 2. Refining actions into skills.

The skill into which an action is refined maychange during that action execution. For exam-ple, several navigation skills may offer different lo-calization or motion control capabilities adaptedto different environment features. A goto(room1)

action can be refined into a sequence of differentnavigation skills.

This hierarchization scheme may rely on dis-tinct knowledge representations, e.g. STRIPS op-erators combined to PRS [42] or RAP [27] pro-cedures. In some cases a single representation isused for specifying planning and acting knowledge,e.g., Golog [63] or TAL [20] languages. Other ap-proaches use a single representation seen at differ-ent levels of abstractions and refined appropriately,as in Hierarchical MDPs [38] for example.

Various computational techniques can be usedto design a deliberate acting system. We pro-pose to organize these approaches into five cate-gories (see table 1) presented in the following sub-sections. Before discussing and illustrating theseapproaches, let us introduce the main functionsneeded to carry out a planned abstract action.

Refinement In most systems, the plan steps pro-duced by the Planning component is not directlyexecutable as a robot command. The goto(room1)action requires sending commands to the robot toperceive the environment, plan the path, executeit avoiding dynamic obstacles, etc. The refinementprocess needs be context dependent in order to se-lect the most appropriate skills according to theonline observation. It should be able to consideralternative refinements in case of failure.Instantiation/Propagation Acting skills are oftenapplicable to a range of situations. Their modelsuse parameters whose value can be chosen at exe-cution time or observed in the environment. Cho-sen or observed value have to be propagated in therest of the skills down to the lowest level to issuecommands.Time management/Coordination Acting is per-formed in a close loop taking into account the dy-namics of the environment. Adequate responsesmust be given in a timely manner. Some systemsreason about time, deadlines as well as durations.Other systems handle a more symbolic represen-tation of time with concurrency, rendez-vous andsynchronization.Handling nondeterminism and uncertainty Ac-tions may have non-nominal effects. Furthermore,exogenous events in a dynamic environment areseldom predictable in a deterministic way. Finally,uncertainties in observations have to be taken intoaccount.Plan repair Some Acting approaches can repair theplan being executed. This is often performed us-ing part of the problem search space already de-veloped and explored by the Planner (hence, withan overlap between acting and planning). The ideais to solve new flaws or new threats that appearedduring execution by minimizing the changes inthe remaining of the plan. Even if repair may notbe more efficient that replanning, there are caseswhere part of the plan has already been sharedwith other agents (robots or humans) which expectthe robot to commit to it.

Another important aspect of Acting is how theskills are acquired and used. Are they completelyhand written or learned? Are they used directlyas programs or as specification models from whichfurther synthesis is performed? Finally, there is theconsistency verification issue between the Actingknowledge the Planning knowledge. We will seethat some of the proposed formalism for represent-

5

ing skills are more adapted to validation and ver-ification, even if this function is not always per-formed.

4.1. Procedure–based approaches

In procedure-based approaches, action refine-ment is done with handwritten skills. In RAP [27],each procedure is in charge of satisfying a partic-ular goal, corresponding to a planned action. De-liberation chooses the appropriate procedure forthe current state. The system commits to a goalto achieve, trying a different procedure when onefails. The approach was later extended with AP [7],integrating the planner PRODIGY [92] producingRAP procedures.

PRS [42] is an action refinement and monitor-ing system. As in RAP, procedures specify skills toachieve goals or to react to particular events andobservations. The system commits to goals andtries alternative skills when one fails. PRS relieson a database describing the world. It allows con-current procedure execution and multi-threading.Some planning capabilities have been added toPRS [19] to anticipate paths leading to executionfailure. PRS is used on various robotics platformsto trigger commands, e.g., through GenoM func-tional modules services [43].

Cypress [94] results from merging the plannerSipe with PRS. It uses a unified representation forplanning operators and PRS skills, which was ex-tended into the Continuous Planning and Execu-tion Framework (CPEF) [75]. CPEF includes sev-eral components for managing and repairing plans.The system has been deployed for military missionplanning and execution.

TCA [88] was initially developed to handle con-current planning and execution. It provides a hier-archical tasks decomposition framework, with ex-plicit temporal constraints to allow tasks synchro-nization. Planning is based on task decomposi-tion. It is mostly focused on geometrical and mo-tion planning (e.g., gait planning, footfall planningfor the Ambler robot). The Task Definition Lan-guage (TDL) [89] extends TCA with a wide rangeof synchronization constructs between tasks. It fo-cuses on task execution and relies on systems likeCasper/Aspen for planning.

XFRM [4] illustrates another approach whichuses transformation rules to modify hand writ-ten plans expressed in the Reactive Plan Language

(RPL). Unlike the above systems, it explores aplan space, transforming the initial RPL relying onsimulation and probabilities of possible outcomes.It replaces the currently executed plan on the fly ifanother one more adapted to the current situationis found. This approach evolved toward StructuredReactive Controllers (SRC) and Structure ReactivePlan (SRP) [3], but still retains the XFRM tech-nique to perform planning using transformationrules on SRP. It has been deployed on a number ofservice robots at Technical University of Munich.

Most procedure-based approaches focus on theRefinement and Instantiation/Propagation func-tions of an acting system. XFRM proposes a formof plan repair in plan space taking into accountthe probabilities of outcomes, while TDL providessome synchronization mechanism between skillsand commands. All skills used by these systemsare hand written, sometimes in a formalism sharedwith the planner (e.g., in Cypress and TCA), butwithout consistency checking. The hand writtenskills map to the robot commands, except forXFRM where some can be transformed online.

4.2. Automata–based approaches

It seems quite natural to express an abstractaction as a program whose I/O are the sensory-motor signals and commands. PLEXIL, a languagefor the execution of plans, illustrates such a rep-resentation where the user specifies node as com-putational abstraction [93]. It has been developedfor space applications and used with several plan-ners such as CASPER, but it remains fairly genericand flexible. A node can monitor events, executecommands or assign values to variables. It mayrefer hierarchically to a list of lower level nodes.Similarly to TDL, PLEXIL execution of nodes canbe controlled by a number of constraints (start,end), guards (invariant) and conditions. PLEXILremains very focused on the execution part. Butof the generated plan, it does not share knowledgewith the planner.

SMACH, the ROS execution system, offers anautomata-based approach [6]. The user providesa set of hierarchical automata whose nodes corre-sponds to components of the robot and the par-ticular state they are in. The global state of therobot corresponds to the joint state of all compo-nents. ROS actions, services and topics (i.e. mon-

6

itoring of state variables) are associated to au-tomata states, and according to their value, theexecution proceeds to the next appropriate state.An interesting property of Automata-based ap-proaches is that the Acting component knows ex-actly in which state the execution is, which easesthe deployment of the monitoring function.

Automata-based approaches focus on the coor-dination function. They can also be used for refine-ment and instantiation/propagation. Models arehand-written. However, the underlying formalismpermits possibly a validation function with au-tomata checking tools.

4.3. Logic–Based approaches

A few systems try to overcome the tedious engi-neering bottleneck of detailed hand specificationsof skills by relying on logic inference mechanismsfor extending high-level specifications. Typical ex-amples are the Temporal Action Logic (TAL) ap-proach (to which we’ll come back in section 5) andthe situation calculus approach. The latter is ex-emplified in GOLEX [37], an execution system forthe GOLOG planner.

In GOLOG and GOLEX the user specify respec-tively planning and acting knowledge in the sit-uation calculus representation. GOLEX providesProlog “exec” clauses which explicitly define thesequence of commands a robot has to execute. Italso provides monitoring primitives to check theeffects of executed actions. GOLEX executes theplan produced by GOLOG but even if the two sys-tems rely on the same logic programming represen-tation, they remain completely separated, limitingthe planning/execution interleaving.

The Logic-based approaches provides refinementand instantiation/propagation functions. But theirmain focus is on the logical specification of theskills, and the possibility to validate and verifytheir models. TAL (see section 5) offers also a Timemanagement handling.

4.4. CSP–based approaches

Most robotics applications require explicit timeto handle durative actions, concurrency and syn-chronization with exogenous events and otherrobots, and with absolute time. Several approaches

manage explicit time representations by extend-ing state-based representations with durative ac-tions (e.g., RPG, LPG, LAMA, TGP, VHPOP,Crickey). A few of them can manage concurrencyand, in the case of COLIN [15], even linear contin-uous change. However, temporal planners that relyon time-lines, i.e., partially specified evolution ofstate variables over time, are more expressive andflexible in the integration of planning and actingthan the standard extension of state-based plan-ners. Their representation ingredients are:• temporal primitives: point or intervals (tokens),

• state variables, possibly parametrized, e.g., po-sition(object32), and rigid relations, e.g., con-nected(loc3, room5),

• persistence of the value of a state variable overtime, and the discrete or continuous change ofthese values,

• temporal constraints: Allen interval relations orSimple Temporal Networks over time-points,

• atemporal constraints on the values and param-eters of state-variables.

The initial values, expected events and goalsare expressed in this representation as an un-explained trajectory, i.e., some required state-variable changes have to be accounted for by theplanner through actions. These are instances of op-erators whose preconditions and effects are simi-larly expressed by time-lines and constraints.

Planning proceeds in the plan-space by detect-ing flaws, i.e., unexplained changes and possibleinconsistencies, and repairing them through addi-tional actions and constraints. It makes use of var-ious heuristics, constraint propagation and back-track mechanisms. It produces partially specifiedplans, that have no more flaw but still containnon instantiated temporal and atemporal vari-ables. This least commitment approach has severaladvantages permitting to adapt the acting systemto the contingencies of the execution context.

The acting system proceeds like the plannerby propagating execution constraints, includingfor observable but non controllable variables (e.g.,ending time of actions). As for any CSP, consis-tency does not guaranty that all possible vari-able values are compatible. Hence the system keepschecking the consistency of the remaining plan,propagating new constraints and triggering a planrepair when needed. Special attention has to bepaid to observed values of non controllable vari-

7

Idle

Idle

Idle

Close

Robot_Nav

Human DinerNurse

dining_roomkitchen Goto

Robot_Manip Idle PickUp

MicroWave

Refrigerator Close Open Close

Close Closed-Cooking Open

PutDown PickUpHolding HoldingIdle PutDo

wn

Idle

Idle

Robot_Nav dining_roomkitchen Goto

Laser_model Idle

Motion Planner

Locomotion Traj. TrackIdle

Idle

Model

Idle

Manipulation Planner

Idle

Robot_Manip Idle PickUp

Gripper

Right_Arm Untuck Approach

Open

PutDown PickUpHolding HoldingIdle PutDown

Plan Motion

Open

Closing

Plan Idle

Idle Retract

Closed

Approach Idle Retract

Opening Open Clo

sing Closed Opening Open

Idle Approach Idle Retract

Plan Idle Plan Idle Plan Idle Plan Idle Plan

Idle Approach Idle Retract Tuck

Idle Idle Plan Idle Plan Idle

Mission Reactor

Navigation Reactor

Manipulation Reactor

Fig. 3. Example of an IDEA/T-ReX Plan built within three reactors (Mission reactor, Navigation reactor, Manipulation

reactor). Note the timelines Robot Nav and Robot Manip shared between reactors.

ables, depending on the requirements of strong,weak or dynamic controllability [72,16].

IxTeT [34] is an early temporal planner alongthis approach that was later extended with ex-ecution capabilities [62]. Planning and actingshare the partial plan structure which is dynam-ically produced during planning phase, and exe-cuted/repaired during the execution phase. An ex-ecution failure is seen as a new flaw in the partialplan space. The repair process minimizes as muchas possible the consequences of a failure on the restof the plan. Repair requires invalidating part ofthe current plan and/or relaxing some constraints.This is done by memorizing for each causal link thereason why it has been inserted (with or without atask). Causal links associated to tasks are not re-moved from the flawed plan. If the repair does notsucceed in some allocated time, the current plan isdiscarded and the planning is restarted from thecurrent situation. To fill the gap with robot com-mands and perceptions, PRS is used jointly withIxTeT for refinement and skills execution.

PS is another early timeline-based planning andacting system. As a component of the New Mil-lennium Remote Agent [44,77,74], it controlled theDeep Space One (DS1) probe for a few days. Vari-ous components were used on DS1, PS and EXECbeing the one of interest for this section (FDIR willbe presented in section 5). EXEC uses a procedure-based approach, as presented in section 4.1, with-

out execution constraint propagation and reason-ing on the current plan. In addition to its im-pressive application success in DS1, this systeminspired the development of two interesting ap-proaches: IDEA (then T-ReX) and RMPL.

IDEA [73] relies on a distributed approachwhere planning and acting use the same represen-tation and differ only in their prediction horizonand allocated computing time to find a solution.The system is distributed into a hierarchy of reac-tors, each being in charge of a particular deliber-ation function: e.g. mission planning, robot navi-gation, robot manipulation, payload management,etc; each has its own set of timelines, planninghorizon, and a computing time quantum. Reactorsuse the Europa planner to perform the constraint-based temporal planning. Two reactors may sharetimelines, accessed and modified by both, possi-bly with priorities. The timelines sharing mecha-nism allows the propagation of the planning re-sults down to commands and similarly the integra-tion from precepts to observations. For example,in Figure 3 the timeline robot nav in the missionreactor will specify on a shared timeline with thenavigation reactor the sequence of locations therobot must reach. From the mission reactor, thetimeline will be seen as an “execution” one, whilefor the navigation reactor it is a “goal”.

In principle the hierarchy of reactors should beable to express a continuum from planning oper-

8

ators down to commands. However, in practice,the shortest computing time quantum that couldbe achieved was in the order of a second, notfast enough for the command level. Hence, thatsystem had also to resort to hand programmedskills. Furthermore, the specification and debug-ging of action models distributed over several re-actors proved to be quite complex and tedious.

IDEA has been experimented with on severalplatforms such as the K9 and Gromit rovers [26]. Itled to the development, along a similar approach,of a system called T-ReX, deployed at MBARI forcontrolling UAVs [81]. T-ReX simplifies some ofIDEA too ambitious goals. For example in T-ReX,reactors are organized in such a way that con-straint propagation is guaranteed to reach a fixedpoint (which was not the case in IDEA). T-ReXalso tries to be planner independent and has beenused jointly with APSI [30], yet most implementa-tions use Europa [28].

Off

Untucked

Approached

Grasped

Holding

Robot ArmModel

Untuck

Approach

Grasp

Tuck

Tuck

Withdraw

Ungrasp

Failed

0.1

0.05

0.01

Off

Localized

Tracked

Object LocalizationModel

initStop

StandBy

Lost

0.1 Lost

Track

Tracking

Found

Searcht >=30s

Failed

Reinit

PickUpObject()::((do-watching (RA = holding) (do-watching (RA = Failed)

(parallel(RA = Untucked)(OL = Search))

(do-watching (OL = Failed)(OL = Localized))

(when-donext (OL = Localized)(parallel

(RA = Approached)(OL = Tracked))

(when-donext ((OL = Tracked) and (RA = Approached))(RA = Grasped)))

);;; <Arm recovery>

) ;;; Success)

Fig. 4. Example of an RMPL automata (system model onthe left) and program (control model on the right). Note

the non nominal outcomes (with probabilities) of actions

in the system model, and the coordination operators in thecontrol model.

RMPL (for Reactive Model-based ProgrammingLanguage) [41], another spinoff of the DS1 experi-ment, proposes a common representation for plan-ning, acting and monitoring. It combines a systemmodel with a control model (Figure 4). The for-mer uses hierarchical automata to specify nominalas well as failure state transitions, together withtheir constraints. The latter uses reactive program-ming constructs (including primitives to addressconstraint-based monitoring, as in Esterel [18]).Moreover, RMPL programs are transformed intoTemporal Plan Networks (TPN) [95]. The resultof each RMPL program is a partial temporal planwhich is analyzed by removing flaws and trans-

formed for execution taking into account onlinetemporal flexibility. Altogether, RMPL offers aninteresting and original integration of state-basedmodels, procedural control and temporal reason-ing used in satellite control applications.

CSP approaches are very convenient for han-dling time. They provide refinement, instantiationand, for some of them, plan repair (IxTeT, IDEA,T-ReX and RMPL). They also rely on hand writ-ten models of skills which are handled by vari-ous CSP algorithms (STN, constraints filtering).RMPL manages also nondeterminism by modelingnon nominal transitions of the system.

4.5. Stochastic–based approaches

The classical framework of Markov DecisionalProcesses (MDPs) offers an appealing approach forintegrating planning and acting. It naturally han-dles probabilistic effects and it provides policies,i.e., universal plans, defined everywhere. The ex-ecution of a policy is a very simple loop: (i) ob-serve current state, then (ii) apply correspondingaction. It can even be extended, in principle, topartially observable systems (POMDP), as illus-trated in the Pearl system of [79]. That frameworkworks fine as long as the state space, together withits cost and probability parameters, can be en-tirely acquired and explicited, and, for POMDPs,remains of small size.4 However, most deliberationproblems in robotics do not allow for an explicitenumeration of their state space, and hence cannotafford a universal plan. Fortunately, most of theseproblems are usually focused on reaching a goalfrom some initial state s0. Factorized and hierar-chical representations of MDPs [9], together withheuristic search algorithms for Stochastic ShortestPath (SSP) problems [65], offer a promising per-spective for using effectively stochastic representa-tions in deliberate action.

SSP problems focus on partial policies, closed forthe initial state s0 (i.e., defined on all states reach-able from s0), terminating at goal states. Theygeneralize to And/Or graphs classical path searchproblems. For very large implicit search spaces,based on sparse models (few applicable actions

4A POMDP is an MDP on the belief space, whose size isexponential in that of the state space. The latter is already

of size kn, for a domain with n state variables.

9

Approaches

ProcedureProcedureProcedureProcedureProcedure

AutomataGraph

AutomataGraph

Logic

CSPCSPCSPCSP

StochasticStochasticStochasticStochastic

FunctionsFunctionsFunctionsFunctionsFunctions

Systems Refinement Instantiation Time handling Nondeterminism Repair

RAP X XPRS X XCypress/CPEF X XTCA/TDL X X XXFRM/RPL/SRP X X X XPLEXIL X X XSMACH X X XGolex X XIxTeT X X X XRMPL X X X X XIDEA/T-ReX X X X XCasper X X X XMCP XPearl XRobel XRessac X

Table 1

Illustrative examples of acting approaches and functions.

per state, few nondeterministic effects per applica-ble action, including deterministic actions), a sig-nificant scaling up with respect to classical dy-namic programming methods can be achieved withheuristics and sampling techniques [65].

Most heuristic search algorithms for SSPs arebased on a two steps Find&Revise general frame-work: (i) Find an unsolved state s in the succes-sors of s0 with current policy, and (ii) Revise theestimated value of s along its current best action(with the so-called Bellman update). A state s issolved when the best (or a good) goal reaching pol-icy from s has already been found. That frameworkcan be instantiated in different ways, e.g.,• with a best-first search, as in AO*, LAO* and

their extensions (ILAO*, BLAO*, RLAO*, etc.)

• with a depth-first iterative deepening search, asin LDFS

• with a random walk along current greedy pol-icy, as in RTDP, LRTDP and their extensions(BRTDP, FRTDP, SRTDP, etc.)

These algorithms assume an SSP problem with aproper policy closed for s0 (i.e., one that reaches agoal state from s0 with probability 1) where everyimproper policy has infinite cost. A generalizationrelaxes this last assumption and allows to seek apolicy that maximizes the probability of reaching agoal state, a very useful and desirable criteria [52].Other issues, such as dead-ends (states from whichits not possible to reach a goal) have to be takencare of, in particular in critical domains [50].

Heuristic search algorithms in SSPs are morescalable than dynamic programming techniquesfor MDP planning, but they still cannot addresslarge domains, with hundreds of state variables,unless these domains are carefully engineered anddecomposed. Even a solution policy for such prob-lems can be of a size so large as to make its enu-meration and memorization challenging to cur-rent techniques. However, such a solution containsmany states of very low probability that would al-most never be visited. Various sampling and ap-proximation techniques offer promising alterna-tives to further scale up probabilistic planning.

Among these approaches, determinization tech-niques transform each non-deterministic actionsinto several deterministic ones (the most likely orall possible ones), then it plans deterministicallywith these actions, online and/or offline. For ex-ample, the RFF planner [90] generates an initialdeterministic plan, then it considers a fringe statealong a non-deterministic branch of that plan:if the probability to reach that state is above athreshold, it extends the plan with a deterministicpath to a goal or to an already solved state.

Similar ideas are developed in sampling ap-proaches. Among their advantages is the capabil-ity to work without a priori estimates of the prob-ability distributions of the domain, as long as thesampling is drawn from these same distributions.Bounds on the approximation quality and the com-plexity of the search have been obtained, with good

10

results on various extensions of algorithms such asLRTDP and UCT , e.g. [49,11,51].

Although MDPs are often used in robotics atthe sensory motor level, in particular within re-inforcement learning approaches, SSP techniquesare not as widely disseminated at the delibera-tive planning and acting level. Contributions aremostly on navigation problems, e.g., the RESSACsystem [91]. On sparsely nondeterministic domainswhere most actions are deterministic but of a feware probabilistic, the approach called MCP [64] re-duces with deterministic planning a large probleminto a compressed MDP. It has tested on a simu-lated multi-robot navigation problem.

Finally, let us mention promising heterogeneousapproaches where task planning is deterministicand SSP techniques are used for the choice of thebest skill refining an action, given the current con-text. An illustration is given by the ROBEL sys-tem [71] with a receding horizon control.

5. Monitoring

The monitoring function is in charge of (i) de-tecting discrepancies between predictions and ob-servations, (ii) classifying these discrepancies, and(iii) recovering from them. Monitoring has at leastto monitor the planner’s predictions supportingthe current plan. It may have also to monitor pre-dictions made when refining plan steps into skillsand commands, as well as to monitor conditionsrelevant for the current mission that are left im-plicit in planning and refinement steps. The latterare, for example, how calibrated are the robot’ssensors, or how charged are its batteries.

Although monitoring functions are clearly dis-tinct from action refinement and control functions,in many cases the two are implemented by thesame process with a single representation. For ex-ample, the early Planex [25] performs a very sim-ple monitoring through the iterated computationof the current active kernel of a triangle table.In most procedure-based systems there are PRS,RAP, ACT or TCA constructs that handle somemonitoring functions. However, diagnosis and re-covery functions in such systems are usually lim-ited and ad hoc.

Diagnosis and recovery are critical in appli-cations like the DS1 probe, for which FDIR, acomprehensive monitoring system, has been de-

veloped [74]. The spacecraft is modeled as a finegrained collection of components, e.g., a thrustvalve. Each component is described by a graphwhere nodes are the normal functioning states orfailure states of that component. Edges are ei-ther commands or exogenous transition failures.The dynamics of each component is constrainedsuch that at any time exactly one nominal transi-tion is enabled but zero or more failure transitionsare possible. Models of all components are com-positionally assembled into a system allowing forconcurrent transitions compatible with constraintsand preconditions. The entire model is compiledinto a temporal propositional logic formula whichis queried through a solver. Two query modes areused: (i) diagnosis, i.e., find most likely transitionsconsistent with the observations, and (ii) recovery,i.e., find minimal cost commands that restore thesystem into a nominal state. This approach hasbeen demonstrated as being effective for a space-craft. However, it is quite specific to cases wheremonitoring can be focused on the robot itself, noton the environment, and where reliability is a criti-cal design issue addressed through redundant com-ponents permitting complex diagnosis and allow-ing for recovery actions. It can be qualified as a ro-bust proprioceptive monitoring approach. It is un-clear how it could handle environment discrepan-cies, e.g., a service robot failing to open a door.

Other robotics monitoring systems are surveyedin [78] and characterized into three classes: an-alytical approaches, data-driven approaches andknowledge-based approaches. The former rely onplanning and acting models, such as those men-tioned above, but also control theory models andfiltering techniques for low-level action monitoring.Data-driven approaches rely on statistical cluster-ing methods for analyzing training data of normaland failures cases, and pattern recognition tech-niques for diagnosis. Knowledge-based approachesexploit specific knowledge in different representa-tions (rules, chronicles, neural nets, etc.), which isgiven or acquired for the purpose of monitoringand diagnosis. This classification of almost 90 dif-ferent contributions to Monitoring in robotics isinspired from the field of industrial control, whereMonitoring is a well studied issue. However, the re-lationship between Monitoring, Planning and Act-ing was not a major concern in the surveyed con-tributions.

That relationship is explored in [29] on the ba-sis of plan invariants. Several authors have synthe-

11

sized state-reachability conditions, called invari-ants, from the usual planning domain specifica-tions. Invariants permit a focus and a speed-upof planning algorithms, e.g., [84,5]. Going further,[29] proposes extended planning problems, wherethe specifications of planning operators are aug-mented by logical formula stating invariant con-ditions that have to hold during the execution ofa plan. Indeed, planning operators and extendedinvariants are two distinct knowledge sources thathave to be modeled and specified distinctly. Theseextended invariants are used to monitor the execu-tion of a plan. They allow to detect infeasible ac-tions earlier then their planned execution, or vio-lated effects of action after their successful achieve-ment. Furthermore, extended invariants allow tomonitor effects of exogenous events and other con-ditions not influenced by the robot. However, thisapproach assumes complete sensing and perfectobservation function. No empirical evaluation hasbeen reported.

Along the same line, the approach of [24] hasbeen tested on a simple office delivery robot. It re-lies on a logic-based representation of a dynamicenvironment using the fluent calculus [86]. Actionsare described by normal and abnormal precon-ditions. The former are the usual preconditions.The latter are assumed away by the planner asdefault; they are used as a possible explanationby the monitor in case of failure. E.g., deliveryof an object to a person may fail with abnormalpreconditions of the object being lost or the per-son not being traceable. Similarly, abnormal effectsare specified. Discrepancies between expectationsand observations are handled by a prioritized non-monotonic default logic, which generates explana-tions ranked using relative likelihood. That systemhandles incomplete world model and observationupdates performed either while acting or on de-mand from the monitoring system through specificsensory actions.

The idea of using extended logical specificationsfor Planning and Monitoring has been explored byseveral others authors in different settings. The in-teresting approach of [8] uses domain knowledgeexpressed in description logic to derive expecta-tions of the effects of actions in a plan to be moni-tored during execution. An interesting variant is il-lustrated in [60] for a hybrid architecture, combin-ing a behavior-based reactive control with model-based deliberation capabilities. At each cycle, con-

current active behaviors are combined into low-level controls. At a higher level, properties of therobot behaviors are modeled using Linear Tempo-ral Logic (LTL). LTL formula express correctnessstatements, execution progress conditions, as wellas goals. A trace of the robot execution, observedor predicted at planning time, is incrementallychecked for satisfied and violated LTL formula.A delayed formula progression technique evaluatesat each state the set of pending formula. It re-turns the set of formula that has to be satisfied byany remaining trace. The same technique is usedboth for Planning (with additional operator mod-els and some search mechanism) and for Monitor-ing. The approach has been tested on indoor nav-igation tasks with robots running the Saphira ar-chitecture [54].

A very comprehensive and coherent integrationof Monitoring to Planning and Acting is illustratedin the approach used in the Witas project [21].That system demonstrates a complex Planning,Acting and Monitoring architecture embedded onautonomous UAVs. It has been demonstrated insurveillance and rescue missions. Planning relies ofTALplanner [58], a forward chaining planner usingthe Temporal Action Logics(TAL) formalism forspecifying planning operators and domain knowl-edge. Formal specifications of global constraintsand dependencies, as well as of operator modelsand search recommendations, are used by the plan-ner to control and prune the search. These spec-ifications are also used to automatically generatemonitoring formula from the model of each oper-ator, and from the complete plan, e.g., constraintson the persistence of causal links. This automatedsynthesis of monitoring formula is not systematicbut rather selective, on the basis of hand pro-grammed conditions of what needs to be moni-tored and what doesn’t. In addition to the plan-ning domain knowledge, extra monitoring formulaare also specified in the same highly expressivetemporal logic formalism.

The TAL-based system produces plans withconcurrent and durative actions together with con-ditions to be monitored during execution. Theseconditions are evaluated on-line, at the rightmoment, using formula progression techniques.When actions do not achieve their desired results,or when some other conditions fail, a recoverythrough a plan repair phase is triggered. Actingis performed by Task Procedures, which provide

12

some level of action refinement through classicalconcurrent procedural execution, down to elemen-tary commands. Altogether, this system proposesa coherent continuum from Planning to Acting andMonitoring. The only component which does notseem to rely on formal specifications is the Act-ing function which uses hand written Task Proce-dures. However the lack of flexible action refine-ment is compensated for by specifying planningoperators (and hence plan steps) at a low-level ofgranularity. For example, there are five differentfly operators in the UAV domain corresponding todifferent contexts, specifying context-specific con-trol and monitoring conditions, and being mappedto different Task Procedures.

6. Perceiving

Situated deliberation relies on data reflectingthe current state of the world. Beyond sensing, per-ceiving combines bottom-up processes from sen-sors to interpreted data, with top-down focus of at-tention, search and planning for information gath-ering actions. Perceiving is performed at:• the signal level, e.g., signals needed in control

loops ,

• the state level : features of the environment andthe robot and their link to facts and relationscharacterizing the state of the world, and

• the history level, i.e., sequences or trajectoriesof events, actions and situations relevant for therobot’s mission.The signal level is usually dealt with through

models and techniques of control theory. Visualservoing approaches [13] for tracking or handlingobjects and moving targets offer a good exam-ple of mature techniques that can be consideredas tightly integrated into the basic robot func-tions. Similarly for simultaneous localization andmapping techniques, a very active and well ad-vanced field in robotics, to which numerous publi-cations have been devoted, e.g., [2,69]. These geo-metric and probabilistic techniques, enriched withtopological and semantic data, as for example in[56,57,53], may involve deliberation and can bequite effective.

But of the above areas, methods for design-ing perceiving functions remain today a limitingfactor in autonomous robotics, a hard and chal-lenging issue to which surprisingly not enough ef-

forts have been devoted. The building blocks forsuch a function can to be taken from the fields ofsignal processing, pattern recognition and imageanalysis, which offer a long history of rich devel-opments. However, the integration of these tech-niques within the requirements of autonomy anddeliberation remains a bottleneck.

The anchoring problem provides an excellent il-lustration of the complexity of integrating patternrecognition methods with autonomous deliberateaction. As defined in [17], anchoring is the problemof creating and maintaining over time a correspon-dence between symbols and sensor data that re-fer to the same physical object. Planning and otherdeliberation functions reason on objects throughsymbolic attributes. It is essential that the sym-bolic description and the sensing data agree aboutthe objects they are referring to. Anchoring con-cerns specific physical objects. It can be seen asa particular case of the symbol grounding prob-lem, which deals with broad categories, e.g., any“chair”, as opposed to that particular chair-2. An-choring an object of interest can be achieved byestablishing and keeping an internal link, calledan anchor, between the perceptual system andthe symbol system, together with a signature thatgives estimate of some of the attributes of the ob-ject it refers to. The anchor is based on a modelthat relates relations and attributes to perceptualfeatures and their possible values.

Establishing an anchor corresponds to a patternrecognition problem, with the challenges of han-dling uncertainty in sensor data and ambiguity inmodels, dealt with for example through maintain-ing multiple hypotheses. Ambiguous anchors arehandled in [47] as a planning problem in a space ofbelief states, where actions have causal effects thatchange object properties, and observation effectsthat partition a belief state into several new hy-potheses. There is also the issue of which anchorsto establish, when and how, in a bottom-up or atop-down process. Anchors in principle are neededfor all objects relevant to the robot mission. Theseobjects can only be defined by intension (not ex-tensively), in a context-dependent way. There isalso the issue of tracking anchors, i.e., taking intoaccount objects properties that persist across timeor evolve in a predictable way. Predictions are usedto check that new observations are consistent withthe anchor and that the updated anchor still sat-isfies the object properties. Finally, reacquiring an

13

anchor when an object is re-observed after sometime is a mixture of finding and tracking; if theobject moves it can be quite complex to accountconsistently of its behavior.

The dynamics of the environment is a strongsource of complexity, e.g., as we just saw in theanchor tracking and re-acquiring problems. Thisdynamics is itself what needs to interpreted forthe history level: what an observed sequence ofchanges means, what can be predicted next frompast evolutions. In many aspects, research at thishistory level is more recent. It relates to acting inand understanding environments with rich seman-tics, in particular involving human and man–robotinteractions, e.g., in applications such as robot pro-gramming by demonstration [1] or video surveil-lance [40,31].

The survey of [55] covers an extensive list of con-tributions to action and plan recognition. Theseare focused on (i) human action recognition, (ii)general activity recognition, and (iii) plan recog-nition level. The understanding is that the formertwo sets of processing provide input to the latter.Most surveyed approaches draw from two sourcesof techniques:• Signal processing : Kalman and other filtering

techniques, Markov Chains, Hidden Markov Mod-els. These techniques have been successfully usedin particular for movement tracking and gesturerecognition[97,67].

• Plan recognition: deterministic [48,83] or proba-bilistic [32] planning techniques, as well as pars-ing techniques [82].Most plan recognition approaches assume to get

as input a sequence of symbolic actions. This as-sumption is reasonable for story understandingand document processing applications, but it doesnot hold in robotics. Usually actions are sensedonly through their effects on the environment.

The Chronicle recognition techniques [22,33] arevery relevant at the history level of the Perceiv-ing function. A chronicle recognition system is ableto survey a stream of observed events and recog-nize, on the fly, instances of modeled chroniclesthat match this stream. A chronicle is a model fora collection of possible scenarios. It describes pat-terns of observed events, i.e., change in the value ofstate variables, persistence assertions, non occur-rence assertions and temporal constraints betweenthese assertions. A ground instance of a chroniclecan be formalized as a nondeterministic timed au-

tomata. Chronicles are similar to temporal plan-ning operators. The recognition is efficiently per-formed by maintaining incrementally a hypothe-sis tree for each partially recognized chronicle in-stance. These trees are updated or pruned as newevents are observed or as time advances. Recentdevelopment have added hierarchization and focuson rare events with extended performances [23].

Very few systems have been proposed for design-ing and implementing a complete Perceiving func-tion, integrating the three levels mentioned earlierof signal, state and history views. DyKnow [39]stands as a clear exception, noteworthy by its com-prehensive and coherent approach. This systemaddresses several requirements: the integration ofdifferent sources of information, of hybrid symbolicand numeric data, at different levels of abstrac-tion, with bottom-up and top-down processing; itmanages uncertainty, reasons on explicit models ofits content and is able to dynamically reconfigureits functions.

These challenging requirements are addressed asa data-flow based publish-and-subscribe middle-ware architecture. DyKnow views the environmentas consisting of objects described by a collectionof features. A stream is a set of time-stamped sam-ples representing observations or estimations of thevalue of a feature. It is associated with a formallyspecified policy giving requirements on its contentsuch as: frequency of updates, delays and ampli-tude differences between two successive samples,or how to handle missing values.

A stream is generated by a process whichmay offer several stream generators synthesizingstreams according to specific policies. Processeshave streams as input and output. They are of dif-ferent types, such as primitive processes, that aredirectly connected to sensors and databases; re-finement processes, that subscribe input streamsand provide as output more combined features,e.g., a signal filter or a position estimator fusingseveral raw sensing sources and filtered data; orconfiguration processes that allow to reconfiguredynamically the system by initiating and removingprocesses and streams, as required by the task andthe context, e.g., to track a newly detected target.

DyKnow uses a specific Knowledge ProcessingLanguage (KPL) to specify processes, streams andcorresponding policies. KPL allows to refer to ob-jects, features, streams, processes, and time, to-gether with their domains, constraints and rela-

14

tionships in the processing network. Formal spec-ifications in KPL defines a symbol for each com-putational unit, but they do not define the ac-tual function associated with this symbol. Theirsemantics is taken with respect to the interpre-tation of the processing functions used. They al-low to describe and enforce streams policies. Theyalso support a number of essential functions, e.g.,synchronize states from separate unsynchronizedstreams; evaluate incrementally temporal logic for-mulas over states; recognize objects and build upanchors to classify and update interpretation asnew information becomes available; or follow his-tories of spatio-temporal events and recognize oc-currences of specified chronicle models.

DyKnow has been integrated to the TALplan-ner system [21] discussed earlier. This system isqueried by planning, acting and Monitoring func-tions to acquire information about the current con-textual state of the world. It provides appropriateand highly valuable focus of attention mechanisms,linking monitoring or control formulas to streams.It has been deployed within complex UAV rescueand traffic surveillance demonstration.

7. Goal reasoning

!"#$ %&'()&*$ *&+&%('$ ,-$ ./&*&$ (**012.3,4*$*310'.(4&,0*'56$34$7,4.%(*.$.,$1(45$&--,%.*$./(.$-,70*$,4$%&'()348$,4'5$*,1&$*09*&.$,-$./&*&$(**012.3,4*:$$!"#"$%&'&(#&)* "'+&$,'%"'#(;$ <'(**37('$ 2'(44348$(**01&*$ ./(.$ &(7/$ -0.0%&$ *.(.&$ 3*$ =&.&%134&=$ 95$ ./&$(7.3,4$ &)&70.&=$ 34$ ./&$ 70%%&4.$ *.(.&:$ -,'#&'."')/*012''&'.$%&'()&*$./3*$95$8&4&%(.348$7,4=3.3,4('$2'(4*$./(.$ (%&$ &)&70.&=$ ,4'5$ >/&4$ (4$ (7.3,4$ =,&*$ 4,.$(7/3&+&$ 3.*$ 34.&4=&=$ &--&7.*$ ?"&(%=&4$ "#* 213$ @AABC:$D47&%.(34.5$ 34$ -0.0%&$ *.(.&$ 2%&=37.3,4$ 3*$ ,-.&4$7(2.0%&=$ (*$ 02$#&21* ,4("$+24&1&#/6$ >/37/$ 7(4$ 9&$1,=&'&=$0*348$E(%F,+$=&73*3,4$2%,7&**&*$?G0.&%1(4$HIIJC:$ K3F/(7/&+$ (4=$ L.&4.M$ ?@AAIC$ ,9*&%+&$ ./(.$./&*&$(22%,(7/&*$*7('&$2,,%'56$(4=$7(44,.$347,%2,%(.&$=,1(34N*2&73-37$ /&0%3*.37$ F4,>'&=8&$ (9,0.$ ./&$&4+3%,41&4.:$ O,$ (==%&**$ ./3*6$ ./&3%$ G<GG$ 2'(44&%$34*.&(=$ %&(*,4*$ (9,0.$ 2%&-&%&47&*$9&.>&&4$04F4,>4$+('0&*$,-$./&$*.(.&$>/&4$8&4&%(.348$./&$2'(4:$5#2#&)*"'+&$,'%"'#(;$#4,./&%$7'(**37('$(**012.3,4$ 3*$./(.$ ./&$ &4+3%,41&4.$ =,&*$ 4,.$ 7/(48&$ ,./&%$ ./(4$./%,08/$ ./&$ &)&70.3,4$ ,-$ (8&4.$ (7.3,4*:$ G'(4$1,43.,%348$ 7(4$ 9&$ 0*&=$ .,$ =&.&7.$ 7/(48&*$ 34$ ./&$&4+3%,41&4.$./(.$7(4$7(0*&$2'(4$-(3'0%&:$P,%$&)(12'&6$&')$"%"'#21* 012''"$(* 2'(4$ -,%$ ($ -3)&=$ .31&$ /,%3M,46$&)&70.&$ ./&$2'(46$(4=$./&4$8&4&%(.&$($4&>$2'(4$-%,1$./&$70%%&4.$ *.(.&:$O/3*$2%,7&**$7,4.340&*$04.3'$ ($8,('$*.(.&$ 3*$ %&(7/&=:$ P,%$ &)(12'&6$ <GQP$ ?E5&%*$ HIIIC$8&4&%(.&*$2'(4*$ .,$(7/3&+&$(3%$ *02&%3,%3.5$ 34$13'3.(%5$7,19(.$ (4=$ %&2'(4*$ >/&4$ 04&)2&7.&=$ &+&4.*$ ,770%$=0%348$ &)&70.3,4$ ?&:8:6$ ($ 2'(4&$ 3*$ */,.$ =,>4C:$ L,1&$%&7&4.$ (22%,(7/&*$ 34*.&(=$ -,70*$ *,'&'5$ ,4$ =54(137$%&2'(44348$ ?&:8:6$ R,OS3"Q$ %&8&4&%(.&*$ ,4'5$ 2(%.$ ,-$3.*$2'(4$>/&4$(4$(7.3,4$-(3'*$?#5(4$"#*21:$@AATCC:$$!&()$"#"* "66")#(;$ <,12'&)$ &4+3%,41&4.*$ (%&$ *09U&7.$

/&('./6$ (4=$ -0&'$ (''$ 7(4$ 7,4.340,0*'5$ 7/(48&$ ,+&%$.31&:$R,>&+&%6$ -&>$ *5*.&1*$ 7(4$2%,7&**$ 7,4.340,0*$&--&7.*$ ?&:8:6$<VKWX$7(4$2'(4$0*348$=0%(.3+&$(7.3,4*$$>3./$'34&(%$7,4.340,0*$&--&7.*$?<,'&*$"#*2136$@AAICC:$$5#2#&)* .,21(;$ <'(**37('$ 2'(44348$ (**01&*$ ./(.$ ./&$8,('*$ (%&$ (''N,%N4,./348$ (4=$ *.(.37:$ W-$ 4,$ 2'(4$ 7(4$(7/3&+&$ (''$ ,-$ ./&$ 8,('*6$ ./&4$ 7'(**37('$ 2'(44&%*$>3''$-(3':$72$#&21* (2#&(62)#&,'* 012''&'.$ ?GLGC$ %&'()&*$ ./3*$(''N,%N4,./348$ 7,4*.%(34.6$ (4=$ 34*.&(=$ -,70*&*$ ,4$8&4&%(.348$ 2'(4*$ ./(.$ (7/3&+&$ *,1&$8,('*$?3:&:6$./&$2'(4$./(.$83+&*$./&$1()3101$.%(=&N,--$9&.>&&4$ .,.('$ (7/3&+&=$ 8,('$ 0.3'3.3&*$ (4=$ .,.('$3470%%&=$ (7.3,4$ 7,*.C$ ?+(4$ =&4$ Y%3&'$ "#* 213$ @AAJC:$V./&%$ %&*&(%7/&%*$ /(+&$ (==%&**&=$ ./&$ '313.(.3,4*$ ,-$*.(.37$ 8,('*:$ P,%$ &)(12'&6$ <,==348.,4$ (4=$ K07F$?@AAJC$ 9&*.,>&=$ (8&4.*$ >3./$ %,#&+2#&,'(6$ >/37/$8&4&%(.&$ 8,('*$ 34$ %&*2,4*&$ .,$ *2&73-37$ *.(.&*:$ P,%$

9(..&%5$7/(%8&$-(''*$9&',>$ZA[6$./&4$ ($ 8,('$ .,$ (..(34$ ($ -0''$ 9(..&%5$ 7/(%8&$ >3''$ 9&$8&4&%(.&=$ ?E&4&80MM3$ (4=$ K07F$ @AATC:$ #4,./&%$(22%,(7/$ 3*$ .,$ ('',>$ -,%$ 8,('*$ .,$ %&-&%&47&$ ,9U&7.*$./(.$ (%&$ 04F4,>4$ (.$ 2'(44348$ .31&:$ 80"'* 9,$1:*

;<2'#&6&":* .,21($ 7,1934&$ 34-,%1(.3,4$ (9,0.$ *&4*348$,9U&7.*$ (4=$ 8&4&%(.348$ 8,('*$ 34.,$ (4$ &)3*.348$ GLG$*5*.&1$ ?O('(1(=020'($ "#* 213$ @AAIC:$ L313'(%'56$!,'=1(4$?@AAIC$=&*7%39&*$($*5*.&1$>3./$043+&%*(''5$\0(4.3-3&=$ 8,('*$ ./(.$ ('',>*$ 2'(44348$ -,%$ *&.*$ ,-$&4.3.3&*$ >/,*&$ 7(%=34('3.5$ 3*$ 04F4,>4$ (.$ 2'(44348$.31&:$ L&+&%('$ *5*.&1*$ ?&:8:6$ GQ<#L$ ?R(>&*$ &.$ ('6$@AAICC$ 8&4&%(.&$ 8,('*$ (.$ &)&70.3,4$ .31&$ 9(*&=$ ,4$ ($

$$ #'./,08/$ ./&*&$ (**012.3,4*$ 7/(%(7.&%3M&$ 7,12'&)$&4+3%,41&4.*6$4,4&$,-$./&*&$2%&+3,0*$&--,%.*$%&'()$(''$-,0%$*310'.(4&,0*'56$>/37/$3*$./&$-,70*$,-$!"#:$$$ O/&%&$ 3*$ ($ %37/$ /3*.,%5$ ,-$ =&+&',2348$ (8&4.$(%7/3.&7.0%&*$ -,%$ 347%&(*348'5$ *,2/3*.37(.&=$&4+3%,41&4.*6$ &:8:6$ O#<#WSNLV#S$ ?],4&*$ "#* 213$HIIIC:$ D4'3F&$ %&(7.3+&$ (%7/3.&7.0%&*6$ *07/$ (*$W<#SDL$ ?K(48'&5$ (4=$ </,3$ @AA^C6$ !"#$ *&2(%(.&*$&4+3%,41&4.('$ (4=$ 8,('$ %&(*,4348$ -%,1$ (7.3,4$*&'&7.3,46$ >/37/$ 2&%13.*$ (==3.3,4('$ %&-'&7.3,4$ (*$%&\03%&=:$$S&7&4.'56$</,3$?@AHAC$/(*$9&&4$>,%F348$,4$&).&4*3,4*$ .,$ ./&$ W7(%0*$ (%7/3.&7.0%&$ >/37/$ 7%&(.&$8,('*$ 0*348$ 7,4*.%(34.N'3F&$ 8,('$ =&*7%32.3,4*6$P0%./&%1,%&6$ ./&$ 8,('*$ 7,4*3=&%&=$ /&%&$ 1(5$ =3--&%$*09*.(4.3(''5$-%,1$./&$70%%&4.$8,('*$(4=$7,4*&\0&4.'5$*/,0'=$ 4,.$ 9&$ 7,4*3=&%&=$ *098,('*:$ O/&5$ 1(5$ 9&$(0.,4,1,0*'5$8&4&%(.&=6$(4=$34+,'+&$,9U&7.*$./(.$(%&$4,.$ F4,>4$ ,%$ (+(3'(9'&$ 04.3'$ &)&70.3,4$ .31&:$ _&$=&.(3'$./&$!"#$-%(1&>,%F$34$L&7.3,4$B:$$

!"!#$%&'()*+,-.#/01%.%23##<,) $?@AATC$WXOSV$*5*.&1$2%,+3=&*$./&$34*23%(.3,4$-,%$*&+&%('$7,47&2.*$34$8,('N=%3+&4$(0.,4,15$>3./$3.*$-,70*$ ,4$ 34.&8%(.&=$ 2'(443486$ &)&70.3,46$ (4=$ 8,('$%&(*,4348:$_&$&).&4=$ ./&*&$ 3=&(*$(4=$7,4*3=&%$ ./&1$(*$($8&4&%('$(8&4.$-%(1&>,%F:$!"#$ 3*$($7,47&2.0('$1,=&'$ ,-$,4'34&$2'(44348$ 34$

(0.,4,1,0*$ (8&4.*:$ P380%&$ H$ 3''0*.%(.&*$ /,>$ !"#!

4+50*-#6;$#$<,47&2.0('$E,=&'$-,%$!,('N"%3+&4$#0.,4,15!

1549

Fig. 5. GDA Model with its different components, from [68].

Goal reasoning is mostly concerned with themanagement of high-level goals and the global mis-

sion. Its main role is to manage the set of objec-tives the system wants to achieve, maintain or su-pervise. It may react to new goals given by theuser or to goal failure reported acting and mon-itoring. In several implementations, this functionis embedded in the planning or acting functions.It clearly shares similarities with the monitoringfunction. Still, Goal Reasoning is not akin to plan-ning as it does not really produce plan, but merelyestablish new goals and manage existing one whichare then passed to the planner. Similarly to mon-itoring, it continuously checks unexpected eventsor situations. These are analyzed to assess currentgoals and possibly establish new goals. Some sys-tems have a dedicated component to perform thishigh-level function. For example, Goal Driven Au-tonomy (GDA) approaches model and reason onvarious and sometime conflicting goals an agentmay have to consider. GDA reasoning focus ongoal generation and management. In [68], the au-thors instantiate the GDA model in the ARTUEagent which appropriately responds to unexpectedevents in complex simulations and games environ-ment. As shown on figure 5, their system includesa classical planner; when it executes a plan, itdetects discrepancy (Discrepancy Detector), gen-erates an explanation, may produce a new goal(Goal Formulator) and finally manages the goalscurrently under consideration by the system. TheGoal Manager can use different approaches to de-cide which goal to keep (e.g., using decision theoryto balance conflicting goals).

Similarly, in [80] the authors point out that plan-ning should be considered from a broader point ofview and not limited to the sole activity of generat-ing an abstract plan with restrictive assumptionsintroduced to scope the field and make the prob-lem more tractable. They propose the Plan Man-agement Agent (PMA) which, beyond plan gener-ation, provides extra plan reasoning capabilities.The resulting PMA system heavily relies on tem-poral and causal reasoning, and is able to plan withpartial commitments, allowing to further refine aplan when needed.

Goal reasoning has been deployed in a numberof real experiments. Notably in the DS1 New Mil-lenium Remote Agent experiment [74] and in theCPEF framework [75]. Yet, overall, the goal rea-soning function is not often developed. It is never-theless needed for complex and large systems man-aging various long term objectives while taking dy-namically into account new events which may trig-ger new goals.

15

8. Integration and Architectures

Beyond the integration of various devices (me-chanical, electrical, electronical, etc), robots arecomplex systems including multiple sensors, actu-ators and information processing modules. Theyembed online processing, with various real time re-quirement, from low-level servo loops up to delib-eration functions which confer the necessary au-tonomy and robustness for the robot to face thevariability of tasks and environment. The softwareintegration of all these components must rely onan architecture and supporting tools which spec-ify how theses components communicate, share re-sources and CPUs, and how they are implementedon the host computer(s) and operating systems.

Various architectures have been proposed totackle this task, among which the following:• Reactive architectures, e.g. the subsumption ar-

chitecture [10], are composed of modules whichclose the loop between inputs (e.g. sensors) andoutputs (e.g. effectors) with an internal au-tomata. These modules can be hierarchically or-ganized and can inhibit other modules or weighton their activity. They do not rely on any partic-ular model of the world or plans to achieve anddo not support any explicit deliberative activi-ties. Nevertheless, there are a number of work,e.g. [59], which rely on them to implement delib-erative functions.

• Hierarchical architectures are probably the mostwidely used in robotics [43,76,20]. They proposean organization of the software along layers (twoor three) with different temporal requirementsand abstraction levels. Often, there is a functionallayer containing the low-level sensors–effectors–processing modules, and a decision layer contain-ing some of the deliberation functions presentedhere (e.g. planning, acting, monitoring, etc).

• Teleo-reactive architectures [26,66] are more re-cent. They propose an integrated planning–acting paradigm which is implemented at dif-ferent levels, from deliberation down to reac-tive functions, using different planning–actinghorizons and time quantum. Each planner–actoris responsible for ensuring the consistency ofa constraint network (temporal and atempo-ral) whose state variables can be shared withother planners–actors to provide a communica-tion mechanism.

Beyond architecture paradigms, it is interestingto note that some robotics systems have achievedan impressive level of integration of numerous de-liberation functions on real platforms. The Linkop-ing UAV project [20] provides planning, acting,perception, monitoring with formal representa-tions all over these components. The NMRA onthe DS1 probe [74] also proposed planning, acting,and FDIR onboard. IDEA and T-ReX, providingplanning and acting have been used respectivelyon a robot [26] and an AUV [66].

9. Conclusion

Autonomous robots facing a variety of open en-vironments and a diversity of tasks cannot relyon the decision making capabilities of a humandesigner or teleoperator. To achieve their mis-sions, they have to exhibit complex reasoning ca-pabilities required to understand their environ-ment and current context, and to act deliberately,in a purposeful, intentional manner. In this paper,we have referred to these reasoning capabilitiesas deliberation functions, closely interconnectedwithin a complex architecture. We have presentedan overview of the state of the art for some of them.

For the purpose of this overview, we found itclarifying to distinguish these functions with re-spect to their main role and computational re-quirements: the perceiving, goal reasoning, plan-ning, acting and monitoring functions. But let usinsist again: the border line between them is notcrisp; the rational for their implementation withinan operational architecture has to take into ac-count numerous requirements, in particular a hier-archy of closed loops, from the most dynamic in-ner loop, closest to the sensory-motor signals andcommands, to the most “offline” outer loop.

Consider for example the relationship betweenplanning and acting. We argued that acting can-not be reduced to “execution control”, that is thetriggering of commands mapped to planned ac-tions. There is a need for significant deliberationto take place between what is planned and thecommands achieving it (Fig. 2). This acting de-liberation may even rely on the same or on dif-ferent planning techniques as those of the plan-ner, but it has to take into account different statespaces, action spaces and event spaces than thoseof the planner. However, if we insisted to distin-

16

guish these two levels, there is no reason to believethat just two levels is the right number. There canbe a hierarchy of planning–acting levels, each re-fining a task planned further up into more concreteactions, adapted to the acting context and fore-seen events. It would be convenient and elegant toaddress this hierarchy within a homogeneous ap-proach, e.g., HTN or AHP. But we strongly sus-pect that conflicting requirements, e.g., for han-dling uncertainty and domain specific representa-tions, favor a variety of representations and ap-proaches.

Many other open issues, briefly referred to inthis paper, give rise to numerous scientific chal-lenges. The relationship from sensing and acting toperceiving is clearly one of these bottleneck prob-lems to which more investigation efforts need tobe devoted. Acting in an open world requires go-ing from anchoring to symbol grounding, from ob-ject recognition to categorization. A developmentperspective is to make robots query when neededand benefit from the growing wealth of knowledgeavailable over the web, within ontologies of tex-tual and symbolic relations, as well as of images,graphical and geometric knowledge.

Deliberation functions involve several other openissues that we have not discussed in this overview,among which the noteworthy problems of:• metareasoning : trading off deliberation time for

acting time, given how critical and/or urgent arethe context and tasks at hand;

• interaction and social behavior that impact allfunctions discussed here, from the perceiving re-quirements of a multi-modal dialogue, to theplanning and acting at the levels of task shar-ing and plan understanding for multi-robots andman-robot interaction;

• learning which is the only hope for building themodels required by deliberation functions andwhich has a strong impact on the architecturethat would permit to integrate these functionsand allow them to adapt to tasks and environ-ments the robot is facing.We believe that the AI–Robotics synergy is be-

coming richer and more complex, and it remainstoday as fruitful for both fields as it used to bein their early beginning. We do hope that thisoverview will attract more practitioners to thechallenging problems of their intersection.

Acknowledgements

We thank the editors of this special issue andthe reviewers for their highly valuable feedback.

References

[1] B. Argall, S. Chernova, M. Veloso, and B. Browning. A

survey of robot learning from demonstration. Robotics

and Autonomous Systems, 57(5):469–483, 2009.

[2] T. Bailey and H. Durrant-Whyte. Simultaneous local-ization and mapping (SLAM): part II. IEEE Robotics

and Automation Magazine, 13(3):108 –117, 2006.

[3] M. Beetz. Structured reactive controllers: controlling

robots that perform everyday activity. In Proceed-ings of the annual conference on Autonomous Agents,

pages 228–235. ACM, 1999.

[4] M. Beetz and D. McDermott. Improving Robot Plans

During Their Execution. In Proc. AIPS, 1994.

[5] S. Bernardini and D. Smith. Finding mutual exclusioninvariants in temporal planning domains. In Seventh

International Workshop on Planning and Scheduling

for Space (IWPSS), 2011.

[6] J. Bohren, R. Rusu, E. Jones, E. Marder-Eppstein,C. Pantofaru, M. Wise, L. Mosenlechner, W. Meeussen,

and S. Holzer. Towards autonomous robotic butlers:Lessons learned with the PR2. In Proc. ICRA, pages

5568–5575, 2011.

[7] R. Bonasso, R. Firby, E. Gat, D. Kortenkamp,

D. Miller, and M. Slack. Experiences with an Archi-tecture for Intelligent, Reactive Agents. Journal of

Experimental and Theoretical Artificial Intelligence,

9(2/3):237–256, April 1997.

[8] A. Bouguerra, L. Karlsson, and A. Saffiotti. Seman-tic Knowledge-Based Execution Monitoring for Mobile

Robots. In Proc. ICRA, pages 3693–3698, 2007.

[9] C. Boutilier, T. Dean, and S. Hanks. Decision-

Theoretic Planning: Structural Assumptions and Com-putational Leverage. Journal of AI Research, 11:1–94,

May 1999.

[10] R. Brooks. A robust layered control system for a mo-

bile robot. IEEE Journal of Robotics and Automation,2:14–23, 1986.

[11] L. Busoniu, R. Munos, B. De Schutter, and

R. Babuska. Optimistic planning for sparsely stochas-tic systems. IEEE Symposium on Adaptive Dynamic

Programming And Reinforcement Learning, pages 48–55, 2011.

[12] S. Cambon, R. Alami, and F. Gravot. A hybridapproach to intricate motion, manipulation and task

planning. International Journal of Robotics Research,

28(1):104–126, 2009.

[13] F. Chaumette and S. Hutchinson. Visual servo con-trol, part ii: Advanced approaches. IEEE Robotics and

Automation Magazine, 14(1):109–118, 2007.

17

[14] F. Chaumette and S. Hutchinson. Visual servoing and

visual tracking. In B. Siciliano and O. Khatib, edi-

tors, Springer Handbook of Robotics, pages 563–583.Springer, 2008.

[15] A. J. Coles, A. Coles, M. Fox, and D. Long. COLIN:

Planning with Continuous Linear Numeric Change.

Journal of AI Research, 2012.

[16] P. Conrad, J. Shah, and B. Williams. Flexible execu-tion of plans with choice. In Proceedings of ICAPS,

2009.

[17] S. Coradeschi and A. Saffiotti. An introduction to theanchoring problem. Robotics and Autonomous Sys-

tems, 43(2-3):85–96, 2003.

[18] E. Coste-Maniere, B. Espiau, and E. Rutten. A task-

level robot programming language and its reactive ex-ecution. In Proc. ICRA, 1992.

[19] O. Despouys and F. Ingrand. Propice-Plan: Toward

a Unified Framework for Planning and Execution. In

European Workshop on Planning, 1999.

[20] P. Doherty, J. Kvarnstrom, and F. Heintz. A temporallogic-based planning and execution monitoring frame-

work for unmanned aircraft systems. Autonomous

Agents and Multi-Agent Systems, 19(3), 2009.

[21] P. Doherty, J. Kvarnstrom, and F. Heintz. A temporal

logic-based planning and execution monitoring frame-

work for unmanned aircraft systems. AutonomousAgents and Multi-Agent Systems, 19(3):332–377, 2009.

[22] C. Dousson, P. Gaborit, and M. Ghallab. Situation

recognition: Representation and algorithms. Proc. IJ-

CAI, 13:166–166, 1993.

[23] C. Dousson and P. Le Maigat. Chronicle recognitionimprovement using temporal focusing and hierarchiza-

tion. In Proc. IJCAI, pages 324–329, 2007.

[24] M. Fichtner, A. Großmann, and M. Thielscher. Intel-

ligent execution monitoring in dynamic environments.Fundamenta Informaticae, 57(2-4):371–392, 2003.

[25] R. Fikes. Monitored Execution of Robot Plans Pro-

duced by STRIPS. In IFIP Congress, Ljubljana, Yu-goslavia, August 1971.

[26] A. Finzi, F. Ingrand, and N. Muscettola. Model-based

executive control through reactive planning for au-

tonomous rovers. In Proc. IROS, volume 1, pages 879–884, 2004.

[27] R. Firby. An Investigation into Reactive Planning in

Complex Domains. In Proc. AAAI, pages 1–5, 1987.

[28] J. Frank and A. Jonsson. Constraint-based attribute

and interval planning. Constraints, 8(4):339–364, 2003.

[29] G. Fraser, G. Steinbauer, and F. Wotawa. Plan ex-

ecution in dynamic environments. In Innovations inApplied Artificial Intelligence, volume 3533 of LNCS,pages 208–217. Springer, 2005.

[30] S. Fratini, A. Cesta, R. De Benedictis, A. Orlandini,

and R. Rasconi. Apsi-based deliberation in goal ori-

ented autonomous controllers. In 11th Symposium onAdvanced Space Technologies in Robotics and Automa-

tion (ASTRA), 2011.

[31] F. Fusier, V. Valentin, F. Bremond, M. Thonnat,M. Borg, D. Thirde, and J. Ferryman. Video under-

standing for complex activity recognition. MachineVision and Applications, 18:167–188, 2007.

[32] C. Geib and R. Goldman. A probabilistic plan recogni-tion algorithm based on plan tree grammars. Artificial

Intelligence, 173:1101–1132, 2009.

[33] M. Ghallab. On chronicles: Representation, on-line

recognition and learning. In International Conferenceon Knowledge Representation and Reasoning, pages

597–606, 1996.

[34] M. Ghallab and A. Mounir Alaoui. Managing effi-

ciently temporal relations through indexed spanningtrees. In Proc. IJCAI, pages 1297–1303, 1989.

[35] M. Ghallab, D. Nau, and P. Traverso. Automated Plan-

ning: Theory and Practice. Morgann Kaufmann, Oc-

tober 2004.

[36] G. Giralt, R. Sobek, and R. Chatila. A multi-levelplanning and navigation system for a mobile robot:

a first approach to HILARE. In Proc. IJCAI, pages

335–337, 1979.

[37] D. Hahnel, W. Burgard, and G. Lakemeyer. GOLEX—

bridging the gap between logic (GOLOG) and a realrobot. In KI-98: Advances in Artificial Intelligence,

pages 165–176. Springer, 1998.

[38] M. Hauskrecht, N. Meuleau, L. P. Kaelbling, T. Dean,

and C. Boutilier. Hierarchical solution of Markov de-cision processes using macro-actions. In Proceedings

of the Conference on Uncertainty in Artificial Intelli-

gence, pages 220–229, 1998.

[39] F. Heintz, J. Kvarnstrom, and P. Doherty. Bridgingthe sense-reasoning gap: DyKnow-Stream-based mid-

dleware for knowledge processing. Advanced Engineer-

ing Informatics, 24(1):14–26, 2010.

[40] S. Hongeng, R. Nevatia, and F. Bremond. Video-basedevent recognition: activity representation and proba-

bilistic recognition methods. Computer Vision and

Image Understanding, 96(2):129–162, 2004.

[41] M. Ingham, R. Ragno, and B. Williams. A Reac-tive Model-based Programming Language for Robotic

Space Explorers. In International Symposium on Arti-

ficial Intelligence, Robotics and Automation for Space,2001.

[42] F. Ingrand, R. Chatilla, R. Alami, and F. Robert. PRS:

A High Level Supervision and Control Language forAutonomous Mobile Robots. In IEEE InternationalConference on Robotics and Automation, 1996.

[43] F. Ingrand, S. Lacroix, S. Lemai-Chenevier, and F. Py.Decisional Autonomy of Planetary Rovers. Journal of

Field Robotics, 24(7):559–580, October 2007.

[44] A. Jonsson, P. Morris, N. Muscettola, K. Rajan, andB. Smith. Planning in Interplanetary Space: Theory

and Practice. In International Conference on AI Plan-

ning Systems, 2000.

[45] L. Pack Kaelbling and T. Lozano-Perez. Hierarchical

task and motion planning in the now. In Proc. ICRA,pages 1470–1477, 2011.

18

[46] O. Kanoun, J-P. Laumond, and E. Yoshida. Planning

foot placements for a humanoid robot: A problem of

inverse kinematics. International Journal of RoboticsResearch, 30(4):476–485, 2011.

[47] L. Karlsson, A. Bouguerra, M. Broxvall, S. Coradeschi,and A. Saffiotti. To secure an anchor – a recovery plan-

ning approach to ambiguity in perceptual anchoring.

AI Communications, 21(1):1–14, 2008.

[48] H. Kautz and J. Allen. Generalized plan recognition.

In Proc. AAAI, pages 32 – 37, 1986.

[49] M. Kearns, Y. Mansour, and A. Ng. A sparse sampling

algorithm for near-optimal planning in large Markovdecision processes. Machine Learning, 49:193–208,

2002.

[50] A. Kolobov, Mausam, and D. Weld. SixthSense: Fast

and Reliable Recognition of Dead Ends in MDPs. Proc.

AAAI, April 2010.

[51] A. Kolobov, Mausam, and D. Weld. LRTDP vs. UCT

for Online Probabilistic Planning. In Proc. AAAI,2012.

[52] A. Kolobov, Mausam, D. Weld, and H. Geffner. Heuris-tic search for generalized stochastic shortest path

MDPs. Proc. ICAPS, 2011.

[53] K. Konolige, E. Marder-Eppstein, and B. Marthi. Nav-

igation in hybrid metric-topological maps. In Proc.ICRA, 2011.

[54] K. Konolige, K. Myers, E. Ruspini, and A. Saffiotti.The saphira architecture: A design for autonomy. Jour-

nal of Experimental and Theoretical Artificial Intelli-

gence, 9:215–235, 1997.

[55] V. Kruger, D. Kragic, A. Ude, and C. Geib. The mean-

ing of action: a review on action recognition and map-ping. Advanced Robotics, 21(13):1473–1501, 2007.

[56] B. Kuipers and Y. Byun. A robot exploration and map-ping strategy based on a semantic hierarchy of spatial

representations. Robotics and Autonomous Systems,

8(1-2):47–63, 1991.

[57] B. Kuipers, J. Modayil, P. Beeson, M. MacMahon,

and F. Savelli. Local metrical and global topologicalmaps in the hybrid spatial semantic hierarchy. In Proc.

ICRA, pages 4845–4851, 2004.

[58] J. Kvarnstrom and P. Doherty. TALplanner: A tem-poral logic based forward chaining planner. Annals

of Mathematics and Artificial Intelligence, 30(1):119–169, 2000.

[59] K. Ben Lamine and F. Kabanza. History checking oftemporal fuzzy logic formulas for monitoring behavior-

based mobile robots. In 12th IEEE International Con-ference on Tools with Artificial Intelligence, 2000. IC-

TAI 2000., pages 312–319, 2000.

[60] K. Ben Lamine and F. Kabanza. Reasoning aboutrobot actions: A model checking approach. In

M. Beetz, J. Hertzberg, M. Ghallab, and M. Pollack,editors, Advances in Plan-Based Control of Robotic

Agents, volume 2466 of LNCS, pages 123–139, 2002.

[61] S. LaValle. Planning Algorithms. Cambridge Univer-

sity Press, 2006.

[62] S. Lemai-Chenevier and F. Ingrand. Interleaving Tem-poral Planning and Execution in Robotics Domains.

In Proc. AAAI, 2004.

[63] H. Levesque, R. Reiter, Y. Lesperance, F. Lin, and

R. Scherl. Golog: A logic programming language fordynamic domains. Journal of Logic Programming,

31:59–84, 1997.

[64] M. Likhachev, G. Gordon, and S. Thrun. Planning

for markov decision processes with sparse stochasticity.

Advances in Neural Information Processing Systems(NIPS), 17, 2004.

[65] Mausam and A. Kolobov. Planning with Markov De-cision Processes: An AI Perspective. Morgan & Clay-

pool, July 2012.

[66] C. McGann, F. Py, K. Rajan, J. Ryan, and R. Hen-

thorn. Adaptive Control for Autonomous UnderwaterVehicles. In Proc. AAAI, page 6, April 2008.

[67] T. Moeslund, A. Hilton, and V. Kruger. A survey ofadvances in vision-based human motion capture and

analysis. Computer Vision and Image Understanding,

104(2-3):90–126, 2006.

[68] M. Molineaux, M. Klenk, and D. Aha. Goal-driven au-

tonomy in a Navy strategy simulation. In Proc. AAAI,pages 1548–1554, 2010.

[69] M. Montemerlo, S. Thrun, D. Koller, and B. Weg-breit. Fastslam 2.0: An improved particle filtering algo-

rithm for simultaneous localization and mapping that

provably converges. In Proc. IJCAI, pages 1151–1156,2003.

[70] H. Moravec. The Stanford Cart and the CMU Rover.

Technical report, CMU, February 1983.

[71] B. Morisset and M. Ghallab. Learning how to com-

bine sensory-motor functions into a robust behavior.

Artificial Intelligence, 172(4-5):392–412, March 2008.

[72] P. Morris, N. Muscettola, and T. Vidal. Dynamic con-

trol of plans with temporal uncertainty. In Proc. IJ-CAI, pages 494–502, 2001.

[73] N. Muscettola, G. Dorais, C. Fry, R. Levinson, andC. Plaunt. A Unified Approach to Model-Based Plan-

ning and Execution. In Proceedings of the Interna-tional Conference on Intelligent Autonomous Systems,

2000.

[74] N. Muscettola, P. Nayak, B. Pell, and B. Williams.

Remote Agent: to boldly go where no AI system has

gone before. Artificial Intelligence, 103:5–47, 1998.

[75] K. Myers. CPEF: Continuous Planning and ExecutionFramework. AI Magazine, 20(4):63–69, 1999.

[76] I. Nesnas, A. Wright, M. Bajracharya, R. Simmons,

and T. Estlin. CLARAty and Challenges of Develop-ing Interoperable Robotic Software. In Proc. IROS,

October 2003.

[77] B. Pell, E. Gat, R. Keesing, N. Muscettola, and Ben

Smith. Robust Periodic Planning and Execution forAutonomous Spacecraft. In Proc. IJCAI, 1997.

[78] O. Petterson. Execusion monitoring in robotics: asurvey. Robotics and Autonomous Systems, 53:73–88,

2005.

19

[79] J. Pineau, M. Montemerlo, M. Pollack, N Roy, and

S. Thrun. Towards robotic assistants in nursing homes:

Challenges and results. Robotics and Autonomous Sys-tems, 42(3-4):271–281, March 2003.

[80] M. Pollack and J. Horty. There’s more to life than

making plans: plan management in dynamic, multia-gent environments. AI Magazine, 20(4):71, 1999.

[81] F. Py, K. Rajan, and C. McGann. A systematic agent

framework for situated autonomous systems. In Proc.AAMAS, pages 583–590, 2010.

[82] D. Pynadath and M. Wellman. Probabilistic state-

dependent grammars for plan recognition. In Proc.Uncertainty in Artificial Intelligence, pages 507–514,

2000.

[83] M. Ramirez and H. Geffner. Plan recognition as plan-ning. In Proc. IJCAI, pages 1778 – 1783, 2009.

[84] J. Rintanen. An iterative algorithm for synthesizing

invariants. In Proc. AAAI, pages 806–811, 2000.

[85] C. Rosen and N. Nilsson. Application of intelligent

automata to reconnaissance. Technical report, SRI,

November 1966.

[86] E. Sandewall. Features and Fluents. Oxford university

Press, 1995.

[87] T. Simeon, J-P. Laumond, J. Cortes, and A. Sahbani.Manipulation planning with probabilistic roadmaps.

International Journal of Robotics Research, 23(7-

8):729–746, 2004.

[88] R. Simmons. Structured control for autonomous

robots. IEEE Transactions on Robotics and Automa-

tion, 10(1):34–43, 1994.

[89] R. Simmons and D. Apfelbaum. A task description

language for robot control. In Proc. IROS, 1998.

[90] F. Teichteil-Konigsbuch, U. Kuter, and G. Infantes.Incremental plan aggregation for generating policies in

MDPs. In Proc. AAMAS, pages 1231–1238, 2010.

[91] F. Teichteil-Konigsbuch, C. Lesire, and G. Infantes. Ageneric framework for anytime execution-driven plan-

ning in robotics. In Proc. ICRA, pages 299–304, 2011.

[92] M. Veloso and P. Rizzo. Mapping planning actionsand partially-ordered plans into execution knowledge.

In Workshop on Integrating Planning, Scheduling and

Execution in Dynamic and Uncertain Environments,pages 94–97, 1998.

[93] V. Verma, T. Estlin, A. Jonsson, C. Pasareanu, R. Sim-

mons, and K. Tso. Plan execution interchange lan-guage (PLEXIL) for executable plans and command

sequences. In Proceedings of the 9th International

Symposium on Artificial Intelligence, Robotics andAutomation in Space, 2005.

[94] D. Wilkins and K. Myers. A common knowledge rep-

resentation for plan generation and reactive execu-tion. Journal of Logic and Computation, 5(6):731–761,

1995.

[95] B. Williams and M. Abramson. Executing reactive,model-based programs through graph-based temporal

planning. In Proc. IJCAI, 2001.

[96] J. Wolfe, B. Marthi, and S. Russell. Combined taskand motion planning for mobile manipulation. In Proc.

ICAPS, 2010.

[97] Y. Wu and T. Huang. Vision-Based Gesture Recogni-tion: A Review, volume 1739 of LNCS, pages 103–116.

Springer-Verlag, 1999.