Manufacturing systems are typically workhorses thatrun reliably for long periods of time under intenseuse. They have not traditionally been known for

their flexibility and changeability. When something does gowrong, the entire system may need to be shut down, withevery minute of down time representing lost production andpossibly damaged product. Further, making changes in a fac-tory line can require extensive redesign, retooling, and repro-gramming.

Recently, a trend in manufacturing technology has beentoward reconfigurable manufacturing systems (Koren et al.1999), in which factory lines are designed to support changesin their structure. Such systems, which are gradually beingdeployed, still need to be configured and rearranged offline,but have advantages in terms of rapid deployment of new con-figurations, standardization of parts, and machine design time.

The next challenge in achieving true flexibility in manu-facturing is to imbue factory systems with online reconfig-urability, that is, the ability to swap parts out online, enableor disable capabilities in real time, and respond quickly tochanges in the system or environment (including faults). Inaddition to the advantages conferred by offline reconfigura-bility, online reconfigurability would allow a manufacturingsystem to run more consistently and reliably, as well as allowit to be more flexible, able to change products or objectivesextremely rapidly. We believe that these goals can be attainedthrough the use of a very high level of modularity, both inhardware and software, combined with intelligent software.

To test this hypothesis, Palo Alto Research Center (PARC)designed and built a prototype highly modular system in theprinting domain. This “hypermodular” printer explores theextremes of modularity, reconfigurability, and parallelism inboth hardware and software. The hardware prototype con-nects four standard Xerox marking engines (the componentof a printer that does the actual printing) in parallel using ahighly modular paper path. This configuration can achieve aprint rate of four times that of an individual print engine.

The system software stack for the prototype consists of

Articles

FALL 2013 73Copyright © 2013, Association for the Advancement of Artificial Intelligence. All rights reserved. ISSN 0738-4602

Online Reconfigurable Machines

Lara S. Crawford, Minh Binh Do, Wheeler Ruml, Haitham Hindi, Craig Eldershaw, Rong Zhou, Lukas Kuhn, Markus P. J. Fromherz,

David Biegelsen, Johan de Kleer, Daniel Larner

n A recent trend in intelligentmachines and manufacturing has beentoward reconfigurable manufacturingsystems. Such systems move away froma fixed factory line executing anunchanging set of operations andtoward the goal of an adaptable factorystructure. The logical next challenge inthis area is that of online reconfigura-bility. With this capability, machinescan reconfigure while running, enable ordisable capabilities in real time, andrespond quickly to changes in the sys-tem or the environment (includingfaults). We propose an approach toachieving online reconfigurability basedon a high level of system modularitysupported by integrated, model-basedplanning and control software. Oursoftware capitalizes on many advancedtechniques from the artificial intelli-gence research community, particularlyin model-based domain-independentplanning and scheduling, heuristicsearch, and temporal resource reason-ing. We describe the implementation ofthis design in a prototype highly modu-lar, parallel printing system.

three integrated layers with (1) a model-based artifi-cial intelligence (AI) planner / scheduler that canquickly find sheet trajectories for all requested print-ing jobs, (2) sheet controllers that coordinate thecontrol of individual sheets to execute these plans,and (3) printer module controllers that control theactual printer hardware. This software architecturesupports flexibility in configuration, graceful degra-dation under component failure, and rerouting of in-process sheets under exception conditions. Thesecapabilities were made possible by utilizing advancedAI techniques in model-based planning, scheduling,search, and temporal reasoning such as state-spaceregression planning, partial-order scheduling, tem-poral planning graph–based heuristic estimates, mul-tiobjective search, and fast, simple temporal networkreasoning. The AI planner / scheduler incorporatesmostly domain-independent techniques from theplanning and scheduling research community,enabling its flexibility and configurability to bedemonstrated on a variety of applications, includingother printer designs, food-packaging machines, andmaterial control systems.

In addition to a novel integration of existing tech-niques that represents the first successful industrialapplication of embedded, domain-independent tem-poral planning, this work also contributes (1) newheuristic evaluation functions for temporal planningthat incorporate some of the effects of resource con-straints, (2) planning algorithms targeting wall-clockgoal achievement time (that is, combining planningtime and plan execution time), and (3) a chained best-first-search algorithm that efficiently handles replan-ning problems in real time. The planner / schedulerwill be described, but for greater detail on these algo-rithms we refer the reader to Ruml et al. (2011).

Reconfigurable Manufacturing Systems

Market trends have led the manufacturing commu-nity to explore alternatives to dedicated manufactur-ing lines, alternatives that would provide more flexi-bility in a variety of dimensions. In particular, theability to vary a product line without having toredesign the manufacturing line enables manufac-turers to respond to more rapidly changing marketrealities.

The 1980s saw the rise of the concept of flexiblemanufacturing (ElMaraghy 2006). The main empha-sis of flexible manufacturing systems (FMSs) is onmaking use of highly flexible machines, such as com-puter numerical control (CNC) machines, to enable awider product mix in the manufacturing line. Thisapproach requires a large capital investment, howev-er, and only allows for a range of products commen-surate with the tools’ capabilities. FMSs also includeto some extent the idea of routing flexibility, inwhich a system is built so that there are multiple

machines that can perform the same operation andthe system can route parts to appropriate machines.

The idea of reconfigurable manufacturing, by con-trast, focuses on changing the structure of the man-ufacturing system, both in hardware and software, toadapt quickly to changing product designs or capac-ity needs. Reconfigurable manufacturing systems(RMSs) were introduced as a concept in the late1990s (Koren et al. 1999), but the prerequisites, inboth software and hardware, for implementing themsuccessfully have proved daunting; very few exam-ples of RMSs exist today in practice. These prerequi-sites include modular, reconfigurable hardware com-ponents as well as the software and controlarchitectures and logic to support them. RMSs caninclude both hard reconfigurability (physical recon-figuration) and soft reconfigurability (logical recon-figuration) (ElMaraghy 2006). This latter conceptincludes the idea of flexible routing as well as replan-ning and rescheduling.

There has been much literature devoted to the pre-cise definition and differentiation of the varioustypes of modern manufacturing approaches, includ-ing flexible manufacturing and reconfigurable man-ufacturing (Mehrabi et al. 2002; Heisel and Meitzner2004; ElMaraghy 2006; Bi et al. 2008; ElMaraghy2009; Wiendahl et al. 2007; Koren and Shpitalni2011). All of these approaches are trying to achievethe same goals in flexibility, though possibly at dif-ferent levels of the system hierarchy or through dif-ferent techniques. These goals include flexibility inoperations, products, routing, volume, controls, andexpansion, among others (ElMaraghy 2006). Theterm changeable manufacturing has been introducedas an umbrella term to include both FMSs and RMSs(Wiendahl et al. 2007). In this article, as we areemphasizing physical and logical reconfigurabilityrather than highly flexible component machines, wewill use the term reconfigurable.

Physical reconfiguration as well as logical recon-figuration requires support in software. In particular,technology for reconfigurable process planning(Azab, ElMaraghy, and Samy 2009) is needed that cansupport hardware reconfiguration. There are severaltypes of automated reconfigurable process planningthat have been identified, ranging from approachesthat create plans for component variations by modi-fying plans for known components to fully genera-tive process planning in which plans are generatedfrom first principles and models of the manufactur-ing system, parts, and so on. (Wiendahl et al. 2007).As far as we are aware, fully generative process plan-ning systems to support reconfigurable manufactur-ing do not yet exist. Such a first-principles approachis a very good match for the artificial intelligenceapproach to planning, however. Artificial intelli-gence techniques such as state-space regression plan-ning, partial-order scheduling, temporal planninggraphs, and temporal networks can provide a new

Articles

74 AI MAGAZINE

take on generative process planning, as well as direct-ly addressing systemwide performance-orientedmeasures. As we will describe in this article, this typeof planning can be made fast enough to executeonline, which enables many features of reconfig-urable manufacturing systems in a dynamic onlinesense, including reconfiguration of hardware,dynamically flexible routing, repurposing ofmachines, and robustness to equipment failure. Plan-ning approaches have been used in the manufactur-ing domain to plan reconfiguration strategies forRMSs ((Tang et al. 2006), for example), but as far aswe are aware they have not been used for online gen-erative process planning.

Online ReconfigurabilityIn this article, we focus on the design, development,and control of what we term online reconfigurable sys-tems. Online reconfigurability encompasses a range ofsystem properties. A reconfigurable system can berecomposed, with parts being removed or added,without reengineering the hardware or software(Fromherz, Bobrow, and de Kleer 2003). This proper-ty means that the system must be highly scalable, inboth hardware and software. The ability to reconfig-ure online implies that the system software canrespond quickly to changes in the system composi-tion, which in turn means that it can adapt quickly tofaults or other changes in system capabilities. Recon-figurability can also mean the ability to use the sameparts differently for a different purpose, such as a dif-ferent system objective; it includes functional as wellas structural reconfigurability. Online reconfigurabili-ty of this type confers the ability to adapt to a real-time change in system goals or even an observedchange in the system environment. To take advantageof these capabilities of adaptation, the system musthave some redundancy or variety in its composition,so that when the system or environment changesthere are alternatives the system can employ to con-tinue to operate near-optimally. As we stated above,the combination of these aspects of online reconfig-urability confers many benefits in manufacturing sys-tems, including flexibility, graceful degradation underfaults, and high levels of customizability.

Model-Based ModularityWe conjectured that a high degree of modularitycombined with integrated, model-based planningand control software would enable all the features ofonline reconfigurability. Modularity can be used tocreate a scalable system with a degree of redundancyuseful for adapting to change. Model-based, onlineplanning allows a system to react seamlessly tochanges in system configuration, changes in systemgoals, or faults. Integration of planning and controlsoftware creates consistency in such a complex sys-

Articles

FALL 2013 75

tem and enables rapid responses to these changes.A high degree of modularity and redundancy intro-

duces challenges of its own. A highly modular systemtends to be large-scale, in terms of number of compo-nents, and fairly complex due to component interac-tions. There are many possible control architecturedesigns and coordination mechanisms for such acomplex system, and choosing among them requiresanalysis of a variety of trade-offs. The best architec-ture may be quite specific to the particular system.

Hypermodular PrinterThe PARC highly modular printer prototype, knownas the “hypermodular” printer, was developed as partof a Xerox research program on parallel printing. Ourgoal has been to enable online reconfigurability inthis system through a high degree of modularity inboth hardware and software. The project focuses onthe hardware and software elements required forpaper-path control, which includes the routing, tim-ing, and control of sheets through the machine. Thisarticle will discuss aspects of the software (systemcontrol) side of this extended experiment into hyper-modular system design.

The hypermodular printing system is shown in fig-ure 1. The system contains four black-and-whitemarking engines, which do the actual printing onone side of sheets fed to them. There are two paperfeeders at the left of the machine and three systemoutputs on the right (two output trays and one purgelocation). The feeders, marking engines, and outputsare connected by reconfigurable paper paths. Thesepaths are made of PARC-designed modules that canbe positioned in a grid frame to create the desiredshape of path. There are three types of these hyper-modules used in the machine: three-way directormodules (which allow path branching), straight-through modules, and roller actuator (called “nip”)modules (Biegelsen et al. 2009, 2011). The paper-pathmodules are inserted and removed using a rotaryaction that supports jam clearance.

The hypermodular paper-path elements are bidi-rectional, meaning the paper can travel in eitherdirection along the internal paths. Bidirectionality isnew for the printing industry and has complexityconsequences for the software. Each hypermodulehas its own processor (a Texas Instruments F2811DSP) and connects to one of five CAN buses at theback of the machine. Each nip module has a nip,actuated with a stepper motor, and infrared sheetsensors on either side of the nip. Each three-way hasflippers, driven by solenoids, to direct sheets to thecorrect output. In the machine, a nip module isplaced in between each pair of three-way or straight-through modules on the grid. Our machine is con-figured to have three main high-speed sheet “high-ways” going the length of the machine, with somecross connections between them. There are “off-

ramps” and “on-ramps” sending paper to and fromthe marking engines, which run at a slower speed.The system has, in total, 36 three-way, 33 straight-through, and 89 nip modules. At full nominal speed,the highways move paper at 1.25 m/s.

The redundancy in the system, in terms of mark-ing engines, paper-path modules, and possible paperpaths, means that if one component fails, the others

can continue to run (with some possible loss of pro-ductivity). The highly distributed nature of the sys-tem enables a large degree of reconfigurability,robustness, and scalability.

Control ArchitectureThe control architecture for the hypermodular print-er prototype was developed with online reconfigura-

Articles

76 AI MAGAZINE

feeders outputs

marking engine marking engine

marking engine marking engine

A B

DC

Figure 1. The Hypermodular High-Speed Printer Prototype.

Paper paths are shown as lines on the schematic, with a thicker line indicating a typical sheet route. Nips are shown as pairs of small circles.The machine contains three types of hypermodules: three-ways (lower left, rear view), straight-throughs, and nip modules (lower right). Fig-ures 1a and 1b are reproduced with permission from Crawford et al. (2009), © IEEE. Figure 1b is reproduced with permission from Hindi(2008), © IEEE. Figures 1c and 1d are reproduced with permission Biegelsen et al (2009), © IS&T.

bility and extreme modularity in mind. There aremany possible architectures for a system of this type,and the ideal solution is tied to the choices made inphysical system instantiation. Our design choiceswere guided by several design principles, whichincluded (Fromherz, Crawford, and Hindi 2005):

Multiscale control. As a basic principle, we decom-pose the control problem both horizontally (overmodules or sheets) and vertically (hierarchically),based on the location and time scale of differentaspects of the control task.

Encapsulation. Whenever possible, we try to keepknowledge and algorithms related to a particular sub-system together and local to that subsystem, avoid-ing replicating knowledge in different places as muchas possible.

Autonomy, delegation, and escalation. Each controlentity is designed to function autonomously, includ-ing monitoring its own behavior and correcting itwhere possible. To enable autonomy, an entity dele-gating a control task must also provide sufficientinformation for the controller to tell if its behavior iswithin requirements. If it is not, and the autonomouscontroller is unable to perform its delegated task, itmust escalate exception information back up thehierarchy.

The hierarchical control architecture shown in fig-ure 2 represents an attempt to locate the best pointalong all the design trade-offs for this system, guidedby the principles described above. Each modular sys-tem component has its own module controller. Eachindividual sheet in the system has associated with ita software entity called a sheet controller, which isresponsible for coordinating the modules to achievemotion for that sheet. The planner / scheduler isresponsible for determining routes and timing todirect the sheets through the system. The informa-tion flow through these levels of the hierarchy isshown in figure 3. The software components will bediscussed in more detail below.

Communication is key for a system with many dis-tributed components. Our approach was based onthe ideal of what we call a model-based contract. Inthis paradigm, each request or command from theplanner / scheduler to a sheet controller is viewed asa contractual obligation on the part of the sheet con-troller. The contract is founded on a model of behav-ior that the sheet controller is expected to execute.By accepting the contract, the sheet controller prom-ises the planner / scheduler that it can use thatbehavior model in its reasoning about the future. Thesheet controller uses the model as a guide to monitorits own behavior, as well; if it is unable to keep to theterms of the contract, it must escalate an error mes-sage to the planner. This situation is assumed to bequite rare. The model-based contracts approach ispart of our effort to more closely integrate planningand control, which we believe allows more rapid,flexible responses to online events.

Planner/SchedulerThe planner / scheduler is at the top of the controlhierarchy. It takes print job descriptions as input andproduces valid sheet itineraries as output. This taskinvolves determining routes and timing through thepaper-path elements and making sure that all sheetsend up with the correct properties (with the correctimages printed on them, for example) and do notinterfere with each other. The planner / scheduleralso tries to ensure print jobs finish at the earliestpossible times. Since the planner / scheduler bothchooses routes and decides timing along the routes,our design combines planning and scheduling intoone entity. The planner / scheduler runs on its ownprocessor (a PC) and communicates to the sheet con-trollers over Ethernet.

The hypermodular prototype presents a numberof challenges for the planner / scheduler. First, thereare many possible paths through the machine, andthe machine can also be reconfigured in real time.Determining the optimal route for a sheet is there-fore not trivial but it must be done in an online fash-ion (routes cannot be precompiled). Second, for aprinter or other manufacturing system, new jobs arecontinuously and incrementally submitted, so theplanner / scheduler must plan continuously. One ofthe main objectives of the system is for the job to fin-ish production as soon as possible after it is submit-ted to the printer; this objective requires minimizingthe sum of the plan execution time and the timerequired to find a plan. The planner / scheduler mustthus find a good quality plan very quickly, and itmust do so in parallel with monitoring other sheetsexecuting on the machine. The plans are then sentto the controllers and are eventually converted intolow-level control signals (figure 3).

The planner / scheduler must also be able torespond to escalations (exceptions) as well as config-uration (system model) changes from the lower hier-archical layers. It therefore needs to be able to replansheets already in flight, taking into account the pre-dicted real-time state of the machine at the end ofthe planning. One example of an exception is apaper jam (figure 4). A jam means that one particu-lar sheet will not be produced correctly, but it alsomeans that part of the machine is unexpectedlyblocked and thus unavailable. In this case, the plan-ner / scheduler must find new plans for all sheetsthat have already been planned, including thosealready in the machine, so that they can be routedaround the obstacle. It must also, of course, find away to replace the damaged sheet and insert it intothe job in the correct order. Sheets already in themachine may be routed to a purge tray if they wouldotherwise come out in the wrong order. During nom-inal operation, a delay in planning only results inlower productivity, but in the case of a paper jam orother fault, planning delay can cause a cascade offailures, such as many sheets piling into the jammed

Articles

FALL 2013 77

location. Until now, production printers have usedtwo approaches to exception handling: stop the pro-duction to allow the operator to remove all sheets, or

use machine-specific customized local rules to purgesheets from the system.

We have developed a model-based, online, tempo-

Articles

78 AI MAGAZINE

planner / scheduler

sheet controller

sheet controller

sheet controller factory

CANbus

n-MC

n-MC

n-MC n-MC n-MC

n-MC

t-MC t-MC

t-MC

s-MC s-MC

t-MC n-MC

embedded controllers

Ethernet

system model job requests

physical system

n-MC

Figure 2. Hypermodular Printer Control Architecture.

Sheets are shown as thick lines in the physical system. Module controllers (MCs) are shown in a configuration matchingthat of their associated physical modules, nip (n), three-way (t), or straight-through (s). Module and sheet controllers areshaded according to their association with one of two different control groups (corresponding to the two sheets shown inthe physical system diagram). One module controller is processing information on both sheets, and as such is (or is aboutto be) part of both control groups. Unshaded module controllers are not part of either control group. Figure 2 is reproducedwith permission from Biegelsen et al (2011), © SPIE.

ral planner / scheduler to address these challenges(Ruml, Do, and Fromherz 2005). A schematic isshown in figure 5. Model-based here means that themachine model is separate from the reasoning algo-rithms, which do not contain any domain- or con-figuration-specific information. Online here meansthat the processes of goals (printing requests) arriv-ing, planning (finding legal routes), and plan execu-tion (printing) all interleave. This approach is key foronline reconfigurability: since the reasoning algo-rithms can handle any of a wide range of machinemodels, the model can be allowed to change online.

Our planning algorithm is a combination ofheuristic state-space regression planning (Bonet andGeffner 1999) and partial-order scheduling (Smithand Cheng 1993, Policella et al. 2007). The planningaspect refers to determining the component actionsrequired to route the sheet through the machine andend up with the correct properties. The schedulingaspect refers to defining the ordering between actionsfor different sheets competing for the same resources,

such as a marking engine. Since the planner / sched-uler must process new jobs continually online, plan-ning is done on a per sheet basis, optimizing thecompletion time for all planned sheets. Plan timingis not fixed until the plan is sent to the control lev-els of the hierarchy. In essence, the time point repre-senting the instant that a given sheet will be at a par-ticular location stays flexible within the constraintsbetween it and related time points (for example, thetime it reaches the previous location). Only whenthe plan is sent down to the controller to execute areall time points fixed at their earliest possible times;this procedure can be carried out quickly by a stan-dard algorithm on the temporal network.

The plan optimization, with the default objectivefunction of finishing printing all known jobs asquickly as possible, is performed using a regressionsearch through the system state space, which buildsthe plan “backward” from the end location of eachsheet (the sheet ends in the tray in the correct orderwithin a given print job) back to the beginning loca-

Articles

FALL 2013 79

modulecontroller

modulecontroller

modulecontroller

modulecontroller

sheetcontroller

planner / scheduler

sheetcontroller

modulecontroller

<Job 1, Sheet 1, simplex, A4, b&w, …> <Job 1, Sheet 2, simplex, A4, b&w, …> <Job 2, Sheet 1, simplex, A4, color, …><Job 1, Sheet 3, simplex, A4, b&w, …>

. .

<feeder1, feed, 3.2> <m9, left2middle, 3.3> <m5, middle2left, 3.4>

. .

t

u

x

t

Figure 3. Information Flow in the Hypermodular Printer Control Architecture.

The planner / scheduler receives a job description as input and outputs a set of itineraries for sheets. Each itinerary is a sequence of actionswith their starting times. The sheet controller creates a set of time and position points for the module controllers, which in turn producethe commands (desired rotational velocities) for the motors. The diagrams are based on figures previously published in Fromherz, Craw-ford, and Hindi (2005) and are © 2005 and reproduced with kind permission of Springer Science+Business Media.

tion (a blank sheet at a feeder). The search isinformed by an admissible heuristic based on a tem-poral planning graph structure (Smith and Weld1999), which can be built quickly in linear time byincrementally estimating the earliest time each

action in a given domain can start in any legal plan.In our domain, the complete temporal planninggraph gives a good estimate as to when a sheet canoptimistically reach a particular location, respectingthe trajectories of sheets that are either printing or

Articles

80 AI MAGAZINE

feeders outputs

marking engine marking engine

marking engine marking engine

1.1

1.2

2.1

A

feeders outputs

marking engine marking engine

marking engine marking engine

1.1

1.2

2.1

B

before jam or failure

after jam or failure

Figure 4. Replanning Example.

(A) Top: Job one (lighter dashed path) has two sheets (1.1, 1.2) finishing in the bottom output tray. Job two (darker dashedpath) has one sheet (2.1) exiting at the top tray. (B) Bottom: If sheet 1.1 jams, sheet 1.2 must be rerouted to the purge trayto avoid being out of order. Sheet 2.1 must also be rerouted around the jam. Sheets 1.1 and 1.2 then need to be regenerat-ed in order.

already committed to previous plans. The planner /scheduler uses this information when it builds theplan backward to choose between different possiblesheet trajectories, selecting the one that completes atthe earliest time.

If the planner / scheduler needs to replan for sheetsalready in flight, it enters a replanning mode inwhich it can consider all the sheets in different loca-tions in the machine at once. In this mode, it uses achained progression search through the state spacethat links the search for different in-flight sheetstogether and builds plans forward using the sheets’real-time states as starting constraints (Do, Ruml, andZhou 2008b). Progression (forward) here means that,unlike in nominal planning where the plans/routesare built backward from the sheets’ end locations, webuild the plans starting from the sheet locations atthe point when the exception happens and extend-ing to the end location (which can be the original fin-isher or a “purge” tray). Chained here means we findthe plans for all the in-flight sheets at once, sequen-tially, and the planning episode only ends when theplans for all in-flight sheets are found. This is in con-trast to the nominal planning case, in which eachplanning episode is for a single sheet. This addition-al requirement stems from the preference for nothalting the machine completely when an exceptionoccurs, which implies all the in-flight sheets in con-cert need a set of nonconflicting routes.

The planner / scheduler uses temporal constraintsto keep track of timing and ordering concerns. These

constraints can be used to model the ordering ofactions of different sheets that might interact as wellas real-time considerations such as setup times ornetwork delays. These temporal constraints are man-aged in a simple temporal network (STN) (Dechter,Meiri, and Pearl 1991). The STN answers simple tem-poral queries the planner / scheduler poses whensearching for a good plan, such as, what is the earli-est time a given sheet can complete its execution, or,is a particular ordering of two actions in two differ-ent sheets feasible or not (for example, can page twobe printed before page one on printer X). In addi-tion, the planner / scheduler has a plan managementcomponent that forms the interface between theplanning algorithm and the real-time world. Thiscomponent keeps track of new job requests comingin, sheet plans that have not yet been sent to thecontrollers, and model changes and exceptions com-ing up from the controllers.

Our planner / scheduler has a proven track record.To our knowledge, this work represents the first suc-cessful industrial application of embedded, domain-independent, temporal planning technology. Whileother planners have been able to handle one or a fewof the planning constraints in our domain, there isno existing online planner / scheduler that can han-dle all of them, including both nominal planningand replanning with very fast planning time require-ments as well as multiple objective functions. Themain objective function is minimizing the comple-tion time of all known jobs, but our planner / sched-

Articles

FALL 2013 81

planner/scheduler

plan manager

STN

sheetcontrollers

problemdescription plans

constraints

failures

time info

updates,escalations (faults)

itineraries

printer model

sheetdescription

Figure 5. Overview of the Planner/Scheduler Structure.

This figure originally appeared in Ruml, Do, and Fromherz (2005), and is reproduced with permission of AAAI.

uler can also handle other objective functions suchas: (1) minimizing printing cost (for example, penal-izing printing black-and-white sheets on colorengines); or (2) optimizing for print quality (forexample, printing any pair of the facing pages in abook on engines with the same or similar quality).Our planner / scheduler has been successfullydeployed to control three physical printer prototypesas well as a large number of hypothetical configura-tions in simulation. It can produce plans quicklyenough to keep these systems running at full capaci-ty, which can be quite challenging (at full speed, thehypermodular system requires a new sheet every 27milliseconds on average). The planner / scheduleralso compares favorably to state-of-the-art domain-independent planners in the literature (Do, Ruml,and Zhou 2008a; Ruml et al. 2011), running hun-dreds of times faster on this domain and often find-ing better quality plans than winners of previousInternational Planning Competitions (IPCs). Overall,our experience demonstrates thatdomain-independent artificial intelligence planningbased on heuristic search can flexibly handle time,resources, replanning, and multiple objectives in ahigh-speed practical application without requiringhand-coded control knowledge. Further details onthe planner / scheduler can be found in Ruml et al.(2011).

Related Planning and Scheduling WorkThere has been much interest in the last 20 years inthe integration of planning and scheduling tech-niques. HSTS (Muscettola 1994) and IxTeT (Ghallaband Laruelle 1994) are examples of systems that notonly select and order the actions necessary to reach agoal, but also specify precise execution times for theactions. The Visopt ShopFloor system of Barták(2002) uses a constraint logic programming approachto incorporate aspects of planning into scheduling.The Europa system of Frank and Jónsson (2003) usesa novel representation based on attributes and inter-vals. Unlike our planner / scheduler, all of these sys-tems use domain representations quite different fromthe mainstream PDDL language (Fox and Long 2003)used in planning research, and all of them weredesigned for offline use, rather than controlling a sys-tem during continual execution.

The online nature of the task and the unambigu-ous objective function in our printer-control domaingive rise to an additional trade-off between planningtime and execution time that is absent from muchprior work in planning and scheduling. In our settingthe set of sheets is only revealed incrementally overtime, whereas in classical temporal planning theentire problem instance is available at once. Addi-tionally, in contrast to much work on continual plan-ning (desJardins et al. 1999), the tight constraints ofour domain require that we produce a complete planfor each sheet before its execution can begin. Our

domain emphasizes online decision making, whichhas received only limited attention to date.

Although we present our system as a temporalplanner, it fits easily into the tradition of constraint-based scheduling (Smith and Cheng 1993; Policellaet al. 2007). The main difference is that actions’ timepoints and resource allocations are added incremen-tally rather than all being present at the start of thesearch process. In our approach, we attempt to main-tain a conflict-free schedule rather than allowingcontention to accumulate and then carefully choos-ing which conflicts to resolve first. Our approach isperhaps similar in spirit to that taken by the IxTeTsystem (Ghallab and Laruelle 1994).

Our basic approach of coordinating separate state-space searches through temporal constraints maywell be suitable for other online planning domains.By planning for individual print jobs and managingmultiple plans at the same time, our strategy is simi-lar in spirit to planners that partition goals into sub-goals and later merge plans for individual goals (Wahand Chen 2003, Koehler and Hoffmann 2000). In ourframework, even though each print job is plannedlocally, the global temporal database ensures thatthere are no temporal or resource inconsistencies atany step of the search.

There are several previously proposed frameworksfor handling exceptions and uncertainty in plan exe-cution. Markov decision processes (Boutilier, Dean,and Hanks 1999) and contingency planning (Pryorand Collins 1996) build plans and policies robust touncertain environments. Planners built on thosetechniques are normally slow, especially in a real-time dynamic environment with complex temporalconstraints like ours. They are not suitable for ourdomain where exceptions do not happen frequentlybut need very quick responses. Fox et al. (2006) dis-cuss the trade-off between replanning and plan-repair strategies for handling execution failure. Theiralgorithms work offline, instead of in an online real-time environment such as ours, and they target a dif-ferent objective function (in their case, plan stabili-ty). The CASPER system at JPL (Chien et al. 1999) usesiterative repairs to continuously modify and updateplans to adjust to the dynamic environment. Unlikeour system, CASPER uses domain control rules, andthus is less flexible. The replanning decision is alsonot needed as quickly as in our domain (in our case,subsecond).

Coordination and ControlIn addition to the complexity involved in planningand scheduling for the hypermodular printing sys-tem, there were also several challenges in designingthe control components of the system because of itshighly distributed nature and the need to supportreconfigurability. These challenges centered on thetwin issues of coordination and communication.

Each module in the system has its own processor

Articles

82 AI MAGAZINE

and local controller, the better to reap the benefits ofmodularity, encapsulation, and autonomy by allow-ing each module to contain and act on its own mod-ule-specific knowledge locally. The module controllerwas designed to be sophisticated enough to handleall the local module-specific control, rather than sim-ply accepting control commands from an outsideagent. Though relatively costly in terms of moduleprocessing power, this choice has benefits in terms ofonline reconfigurability. The nip module controllersuse a proximal time-optimal servo (PTOS) controlscheme (Hindi et al. 2008), which demonstrates goodtracking performance for this system. In addition, themodule controller has the ability to inform higherhierarchical levels when it is being taken offline(necessitating a configuration change).

Independently actuated modules require coordi-nation. The hypermodularity of the system meansthat the sheet is under the control of multiple inde-pendent actuators at all times as it goes through themachine. In a more traditional system, actuatorswould all be controlled centrally, or there would be atmost two controllers cooperating during a brief trans-fer between large control domains. There are severalpossible choices for addressing the coordinationproblem. Coordination could be performed at themodule level, through local communication, or thecoordination functionality could travel with thesheet, or it could be centralized; it could even theo-retically reside at the higher planning level. We choseto define a separate entity for performing the coordi-nation function for a sheet, named the sheet con-troller, with one sheet controller being assigneddynamically to each sheet. Defining a separate coor-dination entity kept the different control roles in thesystem separate (encapsulation): the sheet controllercould handle issues involving multiple modules,while the module controllers did not need to beaware of the existence of other modules. In our pro-totype, the sheet controllers are located on their ownprocessor (a PC).

When a new sheet plan is created by the planner,a new sheet controller is generated and associatedwith that sheet. Each sheet controller is responsiblefor defining and maintaining the control group forits sheet, that is, the dynamic group of modules act-ing on the sheet at any one time. It interprets com-mands, or model-based contract requests, for its sheetcoming down the control hierarchy (from the plan-ner / scheduler) and disseminates correspondingcommands to the control group; it is the sheet con-troller’s job to translate the action/time world viewof the planner / scheduler into position/time trajec-tories for the module controllers to track (figure 3). Itmediates communication, including feedback infor-mation, among modules in the control group. Final-ly, it monitors the sheet’s progress, and, if the plan-ner / scheduler’s specifications cannot be met, itescalates an exception message up the hierarchy. For

example, the sheet controller might determine thatits sheet has jammed. It would then tell the planner/ scheduler not only about the jam but also aboutwhich module paths were blocked.

To coordinate the modules tightly enough toensure accurate control and avoid damaging thesheet, we chose to synchronize the controllers in thecontrol group exactly so that they would always pro-duce identical output (nip actuator commands). Thissynchronization, brokered by the sheet controller,has three components. First, the sheet controllerensures that all nips in the control group are givenmatching tracking commands. Second, each newcontroller (for a nip just ahead of the sheet) joiningthe control group must synchronize its internal statewith the controllers already in the group. Third, tomaintain synchronization among the control group,feedback from the sensors must be shared (throughthe sheet controller) and used simultaneously. Datatraveling over the network is delayed, so synchro-nization initialization and maintenance requiresome care and local processing (Crawford et al.2009).

Self-Awareness and DiagnosisTo fully realize the potential of the model-based con-tracts approach, the system components must beable to recognize when they are not meeting theconditions of their contracts. In the hypermodularprinter prototype, the sheet controller is responsiblefor monitoring the sheet progress and then notify-ing the planner / scheduler if the sheet is not fol-lowing the itinerary sufficiently closely (the contractis being breached). In general, the more componentsare able to model their own internal processes andanalyze their own performance, in other words, themore self-aware they are, the more autonomy anddelegation can be reliably performed in a system.This kind of self-awareness is akin to system diagno-sis, but at a variety of hierarchical levels, and per-formed in an ongoing manner, in parallel with sys-tem operation.

We have been exploring the potential of this typeof diagnosis at the planning level. If the printing sys-tem can determine and monitor the current healthstatus of the components of the machine, the plan-ner / scheduler can then choose plans based on thatknowledge. For example, if exceptions appear tooccur more often whenever sheets are routedthrough a particular module, then perhaps there issomething wrong with that module. The planner /scheduler can then choose a plan either to help iden-tify the problem or to simply avoid the module.Since the goals of the system as a whole includemaintaining high productivity, it is desirable to per-form this type of diagnosis in parallel with the nor-mal machine operation. This idea we call pervasivediagnosis (Kuhn et al. 2008).

Articles

FALL 2013 83

Results: Online Reconfigurable Systems

The software described above was implemented inthe hypermodular printer prototype. The printer iscapable of running at approximately 210 pages perminute at full speed, using all four 55-page-per-minute print engines. With individual highwaymodules artificially “failed” (or removed), the samethroughput can typically be achieved; if a printengine or a module on an on- or off-ramp is failed,throughput is reduced by approximately 25 percent.Failing multiple modules at once has different effectson the throughput, depending on the choice of mod-ules.

Online rerouting in response to a change in con-figuration has also been demonstrated in thismachine, though at a lower sheet speed and density.If a module is artificially “failed” online, this changein configuration is reported up to the planner /scheduler, which can then reroute the sheets in thesystem to avoid the failed module. Additionally, if asheet is artificially “jammed” in the machine, the sys-tem can detect the jam and report it to the planner /scheduler, which will then replan to replace thatsheet, purge any sheets that are now out of order, andreroute sheet traffic around the jam.

The planner / scheduler can handle a wide range ofsystem configurations. As mentioned above, theplanner / scheduler has been demonstrated on otherreal printer designs as well as many hypothetical sys-tems (Ruml et al. 2011). On the hypermodular pro-totype configuration, the planner / scheduler canplan a sheet in an average of 92.8 milliseconds (thetime varies depending on the sheet requirements andthe current states of the sheets in flight). This time issignificantly less than the 270 millisecond averageneeded to support the full machine throughput of220 pages per minute. With other printer designs, allof which are less complex than the hypermodularprototype, the planner / scheduler takes less time persheet on average.

In the case of a paper jam, the planner / scheduleris able to reroute up to five sheets on the fly (in realtime) for the hypermodular printer. This numbermay seem low, but replanning is harder than nomi-nal planning by a factor exponential in the numberof in-flight sheets. For simpler protytpe systems, theplanner / scheduler can reroute all the sheets in thesystem in real time, so an increase in computingcapacity may assist in the hypermodular printer case.Our planner / scheduler is the first to demonstratethis generic, automatic exception handling for print-ers, rather than relying on machine-specific rules oroperator intervention.

In addition, the planner / scheduler has beenextended to other domains including food packaging(Do et al. 2011a) and material control systems (Do etal. 2011b). Though we have not yet demonstrated

the capability of adding new components to a systemonline, we believe the planner / scheduler and con-trol architecture described here are capable of han-dling this aspect of online reconfigurability as well.

Planner/Scheduler Lessons LearnedExploration of the hypermodular printer system hasgiven us a number of insights into the design andcontrol of such distributed, adaptable machines. Asone example, during the course of the system devel-opment we learned some significant lessons regard-ing the planner / scheduler and its use in real-worldapplications.

Modeling for planning is important: In this work, wemodeled printers using a specialized, high-level,human-friendly representation in which machinemodules and the connections between them are themain themes of the language. We then compiled thisrepresentation into the planner / scheduler input lan-guage, taking the capabilities of different modulesalong with their interconnections and producingaction schemata. Through discussion with our usersand industrial partners, we feel that the machine-centered language involving modules, machineinstances, and interconnections is easier for them tounderstand and accept, while the compiled-downrepresentation makes it much easier for us to adoptSTRIPS (Stanford Research Institute Problem Solver)planning techniques. Figure 6 shows the STRIPS-likerepresentation of an action that prints one side of agiven sheet and inverts it after printing. Further, wefound that because we understood the search algo-rithm (regression with three-value state representa-tion) and the heuristics (planning graph with mutex-es) used by the planner, we could manipulate themodeling of the actions, goals, and initial states toproduce quite different computational results. Thesame domain physics can be represented differently,even if limited to STRIPS, and finding the right matchwith the chosen search strategy can dramaticallyaffect the planner’s performance. As applicationdevelopers, not having to work with a fixed bench-mark domain representation allows us to exploitanother dimension in modeling to improve our plan-ner’s performance.

The most suitable planning algorithm depends on theapplication specifications: Even after formulating ourdomain using an extension of STRIPS, we wentthrough several implementations of different plan-ning algorithms before settling on the current one.Our first version was a lifted partial-order planner,which we still think is the more elegant algorithm.We then implemented a grounded forward state-space planner, because that approach has dominatedthe recent international planning competitions. Werealized, however, that a combination of the con-straint that sheets in the same print job should be fin-ished in order and our objective function of mini-mizing the finishing time is not suitable for forward

Articles

84 AI MAGAZINE

state-space search (Ruml et al. 2011). We finally set-tled on a backward state-space framework, which ismuch faster in our domain. The lesson we drew fromthis experience is that just because some approachworks best in a wide range of benchmark domains inthe competition does not mean that it is the bestchoice for a given application, and if it does notwork, it does not mean that other less popularapproaches cannot do significantly better. Therefore,understanding a domain, the important constraintsinvolved, the objective function, and how differentplanning algorithms work can help in selecting themost suitable strategy. Competition results are not areplacement for understanding the variety of appli-cable planning algorithms.

There are many uses for a planner: Besides its mainjob of controlling different printers, the planner /scheduler has also been used extensively for systemanalysis purposes. Thus, the planner / scheduler hasbeen tested against (1) different printer designs tohelp determine the better ones, and (2) printers withvarious broken modules to test the reliability of eachprinter. These analyses can help a product group todecide which printer to build for a given purpose.Another use has been to test the performance of theupstream job submission and sequencing methods.Through these experiences, we learned that there aremany potential applications of a planner / schedulerbeyond direct machine control.

Design Trade-OffsIn addition to choices made in the individual soft-ware components’ designs, such as those described

here for the planner, there were many trade-offsinvolved in designing the overall system architectureand the interplay of the components. These trade-offs included decisions about the level of modulari-ty of the software as well as the level of intelligenceof the various components. Unfortunately, there areno unique answers for design questions such asthese, as the decisions generally must depend on thedetails of the implementation. In making choices,we were guided as much as possible by the designprinciples of multiscale control, encapsulation, andautonomy/delegation/escalation, and by the realitiesof our implementation. For example, we had achoice regarding what the finest-grained level ofmodeling and control would be. We could choose tohave a “module” include one three-way and multi-ple nips, or instead have separate nip and three-way“modules.” We chose the latter option for a higherdegree of modularity and reconfigurability. Thetrade-off, of course, was a greater complexity in theplanner / scheduler algorithms, because of thegreater number of modules to represent. This numer-ical explosion can be combated by grouping the con-trol modules into units for the purpose of planning,but again this choice requires a trade-off in loss ofgenerality, a larger variety of more specific moduletypes, and a mismatch between modeling granulari-ty at different levels of the control hierarchy. Thisgrouping into macromodules proved necessary for alimited number of subsystems (the on- and off-ramps) for computational reasons. For similar rea-sons of flexibility, reconfigurability, and modelinguniformity we chose to have one controller associat-

Articles

FALL 2013 85

PrintSimplexAndInvert(?sheet, ?side, ?color, ?image)

Location(?sheet,Printer1-Input)Blank(?sheet)SideUp(?sheet,?side)Opposite(?side, ?other-side)CanPrint(MarkingEngine, ?color)Location(?sheet, Printer1-Output)

HasImage(?sheet,?side,?image)¬Blank(?sheet)¬SideUp(?sheet, ?side)SideUp(?sheet,?other-side)[13.2 secs,15.0 secs]0.1 secsMarkingEngine at ?start + 5.9 for 3.7 secs

preconditions:

effects:

duration:set-up time: allocations:

¬Location(?sheet, Printer1-Input)

Figure 6. A Simple STRIPS-Like Temporal Action Specification with Resource Allocation.

This figure originally appeared in Ruml, et al. (2011) and is reproduced with permission of JAIR.

ed with each processor, and one processor associatedwith each physical unit (nip, three-way, or straight-through).

The other set of major trade-offs involved where toput the logical boundaries between levels of the con-trol hierarchy. For example, we had a choice as to whatkind of actions the planner / scheduler should pro-duce as output. The planner / scheduler output mustbe converted at some point to trajectories for the nipsto track. The planner / scheduler could output detailedcommands to the nips and directors, which wouldallow it to be more directly responsive to the system,but then it would have to be correspondingly morecomplex, would have to know many more detailsabout the physical system, and could not then com-plete its calculations in the required time frame. At theother extreme, the planner / scheduler could lumpmany modules together into a highway unit, forexample, with several possible entrances and exits,and output actions such as “entrance 1 to exit 2.” Inthis case, significant interpretation would be neededbetween the planner / scheduler output and theinputs to the control actuators, and the macromoduleunits would need to be somehow configuration spe-cific. The choice we made of having the planner /scheduler output discrete actions such as “left to right”for each module (except for the on- and off-ramps, asmentioned above; see figure 3) lies somewhere inbetween these extremes and has the advantage of cor-responding to the level of modularity of the hardwareand the control layer. It therefore provides for greaterease in reconfiguration and has a complexity inbetween those of the extreme solutions just described.On the other hand, this choice requires some otherentity to interpret between the planner / scheduleroutput and the required inputs to the nips and direc-tors, which is at a greater level of detail. The inter-preter must also translate feedback coming from themodules up to the planner. Putting this computationin the modules themselves introduces another set oftrade-offs, including greater module processing needsand communication among the modules (to ensurecoordination). The decision to put this required func-tionality in the sheet controller allowed all the inter-module computations to be encapsulated in one loca-tion, as well as saving on module controllercomplexity.

ConclusionsWe believe that the design principles espoused hereare key to the success of our system. Though hierar-chical control systems are very common, we feel thatthe ideas of encapsulation, delegation, autonomy,and escalation, as implemented, for example, in themodel-based contract interface, allow for clear com-munication and a high level of component inde-pendence and reliability, which are required in sucha large-scale, online reconfigurable system. A high

level of component self-awareness is required to sup-port this framework.

Online reconfigurable systems offer many poten-tial gains in the areas of flexibility and customizabil-ity of product design and graceful degradation in theface of system faults. As such, the research describedhere has applications in a range of areas outsideprinting, including other manufacturing domains aswell as military systems. As we have demonstrated, ahigh level of modularity combined with integrated,model-based software is one path to achieving onlinereconfigurability. Extreme modularity admittedlyadds system complexity, though, which affects bothplanning/scheduling complexity and communica-tion overhead. It is likely that, for a particular targetsystem, the benefits of online reconfigurability couldbe realized by using different levels of modularity indifferent parts of the machine, thus reducing some ofthe communication overhead and system complexi-ty but retaining all the desired flexibility. Additional-ly, communication and computation are improvingat a significant pace, particularly relative to mechan-ical hardware, giving higher levels of modularitymore of an advanted in the future. Determining theideal level of modularity, or combination of levels,for a particular application is still more a matter of artthan science, however.

AcknowledgmentsThe authors would like to thank all of our many cur-rent and former colleagues at PARC and Xerox whohave contributed to this project. Special thanks toBob Lofthus and Martin Krucinski. This work wasfunded by Xerox.

ReferencesAzab, A.; ElMaraghy, H.; and Samy, S. 2009. ReconfiguringProcess Plans: A New Approach to Minimize Change. InChangeable and Reconfigurable Manufacturing Systems, ed. H.A. ElMaraghy, chapter 10, 179–194. Berlin: Springer-Verlag.

Barták, R. 2002. Visopt ShopFloor: On the Edge of Planningand Scheduling. In Proceedings of the Eighth InternationalConference on Principles and Practice of Constraint Program-ming, Lecture Notes in Computer Science, 587–602. Berlin:Springer.

Bi, Z. M.; Lang, S. Y. T.; Shen, W.; and Wang, L. 2008. Recon-figurable Manufacturing Systems: The State of the Art. Inter-national Journal of Production Research 46(4): 967–992.

Biegelsen, D.; Crawford, L.; Duff, D.; Eldershaw, C.;Fromherz, M. P. J.; Kott, G.; Larner, D.; Mandel, B.; Moore,S.; Preas, B.; Schmitz, G.; and Swartz, L. 2009. Hypermodu-lar Parallel Printing Systems. In Proceedings of the Interna-tional Conference on Digital Printing Technologies and DigitalFabrication 2009 (NIP25), 184–187. Springfield, VA: Societyfor Imaging Science and Technology.

Biegelsen, D. K.; Crawford, L. S.; Do, M. B.; Duff, D. G.;Eldershaw, C.; Fromherz, M. P. J.; Hindi, H.; Kott, G.; Larn-er, D. L.; Mandel, B.; Moore, S.; Preas, B. T.; Ruml, W.;Schmitz, G. P.; Schwartz, L.-E.; and Zhou, R. 2011. Integrat-ed Parallel Printing Systems with Hypermodular Architec-ture. In Parallel Processing for Imaging Applications, SPIE Pro-

Articles

86 AI MAGAZINE

ceedings 7872. Bellingham, WA: Society of Photo-OpticalInstrumentation Engineers.

Bonet, B., and Geffner, H. 1999. Planning as HeuristicSearch: New Results. In Proceedings of the 5th European Con-ference on Planning, Lecture Notes in Computer Science, ed.S. Biundo and M. Fox 359–371. Berlin: Springer.

Boutilier, C.; Dean, T.; and Hanks, S. 1999. Decision-Theo-retic Planning: Structural Assumptions and ComputationalLeverage. Journal of Artificial Intelligence Research 11: 1–91.

Chien, S. A.; Knight, R.; Stechert, A.; Sherwood, R.; andRabideau, G. 1999. Using Iterative Repair to Improve theResponsiveness of Planning and Scheduling forAutonomous Spacecraft. In Proceedings of the Sixteenth Inter-national Joint Conference on Artificial Intelligence. San Francis-co: Morgan Kaufmann Publishers.

Crawford, L.; Hindi, H.; Zhou, R.; and Larner, D. 2009. Syn-chronized Control in a Large-Scale Networked DistributedPrinting System. In Proceedings of the IEEE International Con-ference on Robotics and Automation. Piscataway, NJ: Instituteof Electrical and Electronics Engineers.

Dechter, R.; Meiri, I.; and Pearl, J. 1991. Temporal Con-straint Networks. Artificial Intelligence 49(1–3): 61 – 95.

desJardins, M. E.; Durfee, E. H.; Ortiz, Jr., C. L.; and Wolver-ton, M. J. 1999. A Survey of Research in Distributed, Con-tinual Planning. AI Magazine 20(4): 13–22.

Do, M. B.; Lee, L.; Zhou, R.; Crawford, L. S.; and Uckun, S.2011a. Online Planning to Control a Packaging Infeed Sys-tem. In Proceedings of the 23rd International Conference onInnovative Applications of Artificial Intelligence (IAAI-11). PaloAlto, CA: AAAI Press.

Do, M. B.; Uckun, S.; Crawford, L. S.; Zhang, Y.; Ohashi, A.;Okajima, K.; Hasegawa, F.; Kawano, Y.; and Tanaka, K.2011b. Online Planning for a Material Control System forLiquid Crystal Display Manufacturing. In Proceedings of the21st International Conference on Automated Planning andScheduling (ICAPS). Palo Alto, CA: AAAI Press.

Do, M. B.; Ruml, W.; and Zhou, R. 2008a. On-Line Planningand Scheduling: An Application to Controlling ModularPrinters. In Proceedings of the 23rd AAAI Conference on Artifi-cial Intelligence. Palo Alto, CA: AAAI Press.

Do, M.; Ruml, W.; and Zhou, R. 2008b. Planning for Modu-lar Printers: Beyond Productivity. In Proceedings of the 18thInternational Conference on Automated Planning and Schedul-ing (ICAPS). Palo Alto, CA: AAAI Press.

El Maraghy, H. A., ed. 2009. Changeable and ReconfigurableManufacturing Systems. Berlin: Springer-Verlag.

ElMaraghy, H. A. 2006. Flexible and Reconfigurable Manu-facturing Systems Paradigms. International Journal of FlexibleManufacturing Systems 17(4): 261–276.

Fox, M., and Long, D. 2003. PDDL2.1: An Extension toPDDL for Expressing Temporal Planning Domains. Journal ofArtificial Intelligence Research 20: 61–124.

Fox, M.; Gerevini, A.; Long, D.; and Serina, I. 2006. Plan Sta-bility: Replanning versus Plan Repair. In Proceedings of the16th International Conference on Automated Planning andScheduling (ICAPS), 212–221. Palo Alto, CA: AAAI Press.

Frank, J., and Jónsson, A. 2003. Constraint-Based Attributeand Interval Planning. Constraints 8(2): 339–364.

Fromherz, M. P. J.; Bobrow, D. G.; and de Kleer, J. 2003.Model-Based Computing for Design and Control of Recon-figurable Systems. AI Magazine 24(4): 120–130.

Fromherz, M. P. J.; Crawford, L. S.; and Hindi, H. A. 2005.

Coordinated Control for Highly Reconfigurable Systems. InHybrid Systems: Computation and Control (HSCC). Berlin:Springer-Verlag.

Ghallab, M., and Laruelle, H. 1994. Representation andControl in IxTeT, a Temporal Planner. In Proceedings of theSecond International Conference on Artificial Intelligence Plan-ning Systems, 61–67. Palo Alto, CA: AAAI Press.

Heisel, U., and Meitzner, M. 2004. Progress in Reconfig-urable Manufacturing Systems. Journal for Manufacturing Sci-ence and Production 6(1-2): 1–8.

Hindi, H.; Crawford, L.; Zhou, R.; and Eldershaw, C. 2008.Efficient Waypoint Tracking Hybrid Controllers for DoubleIntegrators Using Classical Time Optimal Control. In Pro-ceedings of the 2008 IEEE Conference on Decision andControl. Piscataway, NJ: Institute of Electrical and Elec-tronics Engineers.

Koehler, J., and Hoffmann, J. 2000. On Reasonable andForced Goal Orderings and Their Use in an Agenda-DrivenPlanning Algorithm. Journal of Artificial Intelligence Research12: 338–386.

Koren, Y., and Shpitalni, M. 2011. Design of ReconfigurableManufacturing Systems. Journal of Manufacturing Systems29(4): 130–141.

Koren, Y.; Heisel, U.; Jovane, F.; Moriwaki, T.; Pritschow, G.;Ulsoy, G.; and Brussel, H. V. 1999. Reconfigurable Manu-facturing Systems. Annals of the CIRP 48(2): 527–540.

Kuhn, L.; Price, B.; De Kleer, J.; Do, M.; and Zhou, R. 2008.Pervasive Diagnosis: The Integration of Diagnostic Goalsinto Production Plans. In Proceedings of the 23rd NationalConference on Artificial Intelligence, 1306–1312. Palo Alto,CA: AAAI Press.

Mehrabi, M. G.; Ulsoy, A. G.; Koren, Y.; and Heytler, P. 2002.Trends and Perspectives in Flexible and ReconfigurableManufacturing Systems. Journal of Intelligent Manufacturing13(2): 135–146.

Muscettola, N. 1994. HSTS: Integrating Planning andScheduling. In Intelligent Scheduling, ed. M. Zweben and M.Fox, chapter 6, 169–212. San Francisco: Morgan KaufmannPublishers.

Policella, N.; Cesta, A.; Oddi, A.; and Smith, S. F. 2007. FromPrecedence Constraint Posting to Partial Order Schedules.AI Communications 20(3): 163–180.

Pryor, L., and Collins, G. 1996. Planning for Contingencies:A Decision-Based Approach. Journal of Artificial IntelligenceResearch 4: 287–339.

Ruml, W.; Do, M. B.; Zhou, R.; and Fromherz, M. P. J. 2011.On-Line Planning and Scheduling: An Application to Con-trolling Modular Printers. Journal of Artificial IntelligenceResearch 40: 415–468.

Ruml, W.; Do, M. B.; and Fromherz, M. P. J. 2005. On-LinePlanning and Scheduling for High-Speed Manufacturing. InProceedings of the 15th International Conference on AutomatedPlanning and Scheduling (ICAPS-05). Palo Alto, CA: AAAIPress.

Smith, D. E., and Weld, D. S. 1999. Temporal Planning withMutual Exclusion Reasoning. In Proceedings of the SixteenthInternational Joint Conference on Artificial Intelligence, 26–333.San Francisco: Morgan Kaufmann Publishers.

Smith, S. F., and Cheng, C.-C. 1993. Slack-Based Heuristicsfor Constraint Satisfaction Scheduling. In Proceedings of theEleventh National Conference on Artificial Intelligence, 139–144. Palo Alto, CA: AAAI Press.

Articles

FALL 2013 87

Tang, L.; Koren, Y.; Yip-Hoi, D. M.; and Wang, W. 2006.Computer-Aided Reconfiguration Planning: An ArtificialIntelligence-Based Approach. Journal of Computing and Infor-mation Science in Engineering 6(3): 230–240.

Wah, B. W., and Chen, Y. 2003. Partitioning of TemporalPlanning Problems in Mixed Space Using the Theory ofExtended Saddle Points. In Proceedings of the 2003 IEEE Inter-national Conference on Tools with Artificial Intelligence. Piscat-away, NJ: Institute of Electrical and Electronics Engineers.

Wiendahl, H.-P.; ElMaraghy, H. A.; Nyhuis, P.; Zah, M. F.;Wiendahl, H.-H.; Duffie, N.; and Brieke, M. 2007. Change-able Manufacturing — Classification, Design and Opera-tion. Annals of the CIRP 56(2): 783–809.

Lara Crawford is with the Intelligent Systems Lab (ISL) atthe Palo Alto Research Center. She received a Ph.D. in bio-physics and an M.S. in electrical engineering science fromthe University of California at Berkeley. Her research inter-ests include control and coordination of distributed,embedded systems, the interface between planning andcontrol, optimization, modeling, simulation, robotics, andenergy systems.

Minh Do is a senior research scientist at SGT Inc. workingin the Planning and Scheduling Group at the NASA AmesResearch Center. He received a Ph.D. in computer sciencefrom the Arizona State University. His main research inter-est is in automated planning with concentration on onlinecontinual planning for manufacturing systems with com-plex temporal and resource constraints. He has also workedon other topics in planning such as partial-satisfaction plan-ning, integrating planning and diagnosis, and constraint-based planning.

Wheeler Ruml is an associate professor at the University ofNew Hampshire (UNH). He received a Ph.D. in computerscience from Harvard University in 2002. Before joiningUNH in 2007, he was a member of the research staff andarea manager for embedded reasoning at the Palo AltoResearch Center. His research interests include heuristicsearch and planning, with an emphasis on time-aware deci-sion making.

Haitham Hindi is with Walmart Labs. He was with ISL atPalo Alto Research Center from 2003–2011. He holds a B.Sc.from Imperial College in physics, and an M.S. and Ph.D.from Stanford University in electrical engineering. Hisresearch is in control and optimization and their applica-tion to real-world problems, including online advertising,radiation treatment optimization, energy management sys-tems, dynamic pricing, networked and hybrid control,printing and manufacturing networks, particle accelerators,and disk drives.

Craig Eldershaw is with the Hardware Systems Laboratoryat the Palo Alto Research Center. He received a D.Phil. fromthe University of Oxford on the topic of robot motion plan-ning. His specialty is integrating mechanical, electrical andsoftware designs within a complex, distributed, system. Thework described in this article was a natural extension of hisprevious NASA and U.S. Defense Advanced Research Proj-ects Agency–funded work on modular robotics.

Rong Zhou is with ISL at the Palo Alto Research Center,where he is a member of the High Performance Analyticsarea. He received a Ph.D. in computer science from Missis-

sippi State University. His main research interests includecombinatorial optimization and automated planning withconcentration on large-scale graph search using parallel andmemory hierarchy aware search methods. He has alsoworked on user interface design and human factors.

Lukas Kuhn is a cofounder of Zenhavior. Prior to Zenhav-ior, he worked as a senior system engineer at QualcommR&D on the first generation of mass-scale embedded con-textual computing. He received a Ph.D. and a diploma(equivalent to a Master’s) in computer science from theTechnical University of Munich and the University ofMunich, respectively. Before joining Qualcomm in 2010, hewas a research assistant with the Embedded Reasoning Areaat the Palo Alto Research Center, where he worked on theintegration of model-based diagnosis and planning. His cur-rent work focuses on reasoning for contextual awareness,mobile computing, and behavior modeling.

Markus Fromherz is the chief innovation officer, healthcare, at Xerox. Previously, he was vice president and direc-tor of the Intelligent Systems Lab at the Palo Alto ResearchCenter. He received his Ph.D. in computer science from theUniversity of Zurich, Switzerland, and his M.S. in computerscience from ETH Zurich. His research interests are in thedomain of intelligent embedded software, including con-straint-based modeling, model-based planning, scheduling,and control, andmodel-based design analysis and optimiza-tion.

David Biegelsen was a charter member of Xerox PARC andis currently a Research Fellow. His fields of expertise includeacousto-optic interactions, electron spin resonance, andfundamental aspects of disordered semiconductors, laser-induced thin-film crystallization, scanning tunnelingmicroscopy, heteroepitaxial growth, and new fabricationmethods and use of complex “smart matter” systems.Biegelsen holds more than 100 U.S. patents. He is a Fellowof the APS and has been an editorial board member ofApplied Physics Letters and divisional associate editor of Phys-ical Review Letters. Biegelsen gets his kicks from learning newconcepts and using them in novel ways.

Johan de Kleer is area manager and research fellow in theIntelligent Systems Laboratory. He received his Ph.D. andS.M. from the Massachusetts Institute of Technology. deKleer’s interests include large-scale inference, qualitative rea-soning, knowledge representation, model-based diagnosis,and truth maintenance systems. He is a fellow of the Asso-ciation for the Advancement of Artificial Intelligence andthe Association of Computing Machinery.

Dan Larner has a B.S. in mechanical engineering and a B.S.in computer science and engineering from MIT, and an M.S.in computer science from Stanford. He has worked on avariety of systems in both physical and software areas. He iscurrently a mechanical engineer on autonomous vehicles atGoogle.

Articles

88 AI MAGAZINE