online discrete event supervisory control of hybrid dynamical systems using embedded simulation

ARTICLE IN PRESS

0967-0661/$ - se

doi:10.1016/j.co

�CorrespondE-mail addr

[email protected]

Control Engineering Practice 16 (2008) 1004–1021

www.elsevier.com/locate/conengprac

Online discrete event supervisory control of hybriddynamical systems using embedded simulation

James P. Millana,�, Siu D. O’Youngb

aInstitute for Ocean Technology, National Research Council, St. John’s, Nfld., CanadabFaculty of Engineering and Applied Science, Memorial University of Newfoundland, St. John’s, Nfld., Canada

Received 7 November 2006; accepted 22 November 2007

Available online 28 January 2008

Abstract

This paper describes a technique that automates the synthesis of supervisory controllers for plants that have a mixture of discrete and

continuous dynamics, that is, plants with hybrid dynamics. The control technique utilizes embedded simulations to predict the future

system trajectories of a system and then discrete control decisions are made in order to preserve the system’s safety. This approach is

rooted in the principles of discrete event supervisory control theory, in which the legal behaviour of the controlled system is specified

abstractly by a regular language specification, as generated, for example, by a finite state machine.

Crown Copyright r 2007 Published by Elsevier Ltd. All rights reserved.

Keywords: Discrete-event systems; Control; Hybrid systems; On-line control; Supervisory control; Ship control

1. Introduction

Theoretical developments in the area of hybrid systemcontrol have not been widely applied to any practicalindustrial problems, although a few examples exist, forexample, engine idle speed control (Balluchi, Natale,Sangiovanni-Vincentelli, & van Schuppen, 2004), air-trafficcontrol (Bayen & Tomlin, 2003) and chemical batchprocessing (Potocnik, Bemporad, Torrisi, Music, &Zupancic, 2004). The intractability of control computa-tions for such systems, coupled with a steep learning curvefor control system designers, acts as a barrier to the wideradoption of hybrid system theory by industry. Never-theless, with the proliferation of embedded and distributedcontrol systems, hybrid system theory must play anincreasingly important role in the future development ofcomplex control systems.

e front matter Crown Copyright r 2007 Published by Elsevie

nengprac.2007.11.007

ing author. Tel.: +1709 772 2472; fax: +1 709 772 2462.

esses: [email protected] (J.P. Millan),

un.ca (S.D. O’Young).

1.1. Software for verification and synthesis

A variety of software tools are available for bothhybrid system analysis and verification such as HyTech(Henzinger, Ho, & Wong-Toi, 1997), and CheckMate

(Chutinan & Krogh, 2003). In verification problemsthough, the controller is assumed to be given. Thus, thecontroller synthesis is a manual process that relies upon thedomain knowledge and intuition of the designer. Ideally,the control system designer needs a way of automating(or at least semi-automating) the synthesis of the controllerbased on models of the plant and the desired (specified)behaviour. A MATLABs toolbox is available (Torrisi &Bemporad, 2004) for simulation and control synthesis forhybrid systems modelled by mixed logical dynamical(MLD) and piecewise affine (PWA) hybrid models, butthe designer must settle for linearized continuous dy-namics. With hybrid verification tools, the computationalburden of an exhaustive reachability requires the use ofsimplified continuous dynamical models. In industry,control system development consists of testing designs ina simulation environment. Typically, a general purposesimulation tool such as MATLAB Stateflows, is used toevaluate the safety and correctness of a controller design

r Ltd. All rights reserved.

www.elsevier.com/locate/conengprac

dx.doi.org/10.1016/j.conengprac.2007.11.007

mailto:[email protected]

mailto:[email protected]

ARTICLE IN PRESSJ.P. Millan, S.D. O’Young / Control Engineering Practice 16 (2008) 1004–1021 1005

under a variety of conditions. Due to the ad hoc choice ofthe test conditions, this technique may miss the particularcombination of conditions that leads to design failure.Unfortunately, the ad hoc simulation technique is theaccepted industry solution, which leaves two problemsunresolved: For hybrid systems, how does the designer takea specification and produce the controller design? And,how can the resulting controller design be verified to becorrect?

1.2. Discrete event supervision

This paper presents a technique that addresses bothdesign correctness and automated synthesis for hybridsystems. The approach is to harness the power ofindustrially proven system modelling (simulation tools).These nonlinear continuous models of the plant dynamicsare wrapped in a discrete abstraction layer that is based onquantizing both the input and output spaces of thecontinuous model. The resulting discrete abstraction ofthe continuous dynamics, combined with a discrete event(DE) specification in a limited horizon reachabilitycomputation, produces a DE controller that, within alimited space and time, enforces a specified behaviour. Thecontroller synthesis process can be automated and theresulting design is guaranteed to enforce the specification(Millan & O’Young, 2006).

Limited lookahead (LL) is a strategy for reducing thecomputational complexity of the supervisory controllersynthesis; it has been extensively studied in a discrete eventsystems (DESs) setting by Chung, Lafortune, and Lin(1992). In Raisch and O’Young (1998), discrete abstrac-tions based on the truncated time history of discrete-timeLTI continuous models were used to synthesize DESsupervisory controllers. Others, Su, Abdelwahed, Karsai,and Biswas (2003) and Abdelwahed, Su, and Neema(2005), have also used discrete abstractions of switchedcontinuous systems in an LL framework to effect controlover hybrid systems, but they do not address the issue ofblocking, which this paper deals with. Also similar isStursberg (2004), in which the nonlinear continuousdynamics are retained as embedded simulations, and agraph search algorithm has been described for optimalhybrid control. The approach described in this paperdiffers in that switching is not limited to discrete timeintervals and controller graph pruning is done in amaximally permissive sense with respect to safety as iscommonly done in optimal DES supervisory control(Ramadge & Wonham, 1987). The automated controllersynthesis in this paper is based on specifying the legalbehaviour of the system in terms of a finite stateautomaton; the designer specifies the performance andsafety of the closed-loop system by describing the sequenceof operations and how the system coordinates theseoperations with other agents. This differs from theseemingly similar technique of model predictive control(MPC), which strives to prevent violation of both input

and output constraints through a continuous-domainoptimization of the controlled and manipulated variables(Qin & Badgwell, 2003).

1.3. Paper outline

The paper is organized as follows. Section 2 describes themodelling framework that is employed to represent hybridplant models. Section 3 discusses the synthesis of a DEsupervisor for these hybrid models. The controller isintended to be continuously synthesized in an onlinefashion and a method is given that guarantees that thecontrol system can be safely shut down. Sections 2 and 3are meant to give a brief background of the basis for thehybrid system modelling and DE controller synthesistechniques. The interested reader is referred to Millan(2006) in which a full treatment of the model, controllersynthesis and algorithmic implementation are given. Themain result of this paper is given in Section 4, in which asupervisory controller is synthesized that enforces safetywhile coordinating the motion of a pair of dynamicallypositioned ships.

2. Modelling framework

The hybrid model used in this paper is a blend of theswitched system (Lin & Antsaklis, 2005) and discreteabstraction (Raisch, 2000) approaches to hybrid systemmodelling. The abstraction approach permits the resultingmodel to be synchronized with DE processes. Thus, it ispossible to construct larger plant models by formingsynchronous products of this plant model with otherDES models, and it is possible to use DES supervisorycontrol synthesis techniques to construct supervisors thatenforce legal behaviour as specified by some DE specifica-tion (Ramadge & Wonham, 1989).

2.1. Continuous system model

A continuous dynamical model as an ordinary differ-ential equation is

_x ¼ f ðx; tÞ. (1)

The dynamics described by Eq. (1) will serve as aplaceholder for the complex continuous dynamics thatmay be produced by industrial/commercial simulationpackages. In addition, although this equation representsan unforced state equation, it does not imply the possibilitythat the input to the system is zero; for example, consideran input u as a function of both state and time:

u ¼ gðx; tÞ

then function g can simply be absorbed into the unforcedstate equation of f because both the dynamics and the inputare functions of time and state only. The continuous system

ARTICLE IN PRESS

input

J.P. Millan, S.D. O’Young / Control Engineering Practice 16 (2008) 1004–10211006

model (CSM) with abstraction framework is defined asfollows:

Definition 1 (CSM). Let a CSM, s, be defined as a triples ¼ ðf ;C; x0Þ, where

f is a Lipschitz-continuous ordinary differential equa-tion, _x ¼ f ðx; tÞ.

C is a finite set of partitioning functionals, C ¼fFiðxÞ : R

n ! R; i 2 N;a1g, where each Fi is a continu-ously differentiable functional.

x0 is the initial condition, x at some initial simulationtime t0.

The state space of the continuous model is partitionedinto a finite quotient set X of equivalence classes Qj

X ¼ fQj � Rn : aþ 1pjp2ag ¼ Rn=�p,

where the equivalence relation for ðx1; x2Þ 2 Rn � Rn isdefined as

x1�px2 () signðFiðx1ÞÞ � signðFiðx2ÞÞ ¼ 1; 8i1pipN.

Each equivalence class Qj 2 X can be thought of as adiscrete state as in a finite state automaton. Theseequivalence classes, or regions of the state space will beassociated with a discrete state label for convenience. Acontinuous trajectory of the system x on some time intervalDT ¼ ½t0; t1� for some initial condition is a solution of aninitial value problem (IVP). With f Lipschitz continuous, xexists and is unique. If x crosses a given hypersurface ofany functional F i 2 C, NðFiÞ ¼ fx 2 Rn : FiðxÞ ¼ 0g, thenthis is considered as a discrete state transition.

For more information on the functional partitioningtechnique, see Koutsoukos, Antsaklis, Stiver, and Lemmon(2000). In general, the actual mechanism for partitioning isunimportant; the functional partitioning is merely sug-gested as one particular technique for producing discreteabstractions of the continuous state space.

output

s0

s1

si

Γ

Fig. 1. A schematic representation of the SCM.

2.2. Switched continuous model

The switched continuous model (SCM) is an automaton-like model consisting of a collection of CSMs (each with itsown discrete state-space partitioning) as described inthe previous section. One can think of these modelsas representing different operating modes of a systemhaving different dynamics or, alternatively, the samecontinuous system dynamics with different input signalsapplied. The SCM specifies this set of models and control iseffected by switching between each of them to achieve theoverall goal of the control system. In the control frame-work laid out in this paper, the switching occurs eitherautonomously (from within the system itself) or due to anexternal discrete command from a DES controller.Unscheduled events or disturbance are not contemplatedwithin the scope of this paper, thus all effects areconsidered as being modelled.

Definition 2 (SCM). Let an SCM be defined as anautomaton-like triple G ¼ ðF;G; s0Þ, where

F is a set of CSMs, possibly infinite, each with its owndiscrete abstraction as in Definition 1.G is the enabled system function, that embodies a

selection mechanism. Let A ¼ fa �F : 1pjajo1g be theset of nonempty finite subsets of F, and G :F!A.

s0 is the initial CSM.An execution of an SCM is a sequence of selected CSMs

starting with the initial CSM, v ¼ fs0; s1; . . . ; si; . . .g. Thepoint at which the execution changes from one system toanother is known as a choice point. The term choice refersto the ability of the controller at this point to influence thefuture dynamics of the system, by the selection of the nextCSM from a finite set of possible future CSMs GðsÞ 2A.Note that G is analogous to the transition function of afinite state automaton, returning all of the possible futureCSMs that are eligible to switch to at this point in the statespace of the executing system. It is the role of the DEcontroller to select which of these future systems (andthus executions) are eligible. The SCM can be visualized inFig. 1 as a multiplexer with the discrete input determiningthe CSM to execute from a set of possible choices(determined by G).The future continuous behaviour of the SCM G ¼

ðF;G; s0Þ can be predicted by recursively constructing a setof reachable CSMs SR, using the enabled system functionGðsÞ. This set of CSMs represents a set of simulations, eachwith unique initial conditions, continuous dynamics andpartitioning functions. At each choice point, the futureexecution of the system may take any of a finite number ofchoices. The number of choices (branches in the executionof the system at any choice point) is bounded above by r

ARTICLE IN PRESS

h

qmi

qmo

ovfhigh

med

esd

V1

V2

P

Fig. 2. Schematic of tank system.

Table 1

Valve control vector, uc

sin V1 V 2 P

oo 0 0 0

oc 0 1 0

co 1 0 0

cc 1 1 0

sd 0 0 1

Valve open ¼ 1, closed ¼ 0.

J.P. Millan, S.D. O’Young / Control Engineering Practice 16 (2008) 1004–1021 1007

which is defined as the maximum jGðsiÞj;8si. And, if thelookahead horizon, p (the number of choice points into thefuture) is finite, then SR is also finite, but growsexponentially with the lookahead horizon.

jSRjprpþ1 � 1

r� 1; r41. (2)

From an implementation standpoint, it is important thatthe lookahead horizon be as low as possible in order toreduce the computational burden when evaluating the setof Eq. (2).

2.3. Discrete event behaviour

A brief description of the DE form of the SCM is givenin this section. In order to design a DE supervisor for asystem modelled by the SCM, it is desirable to have a DErepresentation of the SCM that can be synchronized withfinite state automata.

Definition 3 (DE set). Let S be a set of event labels suchthat S ¼ Sin [ Sout and Sin \ Sout ¼ tick. The specialevent tick is an event that occurs on some timebase, butnot necessarily at a regular interval.

The previous section stated that a choice point occursdue either to the executing system solution crossing apartition, or to the conclusion of the simulation timeinterval. At this point, an output event sb 2 Sout isgenerated that uniquely identifies the partition and crossingdirection. In the case of the time interval Dt elapsing, atick event is generated.

At a choice point, each eligible CSM is associated with aunique input event sa 2 Sin, thus permitting external finitestate automata that form either the controller or other partsof the system model to affect the execution of the SCM.

Definition 4 (DE transition). Let G ¼ ðF;G; s0Þ be anSCM and let sa 2F, sa ¼ ðf ;C; x0Þ be a CSM. Let xa 2

Rn be a solution to the IVP posed by sa on a time intervalt 2 ½t0; t1Þ then let the DE equivalent transition beta ¼ ðq0;sa;sb; q1Þ, where q0 ¼ ðt0;xaðt0ÞÞ; q1 ¼ ðt1; xaðt1ÞÞ

2 R� Rn are timed continuous states, the endpoints of thesolution xa, and sa 2 Sin and sb 2 Sout are DEs.

The input event sa can be considered as the selectionmechanism or guard event for the transition. The outputevent sb occurs as the result of a continuous trajectorycrossing a hypersurface in the currently executing CSM, oras a result of reaching the end of the designated simulationtime interval, Dt, in which case the output event is tick.The input event initiates a continuous simulation whichleads to the occurrence of the output event. Thus, thereexists a transition ti for each si 2SR. The tree composedfrom the set of all DE transitions ti 2TR is called a hybridtransition graph (HTG), and it concisely captures only theDE behaviour of the SCM on a limited time lookaheadhorizon. The interested reader is referred to Millan (2006)in which the details of the construction of the HTG from

an SCM are given. In addition, the rules for synchronouscomposition of SCMs and finite state automata are coveredin detail. Synchronous composition of these models allowsfor more complex models to be constructed in a modular,hierarchical fashion and also enables the DE supervisorysynthesis techniques to be applied. A simple example isgiven here to demonstrate this modelling framework.

2.4. Modelling example

The modelled system is a tank of liquid (Fig. 2). It isdesired to control the level of this tank through the openingand closing of valves. While this is a trivial example, it is auseful system to study since it has discrete dynamics (valvesopening and closing) and nonlinear continuous dynamics.The controls available for the tank are valves V1, the fillvalve; V 2, the drain valve; P, the purge valve. The purgevalve is a ‘‘use once’’ emergency shutdown (ESD) controlthat is invoked by the system in the event of emergency.Table 1 lists the combinations of valve positions andassociates these actuator combinations with the input eventset Sin ¼ fcc; co; oc; oo; sdg. For example, the input event co

is associated with the actuator control vector uc ¼ ½0; 1; 0�T

which corresponds to valve positions ½V 1;V 2;P� ¼½closed; open; closed�. The shutdown operation is initiatedby the input event sd, which opens only the purge valve todrain the tank. The completion of this operation isindicated by the esd output event. The continuous

ARTICLE IN PRESS

0 90 1200

5

10

15

20

25

30

35

esd

med

unf

hi

ovf

time (s)h

Fig. 3. The predicted state trajectories of the tank for initial condition

x0 ¼ 26 and prediction time interval t ¼ ð0; 90�.


dynamics for the tank can be described by a nonlineardifferential equation. Assume that opening V1 causes aconstant mass flow into the tank qmi, while opening of V 2

or P causes turbulent flow from the tank, then the generalexpression for the tank dynamics (Palm III, 2000) is

_h ¼ qmi

ffiffiffiffiffiffiffiffirgh

pRt2rA

ffiffiffiffiffiffiffiffirgh

pRtprA

" #uc, (3)

where h is the liquid level, r is the density of the liquid andA is the cross-sectional area of the tank. Turbulent flowresistances for valves V2 and P are Rt2 and Rtp,respectively. The five different actuator control vectorsuc, with Eq. (3) yield five distinct dynamical models for thetank. Each of these actuator settings along with a set ofstate partitioning functional forms a separate CSM whichwill be embedded in the SCM as F ¼ fs1; s2; s3; s4; s5g. Forthis example, the CSMs corresponding to input eventsfcc; co; oc; oog share the same set of functionals C1 ¼

fF1;F 2;F 3;F 4g and the CSM for ESD operation (purgevalve P open) hasC2 ¼ fF 5g. The functionals are defined inTable 2 along with the associated output events. Likewise,trajectories initiated by the sd event do not generate outputevents when crossing functionals F 1 � F4.

If the SCM is executed (or predicted) on a 90 s timeinterval, the set of predicted continuous trajectories reach-able from the initial condition h ¼ 26 is pictured in Fig. 3.The figure shows the branching of the system trajectoriesdue to the crossing of a functional or the exceeding of thesimulation time interval. Note that since the shutdown sd

input event and corresponding dynamics do not share thesame partitioning functionals as the other dynamics, therewould be no event generated (and consequently nobranching) due to F 5 while s1 � s4 are executing.

The HTG matching the predicted DE behaviour of theSCM is pictured in Fig. 4. This figure and Fig. 3 aredirectly related and clearly demonstrate the continuous anddiscrete behaviours of the system. As an example,following a single trajectory of the system demonstrateshow the HTG may be constructed: the initial condition ofthe system is h ¼ 26 and the simulation time is t ¼ 0. Thus,in the HTG, the first timed continuous state is q0 ¼ ð0; 26Þ(labelled in Fig. 4 as ½½0; 26��). If input event co is enabled(V1 fill valve closed, V2 drain valve opened), then the levelin the tank begins to lower. When this trajectory crosses thepartition F3 in the # direction, the corresponding output

Table 2

Output events, with associated functionals and hypersurface crossing

directions

sout Functional Zero-crossing Alarm

ovf F1ðhÞ ¼ h� 33 " Over fill

hi F2ðhÞ ¼ h� 31 " High

med F3ðhÞ ¼ h� 18 # Medium

unf F4ðhÞ ¼ h� 15 # Under fill

esd F5ðhÞ ¼ h� 0:5 # Emergency shutdown

event med is generated. In addition, the SCM is at a choicepoint and a new timed continuous state is created, taking itslabel from the simulation time ðt ¼ 51:36Þ and continuousstate value ðh ¼ 18Þ (labelled in the figure as ½½51:36; 18��).Thus, the first discrete transition in the HTG has beenconstructed as t ¼ ðð0; 26Þ; co;med; ð51:36; 18ÞÞ. The newinitial condition for each of the successor CSMs is h ¼ 18,inherited from the previous simulation. Algorithmically, theentire graph can be constructed in this manner by reprodu-cing this process in a depth-first recursion (for example) forthe entire tree, adding each transition to the set TR.For this LL horizon, the plant generates a language

based on the concatenation of the output events intostrings: the � operator will be used to denote concatenation.For example, following the uppermost branch of the HTG,the events med, unf and tick are generated whichconcatenates to the string med � unf � tick. The plantlanguage summarizing the behaviour of the entire graphfor the lookahead horizon of one simulation Dt of 90 sreduces to just four possible strings:

L ¼ fmed � unf � tick;med � tick; hi � tick; tickg.

Note the apparent nondeterminism of this HTG modelwhen only the output events are considered. However, themodel is deterministic, since each transition at any node isguarded by a unique input event (control action). This is anunconstrained plant model, so at each choice point there isa choice of jGðsÞj ¼ 5 control actions. When combined witha specification, the branching will be constrained since notall trajectories will be viable. This is the essence of the DEsupervisor’s role: to limit the possible choices to only thosethat meet the DE specification.

ARTICLE IN PRESS

[[0,26]]

[[51.36,18]]

co/med

[[70,31]]oc/hi

[[90,15.5]]sd/tick

[[90,18.37]]

oo/tick

[[90,26]]

cc/tick

[[73.52,15]]

co/unf

[[90,14.01]]

sd/tick

[[90,15.49]]oo/tick

[[90,18]]

cc/tick

[[90,20.76]]

oc/tick

[[90,27.4]]co/tick

[[90,28.19]]sd/tick

[[90,28.78]]

oo/tick

[[90,31]]

cc/tick

[[90,32.43]]

oc/tick

[[90,12.95]]

co/tick

[[90,13.4]]sd/tick

[[90,14.08]]oo/tick

[[90,15]]

cc/tick

[[90,16.18]]

oc/tick

Fig. 4. HTG equivalent to the reachable space of Fig. 3.


3. Controller synthesis

The HTG is the DE behaviour of the SCM of a systemfor some limited time horizon, and it is suitable forcontroller synthesis using DES control theory. In the DESsupervisory control theory developed by Ramadge andWonham (1987, 1989), a DES supervisory controller issynthesized by forming the synchronous product of finitestate models of plant P and specification S. The optimalsupervisory controller C is the closed-loop controller thatpermits the largest set of joint behaviour in the concurrentconnection of P and S, denoted PkS. This controller isknown as a maximally permissive controller. Generally,these controllers are meant to be formed offline, and anexhaustive search of the discrete state space, combined withthe ‘‘pruning’’ of the unsafe states, ensures the safety of thecontroller. In this case, only a limited horizon model of thestate space is considered. A significant body of work existswith respect to forming online DES supervisors in an LLframework (called LLP for limited lookahead policy),including Chung et al. (1992) and Chung, Lafortune, andLin (1994). The general approach with LLP control is to

compute the controller based on the N-event truncated DEbehaviour LNðGÞ. The set of strings that make up thislanguage are called the pending strings. The pending stringsthat are unambiguously illegal are first removed. Next,different attitudes may be adopted when deciding which ofthe remaining pending strings will be retained in thecontroller language. Taking a conservative attitude assumesthat the next event after each pending string is illegal, andtaking an optimistic attitude assumes that the next event islegal. If all strings are removed from the lookaheadlanguage LNðGÞ ¼ �, then this is considered a run-time

error. Numerous variations on the basic LLP control havebeen proposed including variable lookahead with stateinformation (Hadj-Alouane, Lafortune, & Lin, 1994) andextension of traces beyond the lookahead horizon (Kumar,Chung, & Marcus, 1998).

3.1. Safety and nonblocking

For the online controller proposed in this paper, aruntime error (or controller block) would be catastrophic.In the absence of a legal control choice, the system must

ARTICLE IN PRESSJ.P. Millan, S.D. O’Young / Control Engineering Practice 16 (2008) 1004–10211010

continue (since time cannot be stopped), and since onlyillegal choices remain, it will be forced to do so with acontrol action that ultimately violates the system safety. Inthis paper’s formulation of the LL control problem,blocking implies unsafe. To address this issue, a designpractice from standard industrial control, the concept of anESD mechanism will be incorporated within the controllersynthesis.

Definition 5 (ESD). An ESD state is a safe, idle state.

An ESD is undesirable from the point of view that itperforms no useful work, but it is preferable to apotentially catastrophic safety violation. Therefore, theESD state is a way of gracefully handling the blockingscenario. An LLP attitude called ultra-conservative LLPcontrol uses additional state information. The supervisor,in addition to assuming a conservative stance towards thepending traces, now has the added requirement that itshould enforce controllable behaviour that permits an ESDstate to be reached within N or fewer events (that is, withinthe lookahead horizon). Initially, if such a trace cannot befound, the controller does not exist; in Chung et al. (1992)this was termed as a starting error. If the controller can bestarted, then it will always be possible to drive the system toan ESD state as a last resort at any point during theruntime.

To illustrate this control concept, consider the followingcontrol synthesis example.

Example 6. Let the graph of Fig. 5 be an HTGrepresenting the safe pending behaviour of an SCM (illegaltraces have already been trimmed from the tree). Since thegraph has been generated from the embedded continuoussimulations, the state information is available. Controllableevents are indicated by graph edges with an arrowhead,while uncontrollable events are indicated by edges with asmall circle at the end (e.g. the transition from state q1to q6). All states are legal, with the exception of state 10,

q0

q1

q2

q3

q4q5

q6

q7

q8 q9

q10

q11

q12

e1

e2

e3

Fig. 5. Limited lookahead tree with ESD state.

which is an ESD state (shaded grey). Taking an optimisticattitude, it is assumed that the extension of each trace byone event leads to a legal state. As a result, each subtree islegal in the graph and events fe1; e2; e3g are all legalcontroller actions. Taking a conservative attitude, theassumption is that all of the pending traces will lead(uncontrollably) to an illegal state in the next event, inwhich case the subtree with the uncontrollable transitionfrom q1 to q6 must be disabled. Thus, the legal controlleractions are fe2; e3g. Finally, adding the requirement that thesystem must be ESD-state coreachable, leads to the furthertrimming of the graph as the otherwise controllable andsafe subtree of e2 is no longer valid since it is uncertainwhether an ESD state can be reached. Thus, the legalremaining controller action is fe3g.

The additional requirement of ESD state coreachabilityresults in the a more restrictive control than that ofprevious LLP literature. The cost of ensuring nonblockingsafe operation within an arbitrary lookahead horizon is amore conservative controller (Millan & O’Young, 2006).

3.2. Online computation

The term online control means that the controller issynthesized during controller execution, implying a real-time running environment. The controller map must beextended to match the moving system state by advancingthe lookahead horizon incrementally. Online controlallows time-varying plant models or specifications to bedealt with. The computational complexity of the controllersynthesis can be reduced using a variety of approaches. Forexample, Vahidi, Fabian, and Lennartson (2006) provide avariety of techniques, including partial formation of theproduct state space PkS.

3.3. Controller propagation

An HTG is a representation of the timed DE behaviourof an SCM at a particular time and state. Furthermore, itmodels this behaviour for a particular prediction horizon.No attempt has been made to produce a closed-form finitestate representation of the DE behaviour of the modelledsystem for all time. Thus, the controller HTG should beconsidered to be a temporary data structure that will beused to choose the control action at a particular point intime and space. After the information contained in aparticular controller HTG is no longer useful, a new onemust be created from the basic SCM information—the‘‘source’’ model. The new model is now relevant for thecurrent state and time of the system. This processcontinues, extending the controller forward in time. If ateach control update step, there is a suitable control choice,then the control system effectively guides the system along,constructing the safe trajectory one event (control decision)at a time, based on a tree of predicted future behaviour.

ARTICLE IN PRESS

108

109


3.4. Control choice

The controller as described so far does not implementcontrol in itself; it actually models the future safe DEbehaviour of the plant, including only the unambiguouslysafe, nonblocking and ESD reachable control trajectories.This implies that there is still a choice to be made, sincethere may be more than one safe and nonblockingtrajectory available to the controller. To complete the fullclosed-loop control implementation (Fig. 6), some sort ofchoice mechanism must be added to the controller. Clearly,it is preferable for the system to continue running safelywithout shutting down, so some priority should be placedon selecting traces that are both nonblocking and ESDreachable (priority 1 choice) over those that are only ESDreachable (priority 2 choice). Once controller actions havebeen sorted according to this priority, a choice may stillremain.

Let C be a controller connected to a hybrid plant P. Ateach hybrid state h, a controller graph is produced, aprediction of the controlled plant behaviour on an LLhorizon. The graph is computed according to the require-ments of nonblocking (legal) behaviour and ESD reach-ability. The changes in the plant state are signalled by DEsoccurring in the plant and by the universal tick event; theseare the choice points that have been encapsulated in theSC modelling framework. At each choice point, there is aset of legal control choices that may be made, the actuatorevent set:

Definition 7 (Actuator event set). The actuator event setA � Sin is the set of eligible input events that are legal.

Definition 8 (Choice mechanism). The choice mechanism‘‘chooses’’ a single actuator event sa from set A. The choicemechanism is modelled as the function M

M : A! A.

There are a variety of possible strategies for choosing thenext control action; this paper demonstrates a simplerandom choice and a heuristic choice mechanism in thesupervisory control example of Section 4.

P

C M

�ah

A

Fig. 6. The closed-loop controller with a control choice mechanism.

3.5. Complexity with control

The computational cost of computing a controller isparticularly important for an online implementation. Thecomplexity is primarily governed by the lookahead horizonand the number of branches at each choice point (seeEq. (2)). The number of possible branches that can bemade at any choice point is dictated by the number ofavailable dynamics r ¼ jGj, and by the number of thesechoices that are disabled by the specification. Thisdisablement is unpredictable and is determined by how‘‘tight’’ the specification is. Any other synchronous modelconnected to the plant may also constrain the plantbranching behaviour, even though it may not necessarilybe considered as part of the specification.A tight specification helps to reduce computational

complexity, but the implication is that a larger set of plantbehaviours will be disabled. This effectively reduces theavailable control choices, increasing the likelihood ofblocking, and an unwarranted ESD. It is not possible toanalytically predict the exact complexity of controllersynthesis for a particular system model, but it is possibleto evaluate it empirically. A controller was designed for thetank of Section 2.4. In Fig. 7, the upper trace is thetheoretical size of the unconstrained plant graph, measuredin transitions, as a function of lookahead horizon in events.The lower trace is the computed controller size based on aspecification that enforces alternated filling and draining ofthe tank. Clearly, the controller graph (the plant and thespecification) is significantly smaller than the uncon-strained plant graph. Computationally, it is possible totake advantage of this size reduction to also reduce thecomputational burden when computing the controlleronline.An experimental software toolbox called HYSYNTH was

written to implement the control theory laid out in thispaper. Designed to operate in the MATLABs environ-ment, it uses a high-level command interface that enablesthe user to develop hybrid models using SCMs and FSMs.

4 6 8 10 12 14 16 18101

102

103

104

105

106

107

Tra

nsiti

ons

Lookahead Horizon (events)

Plant

Controller

Fig. 7. Comparison of complexity results for a tank level controller.


The designer can then develop FSM specifications anddesign online controllers using the techniques described inthis paper. Complex nonlinear MATLABs models in theform of an ordinary differential equation can be encapsu-lated within the SCM object, making it relatively seamlessto transfer to HYSYNTH if these models already exist. It isbeyond the scope of this paper to cover the details of thealgorithmic implementation of HYSYNTH, but the interestedreader is referred to Millan (2006) for further details.

4. Supervisory control of a dynamic positioning vessel

This section introduces a control application thatrequires more complex embedded continuous dynamicsthan the previous examples. This application also is used todemonstrate a controller with a human choice mechanism:an example of human-in-the-loop (HIL) control.

Dynamic positioning (DP) is defined in the marineengineering community as the automatic control of avessel’s position and heading using its thrusters. The DPcontrol system may also be used in combination with thevessel’s rudders, and passive restraints such as a mooringsystem. Typically, DP systems are installed on vessels thatneed to automatically maintain station for long periods oftime in a variety of weather conditions. For a general,nonmathematical treatment of the subject, the reader isreferred to Morgan (1978) and Hancox (2001). For amathematically rigorous version, the reader is referred toFossen (1994). Almost all theoretical study of DP controlhas been devoted to various types of continuous controlstrategies, particularly optimal control. These techniqueshave been refined and in practical use for many years. As inmany other industrial applications, the challenge for the‘‘next generation’’ of ship control systems is the integrationof DP control with other shipboard control functions, suchas power management, which require logic and appropriatesequencing (Hals, 2004; Millan, Smith, & O’Young, 2002;Weingarth, 2002). Currently, such functions are served byhighly skilled operators.

In many areas of the world, the oil from subsea oil fieldsis pumped and stored by a specialized vessel known as afloating production storage and offloading vessel (FPSO).The FPSO is usually moored over a manifold on the seafloor from which the oil is pumped. Risers (large flexibletransfer hoses) carry the oil from the subsea manifold to

Fig. 8. View of an FPSO (left) ve

the FPSO, often entering through a swiveling manifoldsystem on the underside of the vessel. Finally, the oil istransferred at sea from the FPSO to a shuttle tanker, whichtakes up station in tandem, at the FPSO stern (see Fig. 8).The task of oil transfer at sea is complex and dangerousdue to the close proximity of the two vessels. There is a riskof collision if they get too close to each other, or of transferhose breakage and an oil spill if they drift too far apart.The shuttle tanker may use some sort of passive restraintsystem (a rope called a hawser line) as a backup, but notension is applied to it. Thus, the shuttle tanker mustmaintain station behind the FPSO using only its propul-sion system which is controlled by the DP controller.Occasionally, the FPSO may have to turn in order torealign itself if the prevailing environmental conditionschange direction. Since the FPSO rotates about theswiveling manifold on the hull, the shuttle tanker mustalso swing, but through a greater arc. This is known as aweathervaning manoeuvre, and requires coordination andcare by the operators of both vessels. The most importantfactor influencing the ability of these vessels to carry outtheir operations is the power system.The shuttle tanker operation is modelled with an SCM

and a control system is synthesized to supervise the safetyof a weathervaning manoeuvre. The SCM will be designedaround a detailed simulation of the vessel, including thepower system and thruster (actuator) models.

4.1. Vessel power system

On most modern vessels, the propulsion system ispowered electrically, and a complete power generationand distribution system is required on board. As a result,the performance of the power system directly affects thepropulsion, and thus the ability of the DP system tomaintain station. For this reason, the DP control system isoften integrated with the power generation system so thatthese systems may be coordinated. For the sake of thisexample, the power generation system is assumed to havetwo main generators (MG1 and MG2) and a propulsionsystem with four steerable propulsion units, T1� T4(called azimuthing thrusters). The azimuthing thrusterunits are designed so that they can be turned to directthrust in the appropriate direction relative to the vessel. InFig. 9 the electrical schematic for the power distribution

ssel offloading oil to a tanker.

ARTICLE IN PRESS

MG1 MG2

TR1 TR2

SCR1 SCR2 SCR3 SCR4

T1 T2 T3 T4

15 MW 15 MW

4 X 7.5 MW

MG3

Other

Loads

5 MW

TR3

S1 S2

S3

Fig. 9. Power management system for a typical vessel.


and propulsion systems is depicted for a hypothetical DPvessel. Normally, the two main generators with ratedcapacity of 15 megawatt (MW) supply the main propulsionbus via the transformers TR1 and TR2. Switchgearat S1 and S2 enable the generators to be taken off line.A backup generator, designed for so-called ‘‘hotel’’ load(i.e. lighting, domestic loads) can be placed on thepropulsion bus in event of emergency via switch S3. Theazimuthing thrusters, are supplied by thyristor drivesSCR 1–4, which control the propeller speed. Havingazimuthing capability, the thrusters can be rotatedcontinuously through 360 to direct their thrust in themost appropriate direction. For this example, it will beassumed that the main generators are running and thatswitching of generators onto the bus is instantaneous.While it is possible to model the continuous dynamics ofthe switching, they will be modelled as DE dynamics forthis paper.

4.2. Vessel manoeuvring model

For purposes of manoeuvring control, the vessel modelwill be limited to 3 degrees of freedom (DOF), yaw angle,surge displacement and sway displacement. Roll and pitchangles and heave (vertical) displacement cannot becontrolled and so are unnecessary to model. The rotationof the vessel (about its centre of gravity) within the plane isNorth-referenced and is called heading, denoted ce. Therotational component in the body reference frame cb, is thesame as heading, but to distinguish it from the absolutecoordinate, it is called yaw. In general, the subscript e isused to denote earth-referenced (inertial) coordinates and

the body referenced coordinate frame is denoted bysubscript b. Let the vector xe ¼ ½xe; ye;ce�

T represent theearth-referenced 3 DOF position vector of the vessel andx ¼ ½xb; yb;cb�

T denote the position 3 DOF vector in thebody frame. Since heading angle and yaw are equivalent, cis used as default rotation about the center of gravity(CG) of the vessel. The coordinate transformation JðcÞtakes the earth-referenced measurements into the bodyframe of reference:

x ¼ JðcÞxe,

xb

yb

c

264

375 ¼

cosðcÞ sinðcÞ 0

sinðcÞ � cosðcÞ 0

0 0 1

264

375

xe

ye

c

264

375.

The velocity and acceleration vectors are defined accord-ingly as v ¼ _x ¼ ½u; v;f�T and _v ¼ ½ _u; _v; _f�T. The simplified(linear) dynamics of a freely floating (i.e. unmoored)surface vessel can then be characterized by the followingvector differential equation:

M _vþDv ¼ s.

s is the force and moment vector acting upon the vesselthat arises from the sum of the control forces, sc and theenvironmental forces (current, waves and wind), se, eachdefined in the inertial frame

s ¼ sc þ se.

M is a positive definite matrix ðM ¼MTÞ containing theinertial and hydrodynamic added mass terms for the vessel

ARTICLE IN PRESS

Table 4

Nondimensional scaling factors

Quantity Scale factor

Mass rrLength L

Linear velocityffiffiffiffiffiffigLp

Angular velocityffiffiffiffig

L

rForce rgr

Moment rgrL

TimeffiffiffiffiL

g

r


as follows:

M ¼

mþ X _u 0 0

0 mþ Y _v mxG þ Y _r

0 mxG þ Y _r Izz þN _r

264

375,

where m is the vessel’s mass, Izz is the yaw moment ofinertia, X _u, Y _v, are the hydrodynamic added mass in thesurge axis, sway axes, respectively, and N _r is the addedmoment of inertia in the yaw axis. The off-diagonal termsare symmetrical (this follows, since the vessel is symme-trical about both the surge and sway axes), and feature ahydrodynamic added mass term Y _r due to the cross-coupling between the sway and yaw axes. The longitudinaldistance between the center of mass (CG) of the vessel andthe controlled point (CP) of the ship is xG.

D is a matrix containing linear hydrodynamic dampingterms:

D ¼

X u 0 0

0 Y v Y r

0 Nv Nr

264

375

the diagonal terms X u, Y v and Nr, are the surge sway andyaw damping. The off-diagonal terms Y r and Nv are,respectively, the sway–yaw and yaw–sway damping terms.

Assumptions that have been made to simplify this modelare that centripetal and Coriolis forces are negligiblebecause yaw rates are relatively small, and hydrodynamicadded mass and damping are constant. Since most DPvessels are designed for stationkeeping, and vessel velocitiesare low, this assumption is also reasonable. The hydro-dynamic added mass and damping can be determined byspecialized software, estimated from existing ships forwhich these parameters are already known, or determinedempirically by model testing or full-scale ship trials. Thedetailed vessel particulars for the tanker that is to besimulated are given in Table 3.

For the simulation, nondimensional quantities will beadopted for convenience. The so called bis system (Fossen,1994), is a convenient system for low-speed manoeuvringmodels, since it is not based on vessel forward speed asother systems are. The bis system nondimensional scalingfactors for a surface vessel are given in Table 4. Typically,the vessel LOA is used for the length factor L, g is theacceleration due to gravity and r is the density of sea water

Table 3

The tanker particulars

Vessel particular Full scale

Length overall (LOA) 290m

Displacement, r 193,000m3

Mass 197,632 tonnes

Yaw radius of gyration 57m

Beam 45m

Longitudinal CG 145m

ð1025 kg=m3Þ. As an example, the nondimensional mass ofthe vessel at full displacement is m ¼ 1.

4.3. Closed loop control

For this example, the DE controller will supervise aclosed-loop continuous controller (i.e. the DP controlsystem). Therefore, the SCM will be developed aroundCSMs that model the closed-loop dynamics of the vessel. Itshould be noted that the control gains for this embeddedcontroller are arrived at in a conventional controllerdesign process with optimality defined by H2 or H1

norms (for example). This controller design is an offlineprocess which is not addressed by this paper.Fig. 10 is a block diagram of a typical DP control

system. The system is commanded with a 3 DOF setpointcommand in earth-referenced coordinates. The vessel’s 3DOF position xe is measured with a variety of sensors andpassed through a state estimator. The error signal isconverted to body coordinates, and control gains areapplied to determine a controller demand. Measurementsof the wind speed and direction are used to calculate afeedforward wind load, which is summed to the controllerdemand. A thruster allocation block determines how thiscontroller sc demand will be divided amongst the availablethrusters, taking the geometry of their hull arrangementinto account. Not pictured in the figure is the optimal stateestimation of current and wave generated forces andmoment; these are summed into the controller demand.

4.4. Thruster allocation

In Fig. 11 the vessel thruster arrangement is pictured.The relation between the control demand and theindividual actuator demands is as follows:

sc ¼ TaTth,

where Tth is a vector of thruster demands in Cartesiancoordinates, and Ta is the thruster allocation matrix,defined as follows:

Tth ¼ ½T1x T1y . . . T4x T4y�T

ARTICLE IN PRESS

Environment

WindSpeed

Windfeedforward

DP Control System

ThrusterAllocation

++

++

+

OptimalGain

Referenceor setpoint

-

ErrorState

Estimator

PositionHeading feedback

AppliedThruster

Force Demands

EnvironmentForce

Tanker

Fig. 10. Block diagram of DP control system.

T2

T1

T3

l2y

l3x

l4x

α1 T1

T1x

T1y

CG

l1x, l2x

Fig. 11. Schematic of thruster arrangement of tanker vessel.

Table 5

Example vessel thrust limits as a function of power system configuration

Electrical configuration Total thrust Thruster saturation

One main generator, 15MW 3MN 750 kN

Both main generators, 30MW 6MN 1.5MN

Standby generator, 5MW 1MN 250 kN


and

Ta ¼

1 0 1 0 1 0 1 0

0 1 0 1 0 1 0 1

l1y l1x l2y l2x l3y l3x l4y l4x

264

375. (4)

In Eq. (4), matrix entries of 1 indicate that 100% isavailable from the thruster if it is rotated to the appropriatedirection. The bottom row are the lever arm distances thatgenerate moment about the CG. Solving for the unknownTth requires finding the Moore–Penrose generalized inverseof Ta

Tth ¼ Tyasc,

where Tya is the generalized inverse of Ta. Thrust vector Tth

can be converted from Cartesian coordinates to an azimuthangle command and thrust demand pair a T½ �T for eachthruster as follows:

Tth ¼ ½a1 T1 . . . a4 T4�T.

The thrusts are minimal in a least-squares sense, but mayexceed the thrust limit for the actuator. In a real thruster,the maximum thrust is dependent on many factors,including the speed of the thruster through the water andthe proximity and wake direction of other thrusters. In thissimulation, the thrusts T1, T2, T3, T4 will simply beclamped at a saturation limit which will be determined bythe electrical bus power available. The relation betweenthrust and electrical power is summarized in Table 5.The various modelling details given in these sections

provide for a reasonable ship simulation model. In thenext section a DES supervisor for the tanker will bedeveloped.

ARTICLE IN PRESS

Table 6

Output events, with associated functionals and hypersurface crossing

directions for the DP vessel control synthesis problem

sout Functional Zero-crossing Alarm

tcl F1ðxÞ ¼ r� 1:25 # Too close to FPSO

tfb F2ðxÞ ¼ r� 1:45 " Too far from FPSO

o3 F3ðxÞ ¼ yþ 1:4 " Riser area guard

o4 F4ðxÞ ¼ yþ 1:9 # Enter flare safe area

o5 F4ðxÞ ¼ yþ 2:1 # cw exit flare safety area

o6 F4ðxÞ ¼ yþ 1:8 " ccw exit flare safety area

tfp F7ðxÞ ¼ p� ðc� yÞ � 0:2 " Misalignment to port

tfs F8ðxÞ ¼ p� ðc� yÞ þ 0:2 # Misalignment to starboard

esd F9ðxÞ ¼ r� 1:85 " Emergency shutdown

tick F10ðxÞ ¼ sinð2pt=DtÞ # Controller update


4.5. Supervisory controller design

The coordinate frame and general arrangement of thevessels and safe operating areas are detailed in Fig. 12.Since the FPSO is attached to a mooring and rotates aboutthis point, it is convenient for the supervisory controller tocommand the tanker and FPSO in a rotating coordinateframe. The earth-referenced Cartesian coordinate frame isredefined to a polar coordinate frame centered at the FPSOmooring point where r is the radial distance of a vessel(CG) from the origin, and �PpypP is the angle ofrotation, while c is the vessel heading, unchanged from theother coordinate frames. All ranges r in this diagram arenondimensional. The scenario that the controller isdesigned for is a weathervaning manoeuvre that only thetanker carries out. When the flare is lit on the FPSO, theforward deck temperature of the tanker can rise todangerous levels. The written operating procedures forthis vessel require that the operator of the FPSO contactthe tanker and request that it moves in order to minimizethe deck heating. Since the flare is on the starboard side ofthe FPSO, movement of the tanker slightly to the port sideof the FPSO’s stern has the desired effect. During this

r

rcl = 1.25rnom = 1.35

rfb = 1.45

rsd = 1.85

FlareHose &Hawser

MooredPoint

BlueZone

GreenZone Red

Zone

Tanker

ESD Zone

PrevailingEnvironmental

Forces

�1 = −1.4�2 = −1.8�3 = −1.9

�4 = −2.1

�

FPSO

Fig. 12. Overall schematic of vessel positions in polar coordinate frame.

move, the appropriate separation between the vessels muststill be maintained to prevent collision or hose breakage. InFig. 12 the ‘‘green zone’’ is the normal safe area in absenceof a flare. The ‘‘red zone’’ to the starboard of the tanker is akeep-out area due to subsea risers that may be damaged bythe tanker’s thrusters. The ‘‘blue zone’’ area to the port sideof the FPSO is the safe area while the flare is operating.A controller will be designed to enforce safe operationduring a flare event.

4.5.1. Partitions and output events

The modelling process begins by defining the partition-ing functionals for the system SCM in Table 6. In this case,a partial state vector x ¼ ½r; y;c; t�T is used. Note that thepolar coordinate frame with origin at the mooring point ofthe FPSO has been used. Events tfp and tfs signal when thevessel longitudinal axis is out of alignment with thecoordinate frame; the goal is to keep the bow of the tankeraimed towards the FPSO at all times.1 The esd (emergencyshutdown) is assumed to be achieved once the vessel hassafely reached a radial distance rsdX1:85 (the ESD Zone).

4.5.2. Controller actions

The control actions available to the controller for thismodel are listed in Table 7 and are associated with thecorresponding input event labels sin 2 Sin. The controls arethe commands that will be sent to the DP control system.The controls rj, yj and cj are ‘‘jog’’ commands which aresummed with the current state of the system to develop anabsolute setpoint for the DP controller.

rsp ¼ rþ rjog,

ysp ¼ yþ yjog,

csp ¼ cþ cjog.

The control indicated by g ¼ ½S1;S2;S3� 2 f0; 1g is a vectorcorresponding to the generator switchgear of Fig. 9. Withinthe controller simulation, the effect of this switchgear

1Alternatively, the origin of the polar coordinate frame could be placed

at the stern of the FPSO.

ARTICLE IN PRESS

Table 7

Control actions available to the DES supervisor. Controls are specified as

setpoint jog commands to the DP controller, and are in nondimensional

units and the FPSO polar coordinate reference system

sin rjog yjog cjog g Description

a1þ 0 0:1 0 [1 0 0] jog cw, one generator

a1� 0 �0.1 1 [1 0 0] jog ccw, one generator

a2þ 0 0.15 0 [1 1 0] jog cw, two generators

a2� 0 �0.15 1 [1 1 0] jog ccw, two generators

fwd 1 �0.1 0 [1 0 0] Ahead, one generator

back 1 0.1 1 [1 0 0] Astern, one generator

hold 0 0 0 [1 0 0] Hold station, one generator

sd 1.85a b b [0 0 1] Shutdown on emergency power

aIn absolute coordinates.bIndicates a don’t care input.

ticktick

tick

tick

tick

tick

tick

q0

q1

q2

q3

q4

q5 q6

qfs

o4o4

o4

o4

o4

o4

o4

Fig. 13. A specification for the flare weathervaning move.


control input is that it sets the saturation limit of thethrusters as per Table 5.

4.5.3. Modelled environmental load

In heavy environmental loading, it is necessary to align avessel with the prevailing direction of this load in order tominimize the thruster effort required to stay on station, andin the case of the moored vessel, it reduces the vesselmotion. For this scenario, it is assumed that the FPSO isaligned with this environmental load. When the tankermoves around to avoid the flare, it encounters a load thatprogressively increases in proportion to the misalignmentof the vessel with the load vector. As the tanker movesfrom out of the ‘‘shadow’’ of the FPSO and its beam isexposed to the waves and wind, this will tend to drag thevessel off station. It is also assumed that this load is amodelled effect. Thus, this force is predictable and it can beembedded in the CSMs.

4.5.4. Discrete event specification

In this example, the vessel will be directed to rotate fromits initial position, with a heading of c ¼ p=2 andpositioned directly behind the FPSO in the green zone(Fig. 12). The assumption is that the FPSO has commu-nicated to the tanker that it will commence flaring gas sothe tanker must move to the flare safe area. Typically anoffshore marine operation like this has a set of writtenprocedures for the vessel operators to follow; theseprocedures contain detailed written descriptions of variousactivities that involve both vessels, and contained withinthis manual is a specification of the maximum flare dwelltime and the safe area for the tanker. Essentially thedescriptive procedure is encoded as a specification; inFig. 13 it is represented by a finite state machine. Thespecification requires that the vessel move to the flare safearea (an o4 event) within six ticks. This upper bound on thespecification time is derived not from the vessel dynamics,but is based on the maximum time the vessel can linger inthe green zone while the flare is lit, which is derived directlyfrom the operational procedures manual. This will be thespecification used for the controller synthesis and simula-

tion. Adding the following events to the event set of thespecification,

S ¼ fo4g [ ftcl; tfb; o3; o5; o6; tfs; tfpg

effectively prohibits them from occurring.

4.6. Results

The modelling information of the preceding sections wasused to develop an SCM. A software package that wasdeveloped to compute these DE supervisors calledHYSYNTH was used to simulate the closed-loop system,and test results were obtained for supervisory control of theflare event weathervaning manoeuvre. The tick time usedfor these simulations was Dt ¼ 100 (nondimensional).

4.6.1. Control with random choice

Running a simulation of the weathervaning manoeuvreusing the specifications of Fig. 13 will generally result in ashutdown if the event or time lookahead horizon is notlarge enough to include state q6 of the specification. Thecontrol synthesis has to have a sufficiently large enoughlookahead to perceive that state q6 is blocking if the vesselcannot subsequently reach the flare safe area (state qfs).With an event or time lookahead that is too short, thecontrol system is not able to distinguish between an outputstring of five ticks that is moving to safety from an outputstring that is simply staying within the green zone. This isclearly illustrated in Table 8, in which the controller has a


five event lookahead combined with a random controlchoice mechanism. The table is read from left to right: e.g.at step 1, there are a total of jAj ¼ jAej þ jAenj ¼ 6 controlactions available for the operator to choose amongst. Thesecond and third columns are the legal input event subsets,that are eligible as control actions: recall that Ae is the setof control actions that lead only to ESD and Aen is the setof events that lead to ESD reachable and nonblockingstates.

Starting at step 1, the controller has five priority 1choices (ESD reachable and nonblocking), so using arandom choice mechanism, it actually selects aþ2 which isdriving with both generators in the wrong direction. Itcontinues to do this action in the next step, when suddenlythe specification has reached a block in the lookahead; i.e.o4 can no longer be synchronized. The available controlactions all lead to shutdown, Aen ¼ ;. Fortunately, byconstructing the controller as ESD reachable, at least thesystem will be able to shutdown gracefully. The remainingcontrol actions continue to select at random from set Ae

and the system shuts down at step 7. This simulationis pictured in Fig. 14, in which the top trace is a plot of

Table 8

A summary of the vessel controller simulation, see Fig. 14

Step Ae Aen Sizea sin sout

1 fsddg fa�1 ; aþ1 ; a�2 ; aþ2 ; holdg 4503 aþ2 tick

2 fsddg fa�1 ; aþ1 ; a�2 ; aþ2 ; holdg 4378 aþ2 tick

3 fsdd; a�1 ; aþ1 ; a�2 ; aþ2 ; holdg ; 1319 aþ2 tick

4 fsdd; a�1 ; aþ1 ; a�2 ; holdg ; 193 a�2 tick

5 fsdd; a�1 ; aþ1 ; a�2 ; aþ2 ; holdg ; 59 aþ2 tick

6 fsdd; a�1 ; aþ1 ; a�2 ; holdg ; 9 a�2 tick

7 fsddg ; 1 sdd esd

aIn graph transitions.

0 100 200 300 400

1.4

1.6

1.8

2

Ves

sel P

ositi

on

α2+ tick

α2+

tick

α2+

tickα2

- tickα2

+

0 100 200 300 400-5

0

5

T

Ves

sel V

eloc

ity 1

0-3

Fig. 14. Simulation of control with inadequate event horizon lookahea

the earth-referenced position vector xe versus time(all quantities are nondimensional). The controller actionsand the resulting output events have also been placed onthe plot. The upper line of text are the output events, whilethe lower line is the string of controller actions or inputevents. In Fig. 14 the lower trace is a plot of the vesselvelocity as a function of time. A plan view of the tanker’smovements of Fig. 15 aids interpretation of the vesselactions.

4.6.2. Human-in-the-loop-control

The fundamental premise of HIL control is that thehuman, the operator, has some sort of system knowledgewhich enables him/her to make an ‘‘informed’’ decision.The strength of the control approach in this paper is thatthe controller can assist an operator by removing from theentire set of available control choices the ones that leadunambiguously to unsafe states. Presumably the controllercan also consider many more system trajectories than theoperator can in the same amount of time. Yet, without auseful choice mechanism as in the previous section, thecontroller is unable to drive the system to the desiredobjective unless it is very carefully specified, and/or theevent or time lookahead is large enough to encompassenough of the specification to decide unambiguouslybetween blocking or safe outcomes. Fig. 16 presents ablock diagram of the HIL supervisory control hierarchythat is implemented in this section.In this example, the specification of Fig. 13 was used

again for control synthesis, but this time with a humanoperator as the selection mechanism. With an eventlookahead of four events, this simulation demonstratesthat some operator knowledge combined with the DEScontroller easily outperforms the automated randomchoice mechanism. The simulation was produced by

500 600 700 800 900

tickα2

-ticksdd

esd

500 600 700 800 900ime

re

−θe

ψe

uv

φ

d. Using a random choice mechanism, the system invokes an ESD.

ARTICLE IN PRESS

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-2.5

-2

-1.5

-1

-0.5

0

0.5

Ye

disp

lace

men

t (m

/m)

FPSO

Tanker

Xe displacement (m/m)

Fig. 15. An overhead view of the shutdown.

Vessel

DP Control&

Power Management

Supervision

DES Conroller

Human Choice

Fig. 16. A block diagram of the HIL control arrangement for this

example.

Table 9

A summary of the HIL controller simulation, see Fig. 17

Step Ae Aen Sizea sin sout

1 fsddg fa�1 ; aþ1 ; a�2 ; aþ2 ; holdg 4539 a�1 tick

2 fsddg fa�1 ; aþ1 ; a�2 ; aþ2 ; holdg 4486 a�1 tick

3 fsdd; aþ1 ; aþ2 g fa�1 ; a

�2 ; holdg 1507 a�1 tick

4 fsdd; a�1 ; aþ1 ; aþ2 ; holdg fa�2 g 304 a�2 tick

5 fsdd; a�1 ; aþ1 ; aþ2 ; holdg fa�2 g 66 a�2 tick

6 fsdd; a�1 ; holdg fa�2 g 42 a�2 tick

7 fsddg fa�2 g 130 a�2 o48 fsddg fa�2 g 444 a�2 tick

9 fsddg fa�1 ; aþ1 ; a�2 ; aþ2 ; holdg 1448 hold tick

10 ... ..

. ... ..

. ...

aGraph size in transitions.


constructing the ESD reachable controller and giving theoperator an opportunity to select the control action at eachcontroller update. The system is advanced through asimulation to the next time step and then the processrepeats itself by computing the controller for the new timestep, and so on. The result of a HIL control simulation issummarized in Table 9. A knowledgeable operator knowsthat the vessel must move to port to reach its destination(the flare safe area), this rules out the actions that will carrythe vessel away from the target, aþ1 , a

þ2 , and the action that

does nothing, hold. The event a�1 is selected by the operatorbecause it saves fuel by using only one generator. This samedecision approach continues to be exercised for the next

two steps. At step 4 though, the operator is suddenlypresented with the fact that if the system is to continue, thesecond generator must be switched onto the propulsionbus, and so a�2 must be selected. For the next five steps, thea�2 event is the only choice that permits the vessel tocontinue to operate. After the control action of step 7, thevessel crosses into the flare safe area, signified by the o4

event. Once within the area, the choice of control actionsreturns, and the size of the controller grows as more legalmoves are available again. For the remainder of this run(15 controller updates), the operator continues to selectcontrol actions that are in the Aen column and that arereasonable for stationkeeping, i.e. moves requiring onlyone generator.A picture of what happened during this simulation is

given by the time series of the position and velocitypictured in Fig. 17. A plan view of the two vessels duringthe manoeuvre is given by Fig. 18.

5. Conclusions

In this paper a switched continuous modelling frame-work was described that allows generalized nonlinearcontinuous models to be included seamlessly in a discreteevent supervisory control synthesis process. Control iseffected by switching between multiple continuous modelsthat may represent either differing operating modes of asystem, control inputs or differing systems. The controlleris synthesized online in response to events that may occureither due to continuous time or state. At each step thecontroller is a unique set of optimally controlled sub-treessuch that each sub-tree can be safely extended to the nextlook-ahead step without blocking. The nonblockingproperty relies on additional state information of the plantand the coreachability of emergency shutdown states. Thesystem is operated on the basis that if there are no otherviable choices in the future horizon, the system can at leastbe safely shutdown (by moving to its pre-defined ‘‘safecondition’’) within the limited look-ahead (LL) horizon.A detailed industrial example was provided to demonstratethis control technique. In addition, this application

ARTICLE IN PRESS

0 200 400 600 800 1,000 1,200 1,400

1.4

1.6

1.8

2

Ves

sel P

ositi

on

α-tickα-

tickα-

tickα2

-tickα2

-tickα2

-tick o4α2

- α2-tickhold

tickα-

tickα2

+tickhold

tickα1

-tickhold

tickα1

tick

0 200 400 600 800 1,000 1,200 1,400-4

-2

0

2

4

Time

Ves

sel V

eloc

ity 1

03

re −θeψe

uvφ

1 1 1 1-

Fig. 17. The HIL control simulation results showing a time series plot of the vessel position vector (top trace) and the vessel velocities (lower trace).

-1.5 -1 -0.5 0 0.5 1-2.5

-2

-1.5

-1

-0.5

0

0.5

Ye

disp

lace

men

t (m

/m)

FPSO

Tanker

Xe displacement (m/m)

Fig. 18. An overhead view of the tanker movement during the successful

weathervaning manoeuvre of the HIL controller.


demonstrates the use of human-in-the-loop control. Thisapproach combines a very simple heuristic in the controlchoice mechanism with the ‘‘brute force’’ of the exhaustivestate-space search to yield better nonblocking controllerbehaviour despite the LL horizon. Such an approachpoints to the possibility of utilizing this hybrid systemmodelling approach to practical control engineeringproblems.

Future work will be directed to extending this applica-tion from a simulation to a model-scale test for the ship

control problem. The control technique will be appliedto other applications, particularly to see-and-avoidsystems for manned and unmanned aerial vehicles. Withregard to the control technique, three areas of particularinterest are to: (1) study the choice mechanism and itsrelationship to the DES supervisor; (2) investigateother techniques for choosing lookahead horizon, thenumbers of discrete inputs and partition density; (3)explore techniques for including uncertainty and unmo-delled disturbance.

Acknowledgements

The authors wish to thank the anonymous reviewers fortheir careful consideration and helpful suggestions.

References

Abdelwahed, S., Su, R., & Neema, S. (August 2005). A feasible lookahead

control for systems with finite control set. In Proceedings of the 2005

IEEE conference on control applications (pp. 663–668). IEEE.

Balluchi, A., Natale, F. D., Sangiovanni-Vincentelli, A. L., & van

Schuppen, J. H. (March 2004). Synthesis for idle speed control of an

automotive engine. In R. Alur & G.J. Pappas (Eds.), Proceedings of the

7th international workshop on hybrid systems: computation and control

(HSCC04) (Vol. 2993) (pp. 80–94), Lecture notes in computer

science. Berlin: Springer.

Bayen, A., & Tomlin, C. (June 2003). Real time discrete control law

synthesis for hybrid systems using MILP: Application to congested

airspace. In Proceedings of the 2003 American control conference

(pp. 4620–4626).

Chung, S., Lafortune, S., & Lin, F. (1992). Limited lookahead policies in

supervisory control of discrete event systems. IEEE Transactions on

Automatic Control, 37(12), 1921–1935.

Chung, S., Lafortune, S., & Lin, F. (1994). Supervisory control using

variable lookahead policies. Discrete Event Dynamic System: Theory

and Applications, 4(3), 237–268.

ARTICLE IN PRESSJ.P. Millan, S.D. O’Young / Control Engineering Practice 16 (2008) 1004–1021 1021

Chutinan, A., & Krogh, B. H. (2003). Computational techniques for

hybrid system verification. IEEE Transactions on Automatic Control,

48(1), 64–75.

Fossen, T. (1994). Guidance and control of ocean vehicles. New York:

Wiley.

Hadj-Alouane, N., Lafortune, S., & Lin, F. (1994). Variable lookahead

supervisory control with state information. IEEE Transactions on

Automatic Control, 39(12), 2398–2410.

Hals, T. (September 2004). Tandem loading and drilling operations under

changing environmental conditions. In Proceedings of the 2004 MTS

dynamic positioning conference. Marine Technology Society.

Hancox, M. (2001). Towing, positioning and hook-up for offshore

production, Vol. 8. Oilfield Publications Limited.

Henzinger, T. A., Ho, P., & Wong-Toi, H. (1997). HyTech: A model

checker for hybrid systems. Software Tools for Technology Transfer, 1,

110–122.

Koutsoukos, X., Antsaklis, P., Stiver, J., & Lemmon, M. (2000).

Supervisory control of hybrid systems. Proceedings of the IEEE

1026–1048.

Kumar, R., Chung, H. M., &Marcus, S. I. (1998). Extension based limited

lookahead supervision of discrete event systems. Automatica, 34(11),

1327–1344.

Lin, H., & Antsaklis, P. (June 2005). Stability and stabilizability of

switched linear systems: A short survey of recent results. In

Proceedings of the 2005 IEEE international symposium on intelligent

control (pp. 24–29).

Millan, J., & O’Young, S. (June 2006). Hybrid system control using an

online discrete event supervisory strategy. In IFAC conference on

analysis and design of hybrid systems. IFAC.

Millan, J., Smith, L., & O’Young, S. (September 2002). Coordination of

FPSO and tanker offloading operations. In Proceedings of the MTS

dynamic positioning conference 2002.

Millan, J. P. (October 2006). Online discrete event control of hybrid

systems. Ph.D. thesis, Memorial University of Newfoundland.

Millan, J. P., & O’Young, S. D. (July 2006). On-line supervisory control of

hybrid systems using embedded simulations. In Proceedings of the

IEEE 8th international workshop on discrete event systems WODES06.

Morgan, M. (1978). Dynamic positioning of offshore vessels. PPC Books,

Division of Petroleum Publishing Company.

Palm, W. J., III (2000). Modelling, analysis, and control of dynamic systems

(second ed.). New York: Wiley.

Potocnik, B., Bemporad, A., Torrisi, F., Music, G., & Zupancic, B. (2004).

Hybrid modelling and optimal control of a multiproduct batch plant.

Control Engineering Practice, 12(9), 1127–1137.

Qin, S. J., & Badgwell, T. A. (2003). A survey of industrial model predictive

control technology. Control Engineering Practice, 11(7), 733–764.

Raisch, J. (2000). Discrete abstractions—An input/output point of view

[Special issue on discrete event models of continuous systems].

Mathematical and Computer Modeling of Dynamical Systems, 6(1),

6–29.

Raisch, J., & O’Young, S. (1998). Discrete approximation and supervisory

control of continuous systems. IEEE Transactions on Automatic

Control, 43(4), 569–573.

Ramadge, P., & Wonham, W. (1987). Supervisory control of a class of

discrete event processes. SIAM Journal on Control Optimization, 25(1),

206–230.

Ramadge, P., & Wonham, W. (1989). The control of discrete event

systems. Proceedings of the IEEE, 77(1), 81–98.

Stursberg, O. (December 2004). A graph search algorithm for optimal

control of hybrid systems. In Proceedings of the 43rd IEEE conference

on decision and control (pp. 1412–1417).

Su, R., Abdelwahed, S., Karsai, G., & Biswas, G. (October 2003). Discrete

abstraction and supervisory control for switching systems. In IEEE

international conference on systems, man, and cybernetics (Vol. 1),

pp. 415–421. IEEE.

Torrisi, F., & Bemporad, A. (2004). HYSDEL—A tool for generating

computational hybrid models. IEEE Transactions on Control Systems

Technology, 12(2), 235–249.

Vahidi, A., Fabian, M., & Lennartson, B. (2006). Efficient supervisory

synthesis of large systems. IFAC Control Engineering Practice, 14(10),

1157–1167.

Weingarth, L. (February 2002). Avoiding catastrophes in dynamic

positioning; integrating key parameters using a systems approach. In

IADC SPE drilling conference.

online discrete event supervisory control of hybrid dynamical systems using embedded simulation

Documents