combinatorial algebra in controllability and optimal...
TRANSCRIPT
Combinatorial Algebra in Controllability andOptimal Control
Matthias Kawski
Abstract Control theory models, analyzes, and designs purposeful interactions withdynamical systems so that these behave in a desired way. Fundamental to the abilityto synthesize complex responses by strategically combining simple control buildingblocks is a desired lack of commutativity of the flows associated to control actions.In its most simple terms this means that the dynamics are not constrained to evolveon lower dimensional integral manifolds. Mathematically this lack of integrabilityis manifested in terms of (nonabelian) Lie algebras. In recent years it has been rec-ognized that Zinbiel algebras and combinatorial Hopf algebras play an importantunifying role for a deeper understanding of the underlying structures.This book chapter presents fundamental concepts in control theory that are inher-ently linked to combinatorial and algebraic structures. It demonstrates how moderncombinatorial algebraic tools provide both deeper insight and facilitate analysis,computations, and design. The emphasis is on exhibiting the algebraic structuresthat map combinatorial structures to geometric and dynamic objects.
Matthias KawskiSchool of Mathematical and Statistical Sciences, Arizona State University, Tempe, Arizona 85287,USA, e-mail: [email protected] work was partially supported by the National Science Foundation through the grants DMS09-08204 and DMS 09-60589.
1
Contents
Combinatorial Algebra in Controllability and Optimal Control . . . . . . . . . 1Matthias Kawski
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.1 Motivation: idealized examples . . . . . . . . . . . . . . . . . . . . . . 51.2 Controlled dynamical systems . . . . . . . . . . . . . . . . . . . . . . . 71.3 Fundamental questions in control . . . . . . . . . . . . . . . . . . . . 8
2 Analytic foundations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.1 State space models and vector fields on manifolds . . . . . . 102.2 Chronological calculus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.3 Piecewise constant controls and the BCH formula . . . . . . 142.4 Picard iteration and formal series solutions . . . . . . . . . . . . 162.5 The Chen-Fliess series and abstractions . . . . . . . . . . . . . . . 18
3 Controllability and optimality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223.1 Reachable sets and accessibility . . . . . . . . . . . . . . . . . . . . . 223.2 Small-time local controllability . . . . . . . . . . . . . . . . . . . . . . 243.3 Nilpotent approximating systems . . . . . . . . . . . . . . . . . . . . 283.4 Optimality and the Maximum Principle . . . . . . . . . . . . . . . 313.5 Control variations, and approximating cones . . . . . . . . . . . 34
4 Product expansions and realizations . . . . . . . . . . . . . . . . . . . . . . . . . . 414.1 Variation of parameters and exponential products . . . . . . . 424.2 Computations using Zinbiel products . . . . . . . . . . . . . . . . . 454.3 Exponential products and normal forms for nilpotent
systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474.4 Logarithm of the Chen series . . . . . . . . . . . . . . . . . . . . . . . . 51
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3
4 Contents
1 Introduction
Control theory models, analyzes and designs purposeful interactions with dynami-cal systems with the objective to make them behave in a desired way. In the mostsimple systems this interaction may be as simple as selecting the value of a constantcontrol parameter. In applications this may be a flow rate or temperature in a chem-ical plant, a vaccination rate in a model for spread of infectious diseases, an interestrate in a financial system, or a torque or force in a simple mechanical setting. In theclassical setting such constant control parameters may parameterize time-varyingcontrol inputs via their frequency or decay rate. More generally, modern controltheory considers classes of time-varying control inputs that need not necessarily beparameterized in simple ways by a finite number of constant parameters (or gen-erated by dynamical systems that are parameterized by a finite number of constantparameters).
Very simple systems may have for each degree of freedom a corresponding in-put channel. The first complication arises when these inputs are not decoupled, butinstead interact with other. But the main challenge is to control systems that arecharacterized by a high dimensional state space (many degrees of freedom) togetherwith a very small number of control input channels. In this case, the interactionsbetween the controls play a critical role and they may allow one to independentlycontrol a large number of states with very few controls, possibly even a single scalarcontrol. The analysis and exploitation of such interactions of the controls, with eachother and with themselves, is at the heart of modern control theory.
A decisive role is played by the lack of commutativity: in general, control ac-tions taken in different orders result in different outcomes. This demands that onedevelops and employs corresponding mathematical structures that both give insightin the underlying structures, and which allow one to efficiently perform computa-tions for analysis and design. The classical linear theory developed in the 1960s waslargely based on linear algebra and complex analysis. Since the 1970s Lie algebrashave been employed routinely for the study of nonlinear systems. More recently ithas become clear that more can be gained by utilizing further refinements. Indeed,the roots of using combinatorial and Hopf-algebraic tools in control theory can betraced back to the late 1980s, see e.g. [32, 33, 34].
The subsequent sections give a high level survey of fundamental concepts andproblems of control theory, with special focus on items that are intrinsically linkedto deeper algebraic and combinatorial structures. Starting from very conceptual de-scriptions and problems, the sections quickly progress to more abstract structures.The development is intended to be understandable to an audience that has no priortechnical background in control theory. The main contribution consists of unifyingmany findings of the last 30 years and framing them in a form that highlights theircommon algebraic and combinatorial foundations, with special focus on the map-pings from combinatorial and algebraic objects to geometric, analytic, and dynamicobjects. Throughout, whenever feasible, the notation and viewpoint of the chrono-logical calculus of Agrachev and Gamkrelidze [1, 2] are employed.
Contents 5
1.1 Motivation: idealized examples
The following examples will be revisited in the sequel. Control theory is just as ap-plicable in other frameworks, from electrical, biological, to financial systems. Theselecture notes use simple, idealized mechanical systems, as their nontrivial dynamicsare intuitive and familiar from everyday life and more easily accessible than otherapplications. They illustrate the usefulness of a systematic geometric description us-ing the advanced combinatorial and algebraic tools developed in these lecture notes.
Fig. 1 Parallel parking a car (bicycle)
The task of parallel parking a car, is familiar, but not trivial as any first-timedriver will attest, compare figure 1. Natural choices for the controls are the angularvelocity of the steering wheel and the (position of) gas and brake pedals. (Or onemay even consider the torque on the steering wheel as control input).
To model the system by a set of differential equations, first define the states,compare figure 2. To avoid complications due to the different angular velocities ofinside and outside wheels in real world cars, consider the simpler case of a bicycle.More specifically, denote the position of the center of the rear wheel by (z1,z2)∈R2,
6 Contents
and identify the angle of the bicycle with the z1-axis and the angle of the front wheelwith respect to the bicycle with points θ and ϕ on the circle S1. Given the lengthL between the axes of the front and back wheels, the position of the front wheel is(z1 +Lcosθ ,z2 +Lsinθ).
������
�
�
�
-
6
z1
z2 ����L
θ
φ
Fig. 2 The states of the bicycle
The dynamics are determined by a non-slip condition (1) that imposes that thewheels can only roll in the natural direction, but they cannot slip sideways.{
0 = sinθ dz1− cosθ dz2
0 = sin(θ +ϕ) d(z1 +Lcosθ)− cos(θ +ϕ) d(z2 +Lsinθ)(1)
Fig. 3 An inverted pendulum
�����
�����������t
CCO
u ?
g
θ
ϕ
A more challenging second example is an inverted pendulum. Its control wasdramatically illustrated in the Apollo program in the 1960s with the 105 meter tallSaturn V rocket being kept vertical by sophisticated control of the directions of thethrust generated by powerful nozzles at its bottom. Among the many variations ofinverted pendulums and double pendulums, here consider the case recently studiedby Hauser [37]. The reference state that is to be stabilized, if possible, is a verticalinverted pendulum attached to the end of a horizontal arm. The control is the angularacceleration about the shoulder.
Contents 7
The equations of motions are readily derived from first principles. Denoting theratio of the lengths of the arms by β and the control (angular acceleration) by u, oneobtains:
θ = u
ϕ = sinϕ−β cos(ϕ−θ)θ 2 +βusin(ϕ−θ). (2)
These examples will be revisited in sections 2.1 and 3.1 through 3.3 where theircontrollability properties will be analyzed, and suitable approximating systems willbe constructed.
Many similar introductory examples including rolling pennies, sliding skates,balls rolling on each other may be found in the literature, in particular in populartextbooks such as those by Jurdjevic [43], Bloch et. al. [9], Bressan and Piccoli [11],and Bullo and Lewis [12].
1.2 Controlled dynamical systems
Control theory is a broad discipline that studies a broad range of dynamical systems,ranging from discrete systems to systems governed by partial differential equations.What they have in common and what formally distinguishes them from generic dy-namical systems is that they admit inputs (controls), and generally also specifiedoutputs (observed, or measured quantities). Even more distinctive are the questionsasked and problems studied: Instead of analyzing, describing and predicting be-havior, control theory fundamentally asks inverse questions such as: determine, iffeasible, an input that will result in a desired future behavior.
The primary concern of this study here is the analysis of systems that are gov-erned by deterministic ordinary differential equations, generally evolving on smoothmanifolds. In various places it will be apparent how this work may extend, gener-alize, or translate into systems that are infinite dimensional (governed by partialdifferentia equations), or that evolve in discrete time steps, or with stochastic sig-nals. The objective is not utmost generality, i.e., the weakest analytic hypotheses, butinstead to illuminate the underlying geometric, algebraic, and eventually combina-torial structures. The simple mechanical systems presented in the preceding sectionserve as prototypes for applications. These quickly can be made more demandingsuch as the attitude control of a satellite in space, which may itself consist of multi-ple linked rigid bodies.
Classically, control theory distinguishes two complementary ways of modelingsystems. On one side, one considers input signals u(t) that are fed into a box (possi-bly a black box) which produces corresponding output signals y(t). Mathematically,this point of view corresponds to maps between spaces of functions of a singlecontinuous (or discrete) variable (time). Critical are careful considerations of thedomain, range, and the regularity of the operators. This point of view is particularlysuitable for considering system interconnections, where the output of one system is
8 Contents
- -
i- - -
�
6
+
u y
r
u = K(y)
y
Σ
Σ
K
Fig. 4 Open loop and closed loop control with feedback controller K
the input for another system. Of particular interest are feedback interconnections inwhich the output y of the plant, the physical system is the input for the (automatic)controller which computes and generates an output u = K(y) which is fed back asthe input to the plant (usually added to an external reference input r), compare fig-ure 4.
On the other side, systems may be modeled as (ordinary) differential equationswhich depend on generally time-varying parameters (inputs), and which evolve ona manifold, called the state-space. Outputs, or observed measurements, are modeledby smooth functions on the state space. Generally, these are not injective, i.e., at anyinstant of time, they only provide partial information about the actual state of thesystem.
1.3 Fundamental questions in control
Arguably two of the most central objectives and concepts are
• controllability, and• feedback stabilization.
In (uncontrolled) dynamical systems uniqueness of solutions of initial value prob-lems leaves the primary remaining question is: where do solutions go? One studiesthe limit sets. In contrast, in controlled dynamical systems the central objective is tomake a system (trajectory) go to a desired state. The first question is whether the sys-tem is controllable, i.e., decide whether it is possible to transfer the state from anyinitial state to any desired terminal state using some admissible control input u(t).
Rather than steering the system with open-loop controls, that is inputting controlsthat are functions of time, a main objective in automatic control is to close the loopby feeding back control signals u(x) or u(y), that are are automatically generatedfrom information about the state x or the output y, respectively.
Algebraic and combinatorial tools play an important role for deciding control-lability and studying related properties. The algebraic and analytic background offeedback stabilization of nonlinear systems is comparatively less well understood.These lecture notes primarily address issues related to controllability.
Contents 9
In order to study controllability and stabilizability, one needs to develop otherfundamental notions such as
• observability,• system identification,• state-space realization,• equivalence and normal forms,• system approximation.
Given a collection of input-output pairs, system identification is concerned withidentifying (parameters of) the mapping from inputs to outputs. Going one stepfurther, state space realizations represent the system typically by a collection offirst order differential equations (vector fields) on a smooth manifold, together withfunctions on the state space that provide the outputs. Roughly speaking a systemis called observable if the outputs (over some time intervals) provide sufficient in-formation to determine the state of the system. From an algebraic operator pointof view, observability (output-to-state) is in a natural way a notion dual to con-trollability (input-to-state). Two systems in state-space form that generate the sameinput-output behavior may be considered equivalent. Formally one studies equiva-lence of state-space systems under the action of groups of diffeomorphisms (localcoordinate changes) and feedback transformations, identifying invariants under suchactions, and singling out normal forms for each orbit.
Both from the modeling perspective (“no model is perfect”), and for analyticalwork it becomes important to develop suitable notions of when one system is a goodapproximation of another one.
Much of the algebraic and combinatorial work in these lecture notes deals withthe interplay of input-output operators with state-space realizations, and with ap-proximations and normal forms. The emphasis is on matching the control inputs,outputs and observations to corresponding algebraic and combinatorial structures.System identification will not be addressed explicitly.
Once one has a generated a solution for a control problem, it is natural to refineit, and ask for example for
• optimization, and• robustness.
From a geometric point of view optimality is in a well-defined sense dual to con-trollability, and as such will appear in the sequel. Going beyond the mathematicalsolution of the idealized problems (model, system), control engineering is as muchan art as it is a science, and beyond optimizing solutions, much effort is devotedto addressing modeling errors, rejecting disturbances, practical implementations ofdigital controllers and the like. Such aspects go much beyond the framework ad-dressed in these lecture notes.
10 Contents
2 Analytic foundations
For the input-output operator point of view, the domain and range of the operatorsplay key technical roles. In the case of linear systems, best studied are systemsrepresented by bounded linear operators that map L2 input signals (Lebesgue squareintegrable) to L2 output signals. In the case of nonlinear systems the situation isconsiderably less clear cut. This work will concentrate on systems which can berepresented by formal power series which have desirable convergence properties.The focus here are the algebraic and combinatorial aspects. For in-depth treatmentsof analytic issues, especially convergence the interested reader may start with [29,30, 56]. A detailed description of the Faa di Bruno approach is given in [28], whereasthe convergence issues of feedback interconnections [85] have only been resolvedin 2012.
For systems represented in state space form, i.e., by differential equations, ortime-varying vector fields on manifolds, the first questions are about existence ofsolutions, and regularity properties of the reachable sets. The existence proofs thatbasically refine Picard iteration already foreshadow the Hopf-algebraic structures,and will be developed in detail in the next section. Similarly, variation of parametersis a key tool for arguments about controllability and optimality, and it is intimatelyrelated to combinatorial structures.
Throughout, this article employs notation and viewpoints of the chronologicalcalculus. This takes a functional analytic perspective: instead of considering thesolutions of differential equations (i.e. the flows) as families of diffeomorphismson a manifold, they are considered as families of automorphisms on the algebra ofsmooth functions on the manifold.
Before addressing central control issues, the next subsections first review anddevelop some mathematical foundations.
2.1 State space models and vector fields on manifolds
This work focuses on the rich class of smooth systems on manifolds that are affine inthe control. These commonly arise in applications. Very attractive from a mathemat-ical point of view, as well as for engineering applications, is the way the dynamicssplits into time-varying controls that may be manipulated, and static vector fieldsthat determine the geometry of the system. For practical implementations, one maybe able to compute objects related to the geometry off-line and in advance, and inreal-time only handling the less demanding operations on the time varying signals.In traditional notation1, one writes
1 Whereas most of the subsequent theoretical discussion will use the chronological notation, in thisintroduction and in concrete examples, we will use traditional notation, including column vectors.
Contents 11
q = f0(q)+m
∑i=1
ui fi(q) (3)
y = ϕ(q) (4)
The state q ∈Mn takes values in a smooth manifold Mn. The space U of controlsconsists of measurable functions u : [0,T ] 7→U ⊆ Rm that typically are assumed totake values in a compact subset U ⊆ Rm that contains 0 ∈ int U in its interior. Theoutput function ϕ : M 7→ Rp is smooth, and the dynamics are defined by smoothvector fields fi ∈ Γ ∞(M) on the manifold Mn.
For the example of parallel parking a car (1), a natural state space M = R3×S1×S1 which locally may be identified with R5. The state may be chosen in local coor-dinates as q = (v,ϕ,z1,θ ,z2) ∈ R× S1×R× S1×R, where the forward speed v isdefined by dz1 = vcosθ dt and dz2 = vsinθ dt. Manipulating the second constraintin (1) one quickly obtains, using elementary trigonometry
dθ =vdtL· cosθ · tan(θ +ϕ)− sinθ
sinθ · tan(θ +ϕ)+ cosθ=
vL
tanϕ dt (5)
From practical considerations, natural choices for the control inputs are the forwardacceleration v= u1 and the angular velocity of the steering angle ϕ = u2. Combiningthese differential equations, the controlled dynamics may be written in the standardform of (3) with m = 2 by introducing the vector fields f0, f1, f2 written as columnvectors in terms of the coordinates q
f0(q) =
00
q1 cosq4
q1 tanq2
q1 sinq4
, f1(q) =
10000
, and f2(q) =
01000
. (6)
For the example of the pendubot (2), a natural state space is the tangent bundleM = TT2 of the torus which locally may be identified with R4. The state may bechosen in local coordinates as q = (θ ,θ , ϕ,ϕ). The controls take values in a setU = [−u0,u0]⊂R, and the dynamics are determined by a drift vector field f0 and acontrolled vector field f1 which written as column vectors in local coordinates are
f0(q) =
0q1
sinq4−β cos(q4−q2)(q1)2
q3
, and f1(q) =
10
β sin(q4−q2)0
. (7)
12 Contents
2.2 Chronological calculus
In geometric control theory it is standard to define (or identify) vector fields as(with) first order partial differential operators. For example the vector fields, writtenas column vectors in the preceding example, are more precisely understood as thelinear maps f0, f1 : C∞(TT2) 7→C∞(TT2)
f0 = q1 ∂
∂q2 +(sinq4−β cos(q4−q2)(q1)2) ∂
∂q3 +q3 ∂
∂q4 , and
f1 = ∂
∂q2 +β sin(q4−q2) ∂
∂q4 . (8)
It is convenient to go further and also consider points on the state-space and flowsof vector fields (solutions of differential equations) as operators on the commutativealgebra of smooth functions on the state space. This functional analytic approachwas pioneered in nonlinear control theory by Agrachev and Gamkrelidze [1, 2] inthe late 1970s. For a detailed discussion see the recent book [3], or the technicaldetails provided in [51] and [68].
In particular, for a point p ∈ M denote by p the multiplicative linear functionalp : C∞(M) 7→R defined by p(ϕ) = ϕ(p). For ϕ,ψ ∈C∞(M) and c ∈R this satisfies
p(ϕ + cψ) = p(ϕ)+ cp(ψ) and p(ϕ ·ψ) = p(ϕ) · p(ψ). (9)
In the sequel, this article shall suppress the notational distinction between p andp. The most immediate effect is that what traditionally is written as ϕ(p), f (p),and ( f ϕ)(p) for p ∈M, ϕ ∈C∞(M) and f ∈ Γ ∞(M) a smooth vector field on M, issimplified by the consistent notation pϕ , p f , and p f ϕ . Moreover, for a point q ∈Mand a set S of vector fields, differential operators or diffeomorphisms on M we alsouse the notation qS = {q f : f ∈ S}. This is most commonly used when S is the Liealgebra L( f ,g) generated by vector fields f and g (or a subspace thereof).
For the (local) flow of a vector field f ∈ Γ ∞(M) introduce the notation (t,q0) 7→q0et f . This notation will be further justified in the next section. In the chronologicalframework, rather than considering the local flow a local diffeomorphism of M, it isconsidered locally as an automorphism on the algebra of smooth functions on M. Itis (locally) defined for ϕ ∈C∞(M) by
e0 fϕ = ϕ and d
dt et fϕ = f et f
ϕ (10)
Evaluating the latter at any point q ∈M yields the formula ddt qet f ϕ = qet f f ϕ . Note
that et f f ϕ = f et f ϕ . This notation becomes particular handy when differentiatingcompositions of flows. For example for t ∈R, q∈M, ϕ ∈C∞(M), and f ,g∈Γ ∞(M)in the following calculation
ddt qet f etg
ϕ = qet f f etgϕ +qet f etggϕ (11)
the etg in the first term has to be interpreted as the tangent map (or differential) ofthe flow etg. Indeed, in traditional notation one might write
Contents 13
ddt ϕ(Φg
t ◦Φf
t (q)) = (Φgt∗( f ))(ϕ)(Φ f
t (q))+(gϕ)(Φgt ◦Φ
ft (q)). (12)
In the chronological notation it is completely clear how to interpret any expo-nential, depending whether one reads it as acting on smooth functions on the right,or on differential operators or points (which are considered zeroth order differentialoperators) on the left. For example, it makes perfect sense to calculate
ddt
∣∣t=0qet f etge−t f e−tg
ϕ =
= q(et f f etge−t f e−tg + et f etgge−t f e−tg− et f etge−t f f e−tg− et f etge−t f e−tgg
)ϕ∣∣t=0
= q f ϕ +qgϕ−q f ϕ−qgϕ = 0. (13)
It is straightforward [51] to calculate the next derivative and formally derive
12
d2
dt2
∣∣∣t=0
qet f etge−t f e−tgϕ = q( f g−g f )ϕ = q[ f ,g]ϕ (14)
which defines the Lie bracket of two smooth vector fields. Recall that every smoothvector field f ∈Γ ∞(M) is a derivation on the algebra C∞(M), i.e., for ϕ,ψ ∈C∞(M)it satisfies f (ϕ ·ψ) = ( f ϕ) ·ψ +ϕ · ( f ψ). At a point q ∈M this is written as
q f (ϕ ·ψ) = q f ϕ ·qψ +qϕ ·q f ψ. (15)
As one readily verifies, in general the composition (g ◦ f ) : ϕ 7→ g f ϕ is in gen-eral not a derivation. However, the Lie bracket [ f ,g] : ϕ 7→ f gϕ − g f ϕ is always aderivation, i.e., a first order partial differential operator, or a smooth vector field.
Another example that demonstrates how this formalism simplifies appearanceand calculations is a version of the ad formula. This formula is used frequently incontrol to calculate the effect of moving control variations along the flow of anothervector field.
ddt qet f ge−t f = qet f f ge−t f −qet f g f e−t f = qet f [ f ,g]e−t f . (16)
In particular, evaluating this identity at t = 0 yields
ddt
∣∣t=0 qet f ge−t f = q[ f ,g]. (17)
In traditional notation, using Φ f for the local flow of f , this formula appears in termsof the differential of the pullback and gives rise to the Taylor expansion (assuminganalyticity)
Φf−t∗(g(Φ
ft (q))) =
∞
∑k=0
tk
k! (adk f ,g)(q). (18)
where (adk f ,g) is defined recursively by (ad0 f ,g)= g and (adk+1 f ,g)= [ f ,(adk f ,g)]for k ∈ Z+
0 .
This work is mainly concerned with the algebraic and combinatorial aspects. Butwe note that the chronological calculus is also an effective tool for the topological
14 Contents
and analytic aspects. In particular, the commutative algebra C∞(M) of smooth func-tions has a natural topology: A sequence {ϕk}∞
k=0 ∈C∞(M) converges to ψ ∈C∞(M)if and only if for every compact set K⊆M and every finite set of smooth vector fieldsf1 . . . fs ∈ Γ ∞M the sequence { fs . . . f2 f1ϕk}∞
k=0 converges uniformly to fs . . . f2 f1ψ
on K. Thus every tangent vector q f at any point q ∈M is clearly a continuous linearfunctional, and is also an element of the dual space of C∞(M), the space of com-pactly supported Schwartz distributions.
For a more detailed technical description of the topology and structure of C∞(M)as a Frechet space in terms of semi-norms see e.g. the introduction in [3]. Herewe will routinely make use of the fact that smooth vector fields on M and diffeo-morphisms of M may be regarded as continuous linear operators on C∞(M) as atopological vector space.
Finally, we remark that the chronological calculus of Agrachev and Gamkrelidzeachieves its full power when working with time-varying vector fields. However, forthe purposes of this work with its focus on elements of combinatorial Hopf algebrain control, the split into time-varying controls and static vector fields that definethe geometry is critically important. It is this splitting which allows us to developpowerful algebraic and combinatorial tools that apply in this important special case.
Remark. In the sequel we shall write q0 f when we want to emphasize an initialvalue q(0) = q0, but otherwise, in theoretical work appropriately use lower and up-per indices like ui, xi, fi wherever feasible and meaningful. But when dealing withspecific examples, at some we keep the traditional notation like f (0), [ f0, f1](0), x3
1when readability would suffer too much from a change.
2.3 Piecewise constant controls and theBaker-Campbell-Hausdorff formula
Briefly returning to the parking example we illustrate how the exponential prod-ucts provide insight into the controlled dynamics. Observe that in the coordinateschosen, the Jacobian matrices of partial derivatives of the vector fields (6) are allstrictly lower triangular. Consequently, due to the special cascade form of the asso-ciated differential equation, it is possible to write down the solution for any controlusing simple quadratures only (i.e., unlike in the case of general nonlinear differ-ential equations, here the solution can be expressed in terms of a finite number ofiterated integrals of scalar functions only). In particular, for any admissible controlη : [0,T ] 7→ U ⊆ R2 the corresponding integral dynamics are immediately calcu-lated as iterated integrals: First q1(t) =
∫ t0 η1
s ds, and q2(t) =∫ t
0 η2s ds. Then itera-
tively
Contents 15
q4(t) =∫ t
0
(∫ s
0η
1σ dσ
)tan(∫ s
0η
2σ dσ
)ds (19)
q3(t) =∫ t
0
(∫ s
0η
1σ dσ
)cos(∫ s
0
(∫ s
0η
1τ dτ
)tan(∫
σ
0η
2τ dτ
)dσ
)ds
q5(t) =∫ t
0
(∫ s
0η
1σ dσ
)· sin
(. . .q3
s . . .
)ds
To parallel park a car one needs to choose η to achieve a purely sideways motion,i.e. so that q(T ) = (0,0,0,0,C) for some C 6= 0. As everyday experience shows, thisis possible, but not completely trivial. The complexity of the required choices corre-sponds to the number of integrations needed. One possible solution using piecewiseconstant controls is given by the following choice (for some u1,u2 > 0) and results inthe pictured solution curves. The first three show graphs as function of time, and thefourth is a phase-plot showing the location of the car in the (z1,z2) = (q3,q5)-plane.
η(t) =
(u1,u2) if 0 ≤ t < 2(0,−u2) if 2 ≤ t < 6(−u1,u2) if 6 ≤ t < 10(0,−u2) if 10 ≤ t < 14(u1,u2) if 14 ≤ t ≤ 16
. (20)
Forward acceleration and speed Angular velocity and orientations
Components of velocity Position of the car
Fig. 5 Parallel parking a car (bicycle)
The solution of the corresponding sequence of differential equations
16 Contentsq = f0 + u1 f1 + u2 f2 if 0 ≤ t < 2q = f0 − u2 f2 if 2 ≤ t < 6q = f0 − u1 f1 + u2 f2 if 6 ≤ t < 10q = f0 − u2 f2 if 10 ≤ t < 14q = f0 + u1 f1 + u2 f2 if 14 ≤ t ≤ 16
(21)
may be written in exponential notation using τi = t1− ti−1 for the time between theswitching times ti in the example
q(t) = q0eτ1( f0+u1 f1+u2 f2)eτ2( f0−u2 f2)eτ3( f0−u1 f1+u2 f2)eτ4( f0−u2 f2)eτ5( f0+u1 f1+u2 f2)
(22)This product of exponentials may be formally rewritten using the classical Baker-Campbell-Hausdorff formula, compare [13],
eX eY = eX+Y+ 12 [X ,Y ]+ 1
12 [X ,[X ,Y ]− 112 [Y,[X ,Y ]]+ 1
24 [X ,[Y,[X ,Y ]]]... (23)
where each X and Y take the form τ1( f0±u1 f1±u2 f2).The resulting formula as a single exponential is extremely unwieldly and not
useful for practical computations, even in this very simple example of parking acar. Nonetheless it was a popular tool in the 1970s and early 1980s when the firsthigher order conditions for nonlinear controllability and optimality were derived.However, it shows clearly that each summand in the exponent is an iterated Liebracket of the vector fields fi whose coefficient is a homogeneous polynomial in ui
and τ j. Moreover, the degrees of each of these polynomials in u and in τ are equalto the numbers of controlled vector fields f1, . . . fm and equal to total number ofall vector fields f0, . . . fm involved in the iterated bracket, respectively. This will bemade precise in the next sections which aim at providing more efficient alternativesto this classical expansion. Specifically, in realistic control applications the numberof switchings may be very large, and more generally the controls may not evenbe piecewise constant. Moreover, the exponent in the classical Baker-Campbell-Hausdorff formula is written as a linear combination of all right-to-left iterated Liebrackets. But these are always linearly dependent because of relations resulting fromthe Jacobi identity, and hence the coefficients in this linear combinations are notwell-defined. For recent alternatives see e.g. [25, 26, 78] and the later sections inthese notes.
2.4 Picard iteration and formal series solutions
For vector fields that are locally Lipschitz continuous, the classical method of Picarditeration together with a fixed-point argument yields existence and uniqueness oflocal flows. In the context of affine control systems it is motivated by first rewritingthe system (3) as an equivalent integral equation. More specifically suppose that uis a measurable function taking values in a compact set U , fi ∈ Γ ∞(M) are smooth
Contents 17
vector fields, ϕ ∈C∞(M) is a smooth function on a smooth manifold M, and q0 ∈M is an initial point. For notational convenience, introduce the additional controlu0 ≡ 1. Then the output ϕ evaluated along a solution curve t 7→ qt ∈M of (3) for tsufficiently small satisfies the equivalent integral equation
qtϕ = q0ϕ +m
∑i=0
∫ t
0ui
s ·qs fiϕ ds (24)
This motivates one to define and study iterates of the mapping P from absolutelycontinuous curves qt in M to absolutely continuous curves (P[q])t defined for ϕ ∈C∞(M) by
(P[q])tϕ = q0ϕ +m
∑i=0
∫ t
0ui
s ·qs fiϕ ds (25)
Fixed points of this mapping P correspond to solution curves of (24), and thus of(3). Using the chronological formalism it can again be shown that on sufficientlysmall time intervals the iterates of this map converge to such a fixed point, whichindeed is again an absolutely continuous curve. This curve is the unique solution(on that interval) for the initial value problem (3). For a meticulous exposition of alltechnical details in terms of seminorms on C∞(M) see [3].
A small modification of the Picard iteration process yields a formal series ex-pansion of the local flow. Under the same hypotheses as above, formally iterate theintegral equation (24) by substituting the right hand side of the corresponding equa-tion for qt fiϕ (instead of qtϕ) into (24).
qtϕ = q0ϕ +m
∑i=0
∫ t
0ui
s ·(
q0ϕ +m
∑j=0
∫ s
0u j
σ ·qσ f j fiϕ dσ
)ds
= q0ϕ +m
∑i=0
(∫ t
0ui
s ds)(
q0 fiϕ
)+
m
∑i, j=0
∫ t
0
∫ s
0ui
sujσ ·qσ f j fiϕ dσ ds
= q0ϕ +m
∑i=0
(∫ t
0ui
s ds)(
q0 fiϕ
)+
m
∑i, j=0
(∫ t
0
∫ s
0ui
sujσ dσ ds
)(q0 f j fiϕ
)+
m
∑i, j,`=0
∫ t
0
∫ s
0
∫σ
0ui
sujσ u`τ ·qτ f` f j fiϕ dτ dσ ds (26)
= . . .
Note that at each stage, every term except the last one splits into an iterated integralthat depends on the controls u only, and a higher order partial derivative of the outputfunctions evaluated at the initial point.
Continuing this iteration process yields a formal infinite series
CF(t,u, f ,ϕ,q0) = ∑I
(∫ t
0uI)·q0 fIϕ (27)
18 Contents
where the sums is taken over all multi-indices I = (i1, i2, . . . , is) ∈ {0,1, . . .m}s withs≥ 0,
∫ t0 u /0 = 1, f /0ϕ = ϕ , and inductively for (I, j) ∈ {0,1, . . .m}s+1
∫ t
0u(I, j) =
∫ t
0u j
s
∫ s
0uI
σ dσ ds and f(I, j)ϕ = fI( f jϕ) (28)
Note that the convenient indexing is the reverse order from the iteration illustratedabove. In particular, when considering fI as acting on the left on a point or on theright on a smooth output function correspond to reversed orders of composition. Forthe iterated integrals the notation is more ambiguous as in our setting the controlscommute (they are scalars), and the products ui1
t1 ui2t2 = ui2
t2 ui1t1 could be written in either
order. However, the order of integration (indexing of integration variables) must beconsistent with the order of application of the partial differential operators.∫ t
0u(i1,i2,i3) =
∫ t
0ui3
t3
∫ t3
0ui2
t2
∫ t2
0ui1
t1 dt1 dt2 dt3 (29)
q f(i1,i2,i3)ϕ = q fi1 fi2 fi3ϕ (30)
This apparent reversal of the indexing of these terms is not purely a choice of no-tation. Indeed it may be interpreted as the appearance of the antipode in the formalHopf algebraic treatment.
This expansion for qt as a curve in the dual of C∞(M) considered as a formalseries of partial differential operators CF(t,u, f , ·,q0) never converges even weakly(if fi are nontrivial) to a well-defined functional defined on all of C∞(M). Instead theexpression (27) must be considered as an asymptotic series for the output ϕ alongthe solution qt of (3), see e.g. [80]. Technical details may be found in [80] and e.g.section 2.4.4 in [3].
2.5 The Chen-Fliess series and abstractions
A central theme of these notes is that commonly laborious manipulations performedon analytic or dynamic objects are in their core of a combinatorial and algebraicnature. The structure becomes more clearly discernable and the work much easierwhen the manipulations are done at the level of the corresponding combinatorialobjects. The main objective here is to abstract the asymptotic series (27) for theevolution of an output function ϕ along solution curves of a control system to acombinatorial abstract form of the Chen-Fliess series, which may be considered anunevaluated form CF(t,u, f ,ϕ,q0) in (27). This section introduces the basic termi-nology and notation used in the sequel, and discusses some analytic properties ofthe evaluation maps. Their algebraic properties will be elaborated in section 4.3.
To facilitate readability and to conform to standard verbiage in the combinatoricsliterature, it is convenient to allow indexing sets (for vector fields, coordinates, etc.)that are not necessarily sets of integers {0,1, . . .m}. Start with a set Z which in these
Contents 19
lecture notes will generally be finite. Call the set Z an alphabet and its elements let-ters. The set of all finite nonempty sequences taking values in Z is denoted by Z+.The empty sequence /0 is sometimes also written as e. Define Z∗ = Z+∪{e}. Con-catenation of sequences endows Z∗ with a natural noncommutative product struc-ture. For notational simplicity abbreviate sequences and their products by juxtapo-sition, e.g. if w = (ai1 ,ai2 , . . .ais) and z = (b j1 ,b j2 , . . .b jt ), then simply write, e.g.,w= ai1ai2 · · ·ais and wz= ai1ai2 · · ·aisb j1b j2 . . .b jt . Next introduce the associative al-gebra A(Z) of all finite linear combinations of elements of Z∗ with coefficients in abase field, which in these lecture notes is always the field of real numbers R. In otherwords, A(Z) is the algebra of noncommutative polynomials over Z with coefficientsin R. Write A+(Z) for the subalgebra of all finite linear combinations of elementsof Z+, the subalgebra of noncommutative polynomials with zero constant term. TheLie algebra L(Z) is the smallest subspace of A(Z) that contains Z and is closed underthe Lie bracket [ · , · ] : A(Z)×A(Z) 7→ A(Z), [ · , · ] : (w,z) 7→ [w,z] = wz− zw.
The algebras A(Z), and L(Z) have natural graded structures. For each a ∈ Z, themonoid homomorphism δa : Z∗ 7→ Z+
0 defined on the generators b ∈ Z by δa(b) =δa,b (Kronecker delta) counts the number of occurrences of the letter a in a word inZ∗. If Z = {a0,a1, . . .am} it is convenient to introduce for k ∈ Z+
0 and ` ∈ (Z+0 )
m thesubsets Z(k,`) ⊆ Z defined by
Z(k,`) = {w ∈ Z∗ : k = δa0(w) and `= δa1(w)+ . . .+δam(w)} (31)
The distinguished letter a0 plays an important role for e. g. controllability, and corre-sponds to an uncontrolled drift vector field. The corresponding homogeneous sub-spaces A(k,`) ⊆ A(Z) and L(k,`) ⊆ L(Z) are spanned by (Lie) polynomials that arelinear combinations of words in Z(k,`).
Following [51], the algebra A(Z) has a natural topology as a uniform spacewhere the uniform structure is the collection of the sets (called entourages) ES ={(∑w∈Z∗ cww,∑w∈Z∗ dww) : cw,dw ∈ R, and cw = dw if w ∈ S}, indexed by allfinite subsets S ⊆ Z∗ of words. The completion of A(Z) with respect to this uni-form structure is the algebra A(Z) of formal power series over Z. The multiplica-tion on A(Z) is uniformly continuous and thus extends to a continuous map fromA(Z)× A(Z) to A(Z). The basis Z∗ of A(Z) induces a natural pairing 〈·, ·〉 : A(Z)×A(Z) 7→ R, defined by
〈 ∑w∈Z∗
cww , ∑w∈Z∗
dww〉= ∑w∈Z∗
cwdw (32)
where all but a finite number of the coefficients cw vanish. With this pairing andupon identifying p∈ A(Z) with the linear functional q 7→ 〈p,q〉 on A(Z), A(Z) is thealgebraic dual of A(Z), and A(Z) is the topological dual of A(Z).
In 1957 K. T. Chen [15], see also [14, 16], introduced the Chen series as a ge-ometric invariant of curves in Euclidean space Rm. He associated to every curve aformal power series ∑w∈Z∗ cww (on an alphabet Z with m letters) where the coeffi-cients cw ∈ R were defined as iterated line integrals along the curve. In the 1970s,
20 Contents
Fliess [22, 23] reinterpreted the Chen series in the context of control systems. Thecurves are the solution curves of the nonlinear system (3) that is affine in the con-trols, and the coefficients cw are iterated integrals of the controls. For every fixedcurve γ : [0,T ] 7→ Rm or every fixed control η : [0,T ] 7→U , either series is one el-ement in A(Z), or a curve in A(Z) when considering variable end-times t ∈ [0,T ].Recently, in the context of rough paths and stochastic signals a large body of closelyrelated theory has been developed in which these series are commonly referred toas signatures of signals, see e.g. [36, 59].
To make best use of the underlying Hopf algebra structures, it is beneficial to fur-ther abstract the Chen-Fliess series, and to start with the formal representation of theidentity map on A(Z), by identifying the representation of IdA(Z)(·) =∑w∈Z∗〈w , ·〉w.with the formal power series
CF = ∑w∈Z∗
w⊗w ∈ A(Z)⊗ A(Z). (33)
There are numerous ways to map this formal object to a partially evaluated series(27) by specifying some or all of: a set of vector fields, an output function, an initialpoint, a set of control inputs, and a terminal time. For the most important mappingsintroduce the following notation.
For a sequence of smooth vector fields F = { fa0 , . . . , fam} ⊆ Γ ∞(M) indexed bya set Z = {a0, . . .am} define the map
F : Z 7→ Γ∞(M) by F (a) = fa. (34)
where fa ∈Γ ∞(M) is considered as a linear operator (indeed, a derivation) mappingC∞(M) back to C∞(M). Next, for a word w ∈ Z∗ and a letter a ∈ Z, and any functionϕ ∈ C∞(M) define F (aw)(ϕ) = fa (F (w)ϕ). Reusing the same symbols, extendthis map linearly to a Lie algebra homomorphism F : L(Z) 7→ L(F)⊆ Γ ∞(M) andto an associative algebra homomorphism that maps A(Z) to linear combinationsof partial differential operators on C∞(M). Under suitable growth conditions on thecoefficients and conditions on the vector fields (e.g., compactly supported), this mapfurther extends (still using the same symbol) to a map on formal power series A(Z)to, e.g., compactly supported Schwartz distributions on M. For technical details seee.g. [1, 2, 3, 51, 80].
On the other side, rather than directly evaluating on control inputs, it is advan-tageous to map A(Z) to an algebra of iterated integral functionals, compare [51].From an algebraic point of view it is most convenient to work with the primitivest 7→
∫ t0 ua(s)ds of the controls inputs rather than the controls themselves as it was
done in [51]. However, in this context, to enhance readability and facilitate con-nections with control applications, these notes shall stay with the traditional no-tions of the inputs u ∈ U . Note that these are usually presumed to be essentiallybounded integrable functions of time, whereas their primitives are absolutely con-tinuous functions. The main interest lies in a Zinbiel subalgebra (to be introducedlater) of the set MU of all mappings from a space of controlsU (typically to betaken as L1([0,T ],U), where U ⊆ Rm is a compact subset containing the origin in
Contents 21
its interior, compare the assumptions for system (3)) to the set of absolutely contin-uous functions AC([0,T ],R).
Thus without further specifying topological or algebraic structures on the rangeat this time, introduce the map ϒ : A(Z) 7→MU first for a letter a ∈ Z, any wordw ∈ Z∗, for any control u ∈U and t ∈ [0,T ] by
ϒa(u)(t) =
∫ t
0ua(s)ds and ϒ
wa(u)(t) =∫ t
0ua(s) ·ϒ w(u)(s)ds (35)
Extend this map linearly to an associative algebra homomorphism that maps A(Z)to a subset of iterated integral functionals in MU .
Recognize that with this notation the infinite series solution (27) obtained byPicard iteration is the image of the identity on A(Z) under distinguished evaluationmaps.
CF(t,u, f ,ϕ,q0) = ∑w∈Z∗
ϒw(u)(t) ·q0F (w)ϕ. (36)
Rather than thinking of I or w indexing some iterated integrals or partial differentialoperators, these notes take interest in the algebraic properties of the mappings ϒ andF when the domain and codomain are equipped with suitable algebraic structures.(The analytic and topological properties of these maps are largely beyond the scopeof these notes.)
Of particular interest is the duality between the partially evaluated images ofthe abstract series CF . In particular, rather than considering CF itself, or the fullyevaluated expansion (ϒ ⊗F )(CF), of special interest are the two expansions (Id⊗F )(CF) and (ϒ ⊗ Id)(CF). The first associates to every control system defined bya sequence of smooth vector fields a formal power series whose coefficients arepartial differential operators on the state space. Upon further evaluating this on anoutput function, and possibly an initial state q0, this view associates a formal powerseries to every system. The second is a formal power series whose coefficients areiterated integral functionals. Upon further evaluating this on control inputs, this viewassociates a formal power series to every control. These two different kinds of powerseries may naturally be paired to yield the output response for a specific controlwhen fed into a specific system. While analytically, these appear as very differentobjects, algebraically the series representations of controls and of systems exhibit anatural duality.
Further refinements are possible by considering fixed terminal times T , or curvesparameterized by t ∈ [0,T ], or by considering local flows (local diffeomorphisms)by leaving q0 open, or considering specific solution curves of the Cauchy problemfor a specified initial condition q0. The special case where a vector-valued ϕ is alocal diffeomorphism ϕ : M 7→ Rn for some nonempty open set V ⊆ M yields thefamiliar form of the system (3) represented in local coordinates.
22 Contents
3 Controllability and optimality
A control system is (globally) controllable if for every pair of states q0, q1, thereexists an admissible control that steers the system from initial state q0 to the terminalstate q1. Many variations of this concept have been proposed, proven useful, andhave been investigated. Here we focus on the well-studied property of small-timelocal controllability (STLC) which is dual to optimality in a geometric way.
In separate subsections the following items are addressed. The first step is to re-late controllability to a lack of integrability and Frobenius’ theorem. The followingsection accounts for the presence of a distinguished drift vector field which breaksthe symmetry and thus necessitates more refined statements to accurately describethe reachable sets. For most controllable systems of practical relevance, a simplealgorithm allows one to construct nilpotent approximating system which preserveimportant geometric properties, but are much easier to use. Theorems for controlla-bility can be rewritten as statements about optimality, generalizing the PontryaginMaximum Principle. Their proofs rely on approximating cones to reachable sets andfamilies of control variations, and utilize the graded structures and the correspon-dence of Lie brackets and iterated integrals.
3.1 Reachable sets and accessibility
Consider a control system ΣF,U defined by a finite sequence F = ( f0, f1 . . . fm) ofsmooth vector fields on a manifold M of dimension n, a compact convex set U ⊂Rm
that contains 0 ∈ int U in its interior, and the set U of all measurable functions utaking values in U , and each defined for some Tu > 0 on the interval [0,Tu]. For eachsuch control u ∈U the dynamics are governed by the differential equation
qt = qt f0 +m
∑i=1
uit qt fi (37)
For q0 ∈ M, η ∈ U and Tη > 0 sufficiently small, there exists a unique curveξ (·;q0,η) : [0,T ] 7→ M such that ξ (0;q0,η) = q0 and for almost all t ∈ [0,T ] (intraditional notation)
ξ (t;q0,η) = f0(ξ (t;q0,η))+m
∑i=1
ηi(t) fi(ξ (t;q0,η)). (38)
We may abbreviate ξ (t;q0,η) and write ξ (t) when there is no danger of confusion.
Definition 3.1 For a control system ΣF,U as above, q0 ∈M and T ≥ 0 sufficientlysmall, the reachable sets from q0 at time T , and in time at most T , are
q0RT = {ξ (T ;q0,η),η ∈U }, and q0R≤T =⋃
0≤t≤T
q0Rt . (39)
Contents 23
Definition 3.2 For q0 ∈M, the control system ΣF,U is accessible from q0 if for everyT > 0, int q0RT 6= /0.
By concatenating motions along the integral curves of the control vector fieldsfi, one may generate infinitesimal motion in the direction of the Lie brackets of thevector fields. In particular, to interpret the equation (14) in terms of control systems,consider a system without drift (i.e. f0 ≡ 0) and m = 2 controls. For the piecewiseconstant control η : [0,T ] 7→U = [−1,1]2 defined by
η(t) =
(1,0) if 0≤ t < T(0,1) if T ≤ t < 2T
(−1,0) if 2T ≤ t < 3T(0,−1) if 3T ≤ t ≤ 4T.
(40)
One calculates ξ (4T ;q0,η) = q0 +T 2 q0[ f1, f2]+O(T 3). Increasing the number ofswitchings, it is possible to generate infinitesimal movements in the directions ofhigher order iterated Lie brackets.
The Frobenius integrability theorem guarantees that for every sufficiently smalltime T > 0 the reachable set from q0 at time T is contained in the integral manifoldof the involutive closure, the Lie algebra L( f0, f1, . . . fm) generated by the vectorfields fi. The Hermann Nagano theorem [65, 80] guarantees that in the case of ana-lytic vector fields the reachable set has relatively open interior in this manifold, too.In particular, this implies the following algorithmic criterion to decide whether ananalytic system is accessible.
Theorem 3.1 For q0 ∈M, a real analytic control system ΣF,U is accessible from q0if and only if the vector fields f0, f1, . . . fm satisfy the Lie Algebra Rank Condition[LARC]
dim q0L( f0, f1, . . . fm) = n. (41)
For an easily accessible proof see section 4.3 of the textbook [75].
Returning to the parallel parking example, one readily computes in local co-ordinates the iterated Lie brackets fπ3 = [ f1, f0]], fπ4 = [ f0, [ f1, f2]], and fπ5 =[[ f1, f0]], [ f0, [ f1, f2]]] (written in traditional notation as column vectors)
q fπ3 =
00
cosq4
tanq2
sinq4
, q fπ4 =
000
sec2 q2
0
, and q fπ5 =
00
−sinq4 sec2 q2
0cosq4 sec2 q2
.
Evaluating fπ1 = f1, fπ2 = f2 together with these three iterated Lie brackets at q0 = 0
yields five independent vectors q0 fπk = ∂
∂qk
∣∣∣q0
which are a basis for the tangent
space at q0. Thus the system satisfies the LARC and it is accessible from q0.It is noteworthy that the length of the iterated Lie brackets computed above match
the order of the first nonvanishing term in the Taylor expansions (with respect to
24 Contents
time) of the iterated integrals in (19). For example, the five factors in the iteratedLie bracket fπ5 correspond to the formula for q5(t) in (19) being fifth order in time:the lowest order for which dk
dtk q5(t)|t=0 does not vanish is k = 5.
3.2 Small-time local controllability
Whereas accessibility means that the reachable set from a fixed starting point isnot constrained to lie in a lower dimensional integral submanifold, this does notguarantee controllability. Consider the example in the plane with a scalar control
q1 = u (42)q2 = (q1)k. (43)
Here, for k even and q0 = (q10,q
20) ∈ R2 the reachable set q0RT is contained in the
half plane {q ∈ R2 : q2 ≥ q20}. In terms of iterated integrals it is the even power
k in∫ T
0(∫ t
0 u(s)ds)k dt that is responsible. In terms of the vector fields f0 = ∂
∂q1 ,
f1 = (q1)k ∂
∂q2 , one easily computes that (adk f1, f0) = k! ∂
∂q2 . Hence this systemsatisfies the Lie algebra rank condition (41) and hence by theorem 3.1, the reachablesets for every positive time, from any initial condition have nonempty interior in theplane. However, corresponding to the iterated integral
∫ T0(∫ t
0 u(s)ds)k dt, it is the
even number of factors f1 in the (first and only) iterated Lie bracket (adk f1, f0) thatgives the ∂
∂q2 direction.This is the first appearance where the invariance of an iterated integral under an
input symmetry, here u 7→ −u, results in an obstruction to controllability. The de-tailed study of such obstructions to controllability goes back to the 1970s, and wasvery active in the 1980s, see e.g., [39, 76, 77, 80, 82]. There is still no completesolution of this problem, and it requires careful analysis of the combinatorial struc-ture of iterated formal Lie brackets. At this place we state the definitions and majortheorems that suffice for most practical applications. Because of both, its role inconditions for optimal control and the interest in engineering applications to reachnearby states in short time (without long excursions), one of the best studied con-cepts is small-time local controllability.
Definition 3.3 For p∈M, the control system ΣF,U is small-time locally controllable(STLC) from p if for every T > 0, p ∈ int pRT .
Theorem 3.2 (Hermes [38], Sussmann [80]) Suppose m = 1, U = [−1,1], and thesystem ΣF,U is accessible from p. If for all k ≥ 0
∑`≥0
pL(`,2k)( f0, f1)⊆ ∑`≥0
pL(`,2k−1)( f0, f1) (44)
then the system ΣF,U is STLC from p.
Contents 25
In practical terms this means that when successively calculating iterated Lie brack-ets containing an increasing number of factors f1, and evaluating these at the initialpoint p, new directions are only obtained from brackets involving an odd number offactors of the controlled vector field. The rationale is that the corresponding iteratedintegrals contain the control u = u1 an odd number of times, and hence are mappedto their negatives by the action of the input symmetry u 7→ −u. One considers theterms which are lowest order in the control u which for ‖u‖ sufficiently small areexpected to dominate all terms which are higher order in u. For more details see thediscussion on control variations and approximating cones in section 3.5. A corre-sponding necessary condition for STLC was also proven in the mid 1980s.
Theorem 3.3 (Stefani [77]) If m = 1 and U = [−1,1], and the system ΣF,U is STLCabout q0 then for all k ≥ 1
q0(ad2k f0, f1) ∈ ∑`≥0
q0L(`,2k−1)( f0, f1) (45)
Allowing different weights on the time and the magnitude of the control leadsto different ways of bounding some iterated integrals by others. This is formalizedin the following theorem, yielding sufficient conditions for STLC that cover mostapplications of interest.
Theorem 3.4 (Adapted from Sussmann [82]) Suppose U = [−1,1]m, and the systemΣF,U is accessible from q0. If θ ∈ (0,1] and for all k odd, and all `1, . . . `m even,
q0L(k,`1,...`m)( f0, . . . fm)⊆ ∑(ki,`i)
q0L(ki,`i1,...`
im)( f0, . . . fm) (46)
where the sum extends over all (ki, `i) such that
θki + `i1 + . . .+ `i
m < θk+ `1 + . . .+ `m (47)
then the system ΣF,U is STLC from q0.
More general statements allow different weights θi for different vector fields, butthe version above suffices for most applications. The first proofs of theorems of thisform explicitly constructed families of control variations that generated tangent vec-tors to the reachable sets from the initial point. The earliest constructions dependedheavily on analyzing the effects of piecewise constant control variations using toolssuch as for the pullback of a vector field (17) and the Baker-Campbell-Hausdorffformula as illustrated in section 2.2. The resulting coefficients of each iterated Liebracket are polynomial in the parameters of the piecewise constant control varia-tions (amplitude, switching times), compare section 3.5. The tangent vectors ariseas the leading terms in these Taylor series expansions by these parameters.
The next improvement uses the Chen-Fliess series directly and associates iter-ated Lie bracket to iterated integrals of the controls. Highly technical estimates ofthese iterated integrals are key to the proof of the necessary condition theorem 3.3.
26 Contents
However, these calculations can be much simplified once one takes into account theunderlying structure as an exponential Lie series and uses combinatorial bases forfree Lie algebra, compare section 4.
The proof of theorem 3.4 further introduces and exploits the concept of inputsymmetries, such as mapping a control u to its negative −u, or its time reversalu−(t) = u(T − t), or possibly permutations such as (u1,u2) 7→ (u2,u1). Every suchinput symmetry induces action on both the iterated brackets of the vector fields andon the associated iterated integrals. Possible obstructions to controllability are iden-tified with fixed points of these symmetries. However, if not vanishing at the startingpoint, these possible obstructions may be neutralized by lower order brackets, wherethe order of a (formal) bracket is determined by any one of various possible gradedand filtered structures on the Lie algebra of vector fields, and correspondingly on theiterated integrals. To make these informal descriptions precise, one needs to abstractto free Lie algebras, their enveloping Lie algebras and work with formal combina-torial objects, compare section 4. For an authoritative discussion of controllabilitythat goes from the finite dimensional systems considered here all the way to systemsgoverned by partial differential equations see the recent monograph [18].
Revisiting the parallel parking example, everyday experience suggests that thesystem is STLC. Maybe the only question that does not have an evidently affirma-tive answer is whether e.g. sideways motions can be achieved in arbitrarily small-time without large excursions of the state from the reference point. Indeed this isconfirmed analytically. Inspection of the iterated Lie brackets evaluated above con-firms that none are of the type (odd, totally even) which may be possible obstruc-tions to controllability. Thus by theorem 3.4 this system is STLC from the initialpoint q0 = 0. However, from a very large collection of combinatorially possible Liebrackets of the fields fi, the above calculation managed to get by with only con-sidering a few iterated Lie brackets. For practical purposes one wants to have analgorithm that minimizes the number of iterated Lie brackets that need to be calcu-lated. Thus in general one should start with formal brackets that form a basis for afree Lie algebra, compare definition (4.1).
The iterated Lie brackets that need to be considered for the pendubot system arean example for the delicate internal structure of iterated Lie brackets (and of theassociated iterated integral functionals) that may decide controllability. Indeed inthis case the established sufficient conditions that are found in the literature cannotestablish STLC using only a count of the factors. Instead, finer combinatorial studiesand analytic estimates are needed.
In local coordinates in traditional column notation the system vector fields forthe pendubot were given in equation (7). It is immediate that q0 f1 = ∂
∂q1
∣∣∣q0
and
q0[ f0, f1] =− ∂
∂q2
∣∣∣q0
. Calculating increasingly longer iterated brackets, and evaluat-
ing them at q0 does not yield any new linearly independent tangent vectors until onegets to iterated Lie brackets involving five factors. While in general for lower orderbrackets one only needs to count the number of times each vector fields appears asa factor, at this length the internal combinatorial structure of the Lie brackets begins
Contents 27
to play a role. Indeed, in the free Lie algebra L(a,b) over a two element set {a,b}each of the homogeneous subspaces L(2,3)(a,b) and L(3,2)(a,b) has dimension two.Among the many combinatorial choices for the internal structure of the iterated Liebrackets, the following four arise from commonly used Hall sets, see definition 4.1.Two brackets computed from different numbers of copies of the vector fields f0 andf1 both vanish at the reference point,
q0[ f0, [ f0, [[ f0, f1], f1]]]] = q0[ f0, [[ f0, f1], f1], f1]] = 0. (48)
while complementary iterated brackets with the same numbers of copies of f0 andf1 both yield the same new direction at the reference point,
q0[[ f0, [ f0, f1]], [ f0, f1]] = −β∂
∂q3
∣∣∣q0, and
q0[[ f0, f1], [[ f0, f1], f1]] = −2β2 ∂
∂q3
∣∣∣q0
(49)
Many other choices for the bases of the homogeneous subspaces are possible, butthey are related to each other by the Jacobi identity and anticommutativity.
One additional bracket with the drift vector field, corresponding to one additionalintegration with respect to time, yields the final direction
q0[ f0, [[ f0, [ f0, f1]], [ f0, f1]]] = −β∂
∂q4
∣∣∣q0, and
q0[ f0, [[ f0, f1], [[ f0, f1], f1]]] = −2β2 ∂
∂q4
∣∣∣q0
(50)
These two are chosen for simplicity, but they do not arise from evaluating Hall trees.Indeed the dimensions of the corresponding homogeneous subspaces in the free Liealgebra L(3,3)(a,b) and L(4,2)(a,b) are three and two, respectively. The numbers ofcombinatorially not equivalent iterated Lie brackets involving six factors demandthat one makes strategic choices which are based on the underlying combinatorialstructure.
The calculations above prove that the pendubot is accessible from the given ini-tial condition, but theorem 3.4 alone does not allow one to conclude that the systemsis STLC. Using intricate constructions of controls, Hauser [37] proved that the pen-dubot is indeed STLC, provided that the upper bounds on the magnitude of the con-trol is not too small. This appears to be the first nonacademic, at least reasonablyrealistic, example in which the general theorem 3.4 does not allow one to decidecontrollability. Further academic examples were analyzed in [44, 46], including anotion of two possible obstructions to controllability balancing each other. But thisis the first system of practical interest that lies in the still not completely knownterritory between sufficient and necessary conditions for STLC.
28 Contents
3.3 Nilpotent approximating systems
For many control applications such as feedback stabilization and path planning it isdesirable to approximate a system by a simpler model, but one which still capturescritical geometric properties, such as controllability. Such approximating systemsmay, in turn, also facilitate the analysis of the original system – and the proofs oftheorems such as theorem 3.4 intrinsically use nilpotent approximations inside thetechnical work. For highly nonlinear control systems nilpotent approximating sys-tems play a role similar to that of (Jacobian) linearizations of nonlinear dynamicalsystems about their singularities, see e.g. [5, 40, 41, 42, 73, 74, 72, 77].
The latter qualitatively determine the local behavior about singularities, such asstability, except in more degenerate cases. While (Jacobian) linearizations have animportant role in control systems, too, they cannot capture characteristic features ofhighly nonlinear systems, most notably the angular momentum equations of a rigidbody with two controls. Nilpotent approximating systems are the next most simplesystems, and this class is sufficiently rich that approximating systems chosen fromthis class can preserve desirable local properties.
Definition 3.4 A control system of form (3) is called nilpotent if the Lie algebraL( f0, f1, . . . fm) generated by the system vector fields is nilpotent, that is, if there ex-ists an integer N such that every iterated Lie bracket [ fi1 , [ fi2 , [ fi3 , . . . [ fis−1 , fis ] . . .]]])with i j ∈ {0,1, . . .m} and s≥ N vanishes identically.
Nilpotent systems are particularly attractive as solution curves correspondingto controls that are piecewise elementary functions may be computed via simplequadratures only. Compare this to the complexity of the solution curves of evensimple quadratic polynomial systems such as the chaotic Lorenz system.
Theorem 3.5 [47] Suppose that f0, f1 . . . fm are analytic vector fields on a manifoldMn that generate a nilpotent Lie algebra which at a point q0 ∈Mn spans the tangentspace TpM. Then there exists a local coordinate chart about p, such that in thesecoordinates each of the vector fields has polynomial components and the Jacobianmatrices of partial derivatives are strictly lower triangular.
For uncontrolled dynamical systems, whose linearization about an equilibriumdoes not have full rank, many efforts have been devoted to classifying normal formsof the singularities. However, no general procedure is available that identifies theleading part in cases of higher orders of degeneracy. However, for affine controlsystems of form (3) that satisfy the sufficient conditions for STLC in theorem 3.4, asimple algorithmic procedure allows one to construct a nilpotent approximating sys-tem (on the same space) that preserves controllability. Normal forms are addressedin detail in section 4.4.
Moreover, feedback laws that are homogeneous in a well-defined sense, andwhich asymptotically stabilize the approximating system, also locally asymptoti-cally stabilize the original system, compare e.g. [41]. This generalizes Lyapunov’sindirect method and supports the claim that nilpotent approximating system are thenatural generalization of (Jacobian) linearizations.
Contents 29
The following algorithm follows the presentation given in [40], see also [41,77] for more details about adapted coordinate charts. For a recent more abstractalgebraic description see [74, 72].
Since vector fields fi on a manifold may satisfy relationships such as [ f0, f1] = f0,it is generally not meaningful to talk about the length of a Lie bracket of vectorfields, or its degree of homogeneity. Thus technically the following algorithm needsto be defined in terms of the free magma M (Z) that is the set of formal iteratedbrackets, or equivalently, rooted binary trees labeled by the alphabet Z. See defini-tion (78) and section 4.1 for the technical details.
Algorithm for constructing nilpotent approximations. Suppose the vectorfields f0, f1, . . . fm are analytic and define a control system of form (3) on Mn = Rn
that is accessible from q0 = 0, i.e., it satisfies the Lie algebra rank condition (the-orem 3.1), and q0 f0 = 0. For simplicity of exposition here consider only the casewhen all controlled vector fields fi, i≥ 1 are assigned the same weights θi = 1.
1. Choose a weight θ = θ0 ∈ (0,1] for the drift vector field f0. (In general, theweight θ = 0 yields solvable, not necessarily nilpotent approximating systems.)Let δ θ : M ({0,1, . . .m}) 7→R+ be the homomorphism defined on generators byδ θ (0) = θ and δ θ (i) = 1 if i≥ 1, i.e., for formal brackets `,`′ ∈M ({0,1, . . .m}),δ θ ([`,`′]) = δ θ (`)+δ θ (`′).
2. Choose formal Lie brackets `π1 , `π2 , . . . `πn ∈M (0,1, . . .m) such that
• the corresponding Lie brackets of vector fields fπi = F (`πi), i = 1, . . .n spanthe tangent space Tq0Rn at 0, i.e. {q0 fπ1 ,q0 fπ2 , . . .q0 fπn ,} are linearly inde-pendent, and
• the weights r =(δ θ (`π1), . . . ,δ
θ (`πn))
are minimal among all such n-tuplesof formal Lie brackets (in the lexicographical order).
3. Perform a constant linear coordinate change such that in the new coordinatesq0 fπi =
∂
∂xi
∣∣∣q0
for 1≤ i≤ n.
4. Define the group of dilations {∆s : s > 0} on Rn by ∆s(x) = (sr1x1, . . . ,srnxn).
5. Expand each component 〈dx j, fi〉 for 0 ≤ i ≤ m in a power series f ji = ∑
α
c jiα xα
in the new x-coordinates and truncate it, keeping only terms of order at mostri− r j, i.e. define c j
iα = c jiα if α · r ≤ ri− r j, and c j
iα = 0 otherwise. Define theapproximating vector fields fi = ∑
j∑α
c jiα xα ∂
∂x j .
6. If desired, perform a finite sequence of strictly triangular polynomial coordinatechanges to an adapted chart that homogenizes the approximating vector fields.For details see e.g. [41, 77], and the example below.
Theorem 3.6 [40, 41]) The vector fields f0, f1, . . . fm constructed above generatea nilpotent Lie algebra. They define a control system of form (3) that is STLC if itfollows from theorem 3.4 (with the same weight θ as in the construction above) thatthe original system is STLC.
30 Contents
Moreover, if the fields have been homogenized by choosing an adapted chart as inthe last step, then there is a natural (but coordinate-dependent) way to measure thedistance between trajectories of the original and the approximating systems.
Regarding the last, optional step of further adapting the coordinates, consider thesimple example of f0(x) = x1
∂
∂x2 +(x2x1 + x991 ) ∂
∂x3 and f1(x) = ∂
∂x1 . The term x2x1is not seen by any Lie brackets, and it has no impact on the controllability properties,i.e. the leading term is x99
1 . The simple coordinate change (y1,y2,y3) = (x1,x2,x3−12 x2
2) eliminates this ghost term and renders the same vector fields homogeneous inthe new coordinates f0(y) = y1
∂
∂y2 + y991
∂
∂y3 and f1(y) = ∂
∂y1 .Such ghost terms can be avoided completely by initially defining adapted coordi-
nates via the flows of the selected vector fields fπi . However, in principle that wouldrequire the exact solution of initial value problems for nonlinear differential equa-tions. The above procedure avoids any such use of the implicit function theorem andonly uses constant and triangular polynomial coordinate changes.
We illustrate the algorithm for the example of the pendubot.1. Given the iterated Lie brackets evaluated previously, choose the weight θ = 1
which maximizes the importance of the number of integration, i.e., the role ofthe time variable, and minimizes the importance of the control amplitude.
2. Corresponding to the iterated Lie brackets calculated, compare (49) and (50),that are linearly independent at q0 = 0 select the formal iterated Lie brackets (orbinary trees labeled by the alphabet by Z = {0,1})
`1 = 1`2 = (0,1)`3 = ((0,(0,1)),(0,1))`′3 = ((0,1),((0,1),1)) (51)`4 = (0,((0,(0,1)),(0,1)))`′4 = (0,((0,1),((0,1),1)))
With the choice of the weight θ = 1, either choice of binary tree `i or `′i fori = 3,4 yields the same exponents r = (1,2,5,6) for the group of dilations.
3. In this case no constant linear coordinate change is needed as the selected iteratedbrackets of vector fields are already aligned with the coordinate axes at the origin.For further computations (with a computer algebra system) it is convenient tostraighten out the field f1 so that in the new coordinates f1 ≡ ∂
∂x1is constant.
This is achieved by the triangular coordinate change x1 = q1, x2 = q3, x3 = q3−βq1 sin(q4−q2), x4 = q4).
4. Using the x coordinates define the dilation group ∆s(x) = (sx1,s2x2,s5x3,s6x4).5. Expand each component of the vector fields f0 in the x-coordinates in a Taylor
series, and for each 1 ≤ i ≤ 4 truncate the ith components at terms of orders0, 1, 4, and 5 relative to the dilation.
6. This yields the approximating vector fields
f0 = x1∂
∂x2+(β 2x2
1x2− 12 βx2
2)∂
∂x3+ x3
∂
∂x4and f1 =
∂
∂x1(52)
Contents 31
This nilpotent approximating system, and also the original system will be small-timecontrollable depending on the bound on the control values relative to the physicalscale parameter β (ratio of the length of the arms). Since both the sign-definiteand the sign-indefinite term in the third component of f0 correspond to the samenumber of integrations, controllability is determined by comparing the sizes of thetwo iterated integral functionals∫ T
0
(∫ t1
0
∫ t2
0u(t3)dt3 dt2
)2dt1 and
∫ T
0
(∫ t1
0!!∫ t2
0u(t3)dt3 dt2
)(∫ t1
0u(t2)dt2
)dt1
(53)Such comparison is similar to that carried out in [44, 48] using integration by partswhile imposing the constraints such as∫ T
0
∫ t1
0u(t2)dt2 dt1 =
∫ T
0u(t)dt = 0. (54)
The small-time local controllability of this model of a pendubot was also analyzedby Hauser [37] via direct construction of control variations.
3.4 Optimality and the Maximum Principle
Unlike classical dynamical systems that are characterized by unique solutions toinitial value problems, in controlled dynamical systems if there is one admissibletrajectory that goes from initial point q0 to the final point qT , then generally thereare many others. As throughout mathematics, picking one solution from many maybe a difficult task. A commonly employed strategy is to add additional constraintsor performance criteria so that the new problem has only one solution, or one bestsolution, which may then be computed algorithmically. There are many possiblechoices of cost functionals. Some arise naturally from physical considerations orvariational principles, or are dictated by specific applications. Others may be cho-sen simply because they do the job (singling out a unique trajectory that steers thesystem to a desired equilibrium state) and they are mathematically tractable. Foran introductory discussion it is convenient and customary to consider more generaldynamic constraints of the form
qt = f (t,qt ,ut). (55)
These include the special class of control-affine systems (3) considered in most ofthese notes. On the space U of admissible controls one adds to (55), now consid-ered as a dynamic constraint, a cost functional J : U 7→ R, that is to be minimizedover the space U . The general form of the cost functional in the Bolza problem inclassical calculus variations is
minimize J(u) = ψ(qT )+∫ T
t0L(t,qt ,ut)dt (56)
32 Contents
It is straightforward to rewrite this as an equivalent problem, e.g. by adding addi-tional states, in a form that incorporates the running cost L(qt ,ut) into the termi-nal cost ψ(qt)), or the other way around. These problems are known as Mayerform (L ≡ 0) and Lagrange form (ψ ≡ 0). For the purposes of this introduc-tion it is convenient to add the time and running cost as new states and considerq = (q0,q,qn+1) ∈ R×Mn×R. Add the additional dynamic constraints q0 = 1 andqn+1 =−L(q0,q,u), and the initial conditions (q0,qn+1)(t0) = (t0,0). For simplicitywe relabel the new state q again as q. Numerous technical difficulties that largelydeal with the topology, the analytic and regularity properties of the various function-als, existence of solutions, are treated in the large body of literature on the calculusof variations and optimal control theory.
The main basic tool to characterize optimal control-trajectory pairs is the Pon-tryagin Maximum Principle. It is commonly formulated in terms of a Hamiltonianfunction H : T ∗M×U 7→ R that adjoins the dynamic constraints (55) to the La-grangian in (56) by means of a time-varying Lagrange multiplier p : [t0,T ] 7→ T ∗M
H(q, p,u) = 〈p, f (q,u)〉−L(q,u). (57)
Note that in the Mayer form with the enlarged state as above L ≡ 0. The followingstatement of the maximum principle dispenses with many technical hypotheses forwhich we refer the reader to the literature. The textbook [11] gives a concise state-ment at an introductory level, [17] is a standard reference in the setting of nonsmoothanalysis, whereas [83] illustrates the state of the art at the interface of differentialgeometric tools with those from nonsmooth analysis.
Theorem 3.7 If (ξ ∗,η∗) is an admissible trajectory-control pair that minimizes thecost J, then there exists an absolutely continuous curve λ ∗ : [t0,T ] 7→ T ∗M \ {0}such that is λ ∗ never zero, and that satisfies the following conditions:
(i) For almost every t ∈ [t0,T ] and every u ∈U (pointwise maximization)
H∗(ξ ∗t ,λ∗t )
def= H(ξ ∗t ,λ
∗t ,η
∗t )≥ H(ξ ∗t ,λ
∗t ,u). (58)
(ii) ψ(ξ ∗T )+H(ξ ∗T ,λ∗T ,η
∗T ) = 0 (terminal constraint).
(iii) For almost every t ∈ [t0,T ] the co-state satisfies the adjoint equation
λ∗t =− ∂H∗
∂q (ξ ∗t ,λ∗t ). (59)
(iv) If the final state qT is free in the optimal control problem, then λ ∗T = ∂ψ
∂q (ξ∗T ).
In the sequel we illustrate the connection of the maximum principle with familiesof control variations and with controllability, both of whose analysis rely from toolsfrom combinatorial algebra. In that framework the adjoint equation (59), too, is ofinterest and deserves more attention, but this is beyond the scope of this work.
For a brief pictorial description of the items in, and the ideas behind the max-imum principle consider figure 6. Suppose that (ξ ∗,η∗) is an optimal trajectory-control pair. Assuming that for t ≥ 0 the reachable sets q0Rt are compact, it is easy
Contents 33
(qT ,qn+1T )
qn+1
q = (q1, . . . ,qn)
q0 = t
S
ζ
ξ (T ;ηs)
pt
Fig. 6 Pontryagin Maximum Principle graphically
to see that the minimum must be attained at a boundary point of the reachable setq0RT at time T from q0. Using continuous dependence of the flow associated withthe fixed control η∗ and the invariance of domain principle, it is clear that if a tra-jectory lies in the interior of the reachable set q0Rt at some time t ∈ (t0,T ], thenit must also lie in the interior q0Rτ at all later times τ ≥ t. Hence, the contraposi-tive implies that any trajectory that is optimal on the interval [t0,T ] must lie at eachtime t ∈ [t0,T ] on the boundary of q0Rt .
Now consider a one-parameter family of admissible controls ηs : [t0,T ] 7→U suchthat η0 = η∗. Assuming that these control variations are sufficiently smoothly pa-rameterized so that the resulting endpoints ξ (T ;ηs) define a (not necessarily con-tinuous) curve in q0RT that is differentiable at s = 0, the derivative
dds
∣∣s=0 ξ (T ;ηs)
defines a tangent vector to the reachable set q0RT at the point ξ ∗T . Under suitabletechnical hypotheses, the set of all such variational vectors can be shown to be aconvex cone that approximates the reachable set in a natural way, and which is con-tained in a half-space. Consequently, there exists a separating hyperplane, labeledS in figure 6, and a nonzero co-state vector λ ∗T such that for all variational vectors ζ
as above, the inequality 〈λ ∗T ,ζ 〉 ≤ 0 holds. The adjoint equation (59) transports thisco-state vector back along the reference trajectory ξ ∗(·) to the initial point q0. Themaximization condition encapsulates that at every time t ∈ (t0,T ] any variations of
34 Contents
the reference trajectory ξ ∗(·) must point inwards to the interior of the reachable setat time t.
In practical applications, one forms the control Hamiltonian (57) and maxi-mizes it pointwise (58). This characterizes all candidate optimal controls as func-tions u∗ = u∗(q, p). Using the thus singled out controls u∗, the pair of differen-tial equations (55) and (59), the adjoint equation (for all possible initial states q0and terminal co-states λT ) characterize all candidate optimal trajectories. These in-clude, in particular, all trajectories that lie at all times on the boundary of the funnel⋃0≤t≤T
{t}×q0Rt (for each choice of the initial state q0).
In contrast, the system (55) is controllable about a not necessarily constant ref-erence trajectory ξt if this trajectory lies in the interior of the reachable sets ξ0Rtfor all t > 0. In this case, one expects that suitably defined approximating conesof variational vectors as above do not lie in any half-space, but their positive lin-ear combinations span the whole tangent space. Much of the literature on this topicinvestigates possible notions of families of control variations and associated approx-imating cones, together with high order open mapping theorems that guarantee thatif the approximating cones is not contained in a half-space, then the reference trajec-tory lies in the interior, and hence cannot be optimal. All of these are generalized tosystems whose data are required to only satisfy ever weaker regularity hypotheses.
The next section elaborates the connection between control variations and iter-ated Lie brackets and iterated integrals again in the special case of affine systems de-fined by smooth vector fields, and a constant reference trajectory ξ ∗t ≡ q0. Whereasthe original Pontryagin Maximum Principle is a first order necessary condition foroptimality, the emphasis here is on higher order conditions for optimality and forsmall-time local controllability.
3.5 Control variations, and approximating cones
To best illustrate the connections between approximating cones of variational vec-tors and combinatorial iterated Lie brackets, consider the special case of controlla-bility about (or optimality of) the stationary reference ξ ∗t ≡ q0 for the system (3)with smooth vector fields fi and the control set U = [−1,1]m.
In this case, a very simple notion of variational vectors to the family of reachablesets from q0 suffices to obtain strong theorems on controllability and optimality.Since this discussion is of a local nature, we may assume without loss of generalitythat Mn = Rn, and identify the tangent spaces TqRn with the state-space Rn.
Definition 3.5 For α > 0, a vector ζ ∈ Rn is called a tangent vector of order α
at q0 to the family of reachable sets from q0 if there exists a family of admissiblecontrols ηs : [0,s] 7→U and the endpoints satisfy
ξ (s;ηs) = q0 + sαζ +o(sα) (60)
Contents 35
Let C α be the set of all such tangent vectors and define Cα= {sζ : s > 0, ζ ∈C α}.
Here we use the notation o(sα) for functions r : [0,s0] 7→ Rn for which there existC, s1, β > 0 such that for all 0 < s < s1 the upper bound ‖r(s)‖ ≤Csα+β holds.
This definition is a minor adaption from [45], and it is a very special case of amuch more general notion of variational vectors investigated in e.g. [17, 24]. It iseasy to prove that sets C α from an increasing sequence of truncated cones [45].We present the short proof as it gives insight into the scaling properties of controlvariations, and into their concatenations.
Lemma 3.8 Using the hypothesis on the system and definitions as above,
(i) if τ ∈ [0,1] and ζ ∈ C α then τζ ∈ C α ,(ii) if α ≤ β then C α ⊆ C β , and
(iii) if ζ1, ζ2 ∈ C α and τ ∈ [0,1] then τα ζ1 +(1− τ)α ζ2 ∈ C α .
Proof. Without loss of generality assume q0 = 0. Suppose s ∈ [0,1], ζ ∈ C α andchoose a one-parameter family {ηs}s≥0 of control variations ηs : [0,s] 7→ U suchthat the endpoints satisfy ξ (s;ηs) = sα ζ + o(sα). Define the scaled controls ηs :[0,s] 7→U by
ηs(t) ={
0 if 0≤ t ≤ (1− τ)sηs(t− (1− τ)s) if (1− τ)s < t ≤ s. (61)
Then clearly ξ (s; ηs) = ξ (τs;ητs) = sα(τα ζ )+o(sα) proving (i).Suppose τ, ζ and {ηs}s≥0 are as above and α ≥ β . Without loss of generality, s≤ 1and, using σ = s
α
β , define
ηs(t) ={
0 if 0≤ t ≤ s−σ
ηs(t−σ) if σ − s < t ≤ s (62)
Then ξ (s,ηs) = ξ (σ ,ησ ) = σβ ζ +o(σβ ) = sα ζ +o(sα) proving (ii).
Suppose {η(1)s }s≥0 and {η(2)
s }s≥0 are families of control variations that generate theα-th order tangent vectors ζ1 and ζ2, respectively. For τ ∈ [0,1] define the family{ητ
s }s≥0 : [0,s] 7→U by
ητs (t) =
{η(1)τs (t) if 0≤ t ≤ τs
η(2)1−τs(t− τs) if τs < t ≤ s.
(63)
The conclusion ξ (s,ητs ) = sα(τα ζ1 +(1− τ)α ζ2)+ o(sα) is an immediate conse-
quence of Gronwall’s lemma.
Note that with this very simple definition, even if s 7→ ξ (s,η(1)s ) and s 7→
ξ (s,η(1)s ) are smooth curves, in general, the map s 7→ ξ (s,ητ
s ) as in (iii) need noteven be continuous.
36 Contents
The properties (i) and (ii) of the truncated cones C α strongly depend on ξt ≡ 0being an equilibrium solution. For more general settings, especially nonstationaryreference trajectories, it is a very delicate process to rescale or combine families ofcontrol variations. This applies in particular to higher order conditions for optimal-ity and controllability. Classical work detailing when such combinations are feasibledates back to Krener’s High Order Maximal Principle [53], and the careful condi-tions provided in [52], see also [8]. As shown in [7], in general, meaningful higherorder approximating cones along nonstationary reference trajectories need not evenbe convex.
The value of an approximating cone of variational vectors stems from an asso-ciated open mapping theorem. In general, one distinguishes topological argumentsthat apply in finite dimensions from tools used in infinite dimensional settings, com-pare the discussion of Type L and Type T arguments in [84]. But in this simplesetting it is even possible to give simple constructive proofs of the following [45](which is again a special case of [24]).
Theorem 3.9 Using the hypothesis on the system and definitions as above, if C ′ isa closed convex cone (with vertex 0 ∈ Rn) such that C ′ \ {0} ⊆ int C α for someα < ∞, then there are constants C > 0, T > 0 such that C ′∩B(0,Ctα)⊆ q0Rt forall 0≤ t ≤ T .
Corollary 3.10 Using the hypothesis on the system and definitions as above, ifC m = Rn then there are constants C > 0, T > 0 such that B(0,Ctm) ⊆ q0Rt forall 0≤ t ≤ T .
The corollary gives a lower bound on infinitesimal growth rates of the reach-able sets and a sufficient condition for small-time local controllability. The theoremprovides high order necessary conditions for optimality of a stationary referencetrajectory. In general, statements of this form become stronger if they are combinedwith constructions that yield more variational vectors, and thus larger approximat-ing cones. The remainder of this section is devoted to linking the variational vectorsto iterated Lie brackets and gradings of the associated Hopf algebra.
At an elementary level, the classical calculus of variations derives first order nec-essary conditions for optimality by considering directional derivatives of the func-tional (56). In very general terms, to derive necessary conditions for optimality of anadmissible curve u∗, and one fixes a variation v and then manipulates the condition
dε
∣∣ε=0 J(u∗+ εv) = 0. (64)
to derive the familiar Euler-Lagrange equations. In many settings such as classicalmechanics or finding locally length minimizing curves (geodesics) in classical dif-ferential geometry, these may suffice to single out unique candidates u∗ that satisfythe desired boundary conditions. However, in nonlinear control theory, as in manynonholonomic mechanic system, such first order conditions may be too weak: theyhave too many solutions, most of which are not optimal. In this case one looks for
Contents 37
stronger necessary conditions which may combine higher derivatives and more gen-eral variations than uε = u∗+ εv.
One key innovation of optimal control theory introduces needle variations thatare scaled by time, rather than by amplitude. In their most simple form, for a candi-date reference control η∗ : [t0,T ] 7→U , and a Lebesgue point t1 ∈ (t0,T ) of u∗ andv ∈U , define a family of control variations ηδ for some δ0 > 0 and 0≤ δ ≤ δ0 by
ηδ (t) =
η∗(t) if t0 ≤ t < t1v if t1 ≤ t < t1 +δ
η∗(t) if t1 +δ ≤ t < T.(65)
Differentiating the endpoint map δ 7→ ξ (T ;ηδ ) and evaluating at δ = 0 yields againa tangent vector to the reachable set q0RT at ξ ∗(T,η∗). In some sense, one maythink of these needle variations being associated with the L1 norm in the space ofcontrols as opposed to the L∞ norm associated to the variations of the form η∗+ εvfamiliar from the classical calculus of variations.
In order to get usable approximating cones and associated open mapping princi-ples, one needs to be able to take convex combinations of such control variations.These may be defined by finite sequences of times (t1, t2, . . . tr)∈ (0,T )r and controlvalues (v1,v2, . . .vr) ∈U r. As long as the times ti at which the variations are takenare pairwise distinct, for sufficiently small δ0,i the intervals [ti, ti + δ0,i] on whichdifferent control values are taken will not overlap. Many articles in optimal con-trol theory deal with proving that under increasingly weaker regularity conditionssuch combinations of families of control variations indeed yield the expected con-vex combinations of the associated tangent vectors to the reachable set q0RT and toprove the associated open mapping theorems.
To obtain ever stronger necessary conditions for optimality, or, equivalently,stronger sufficient conditions for small-time local controllability, one may devisemore complex families of control variations, consider higher derivatives, and try tocombine any of the above.
As mentioned before, the technical details very quickly become intricate, andwe refer the interested reader to [8, 52, 53]. One simple issue deals with generat-ing convex combinations of the tangent vectors generated by two different fami-lies of needle variations taken at the same time ti = t j. One’s intuition may sug-gest that one might be able to move the support of one of the variations ever soslightly, e.g. to construct a new family of needle variations that takes the values viand v j on the intervals [ti, ti + δi] and [ti + δi, ti + δi + δ j], respectively. Using con-tinuous dependence (or even smooth dependence for smooth controls), one may betempted to expect that this vanishing shifting of the support produces only higherorder effects in the endpoint map. However, a counterexample constructed in [7]demonstrates that for higher order variations (higher-order derivatives) this neednot be the case even in systems with polynomial vector fields. Indeed even for ar-bitrarily short times, that counterexample has reachable sets one of whose naturalcross sections is best approximating by the union of two half spaces akin to theset {x ∈ R2 : x1 ≥ 0 or x2 ≥ 0}. In some sense this counterexample justifies the
38 Contents
highly technical conditions on admissible control variations that are proposed ine.g. [8, 52, 53].
The remainder of the section considers the case of a stationary reference trajec-tory ξ ∗t ≡ q0 corresponding to an equilibrium point f0(q0) = 0 of the drift vectorfield and trivial control η∗ ≡ 0 in which none of these problems arise. The simplenotion of tangent vectors of definition 3.5 together with the open mapping theorem3.9 rely much on bounds on the growth rates of analytic vector fields near equilibriatogether with the ability to shift a control variation in time without changing the ter-minal points (by prepending an interval with zero control): ξ (T ;η) = ξ (T + τ,ητ)where ητ(t) = 0 for 0≤ t < τ and ητ(t + τ) = η(t) for 0≤ t ≤ T .
To obtain algorithmically computable conditions for controllability and optimal-ity one relates the control variations as above to relations between the iterated Liebrackets of the vector fields fi defining the system. One may construct and combineincreasing numbers of families of needle variations as above so that their lower ef-fects cancel and then relate the higher order effects to iterated Lie brackets. The clas-sical Baker-Campbell-Hausdorff formula (23) as in section 2.3 is clearly not suitedfor such analysis. Instead, one may directly use the iterated integrals in improvedversions of the Chen-Fliess series (27) to design controls variations that generatespecific directions.
6
6
6
-
-
-
T
T
T
t
t
t
u1
u0.5
u0.2
Fig. 7 Needle variations also scaled by amplitude
The first step is to consider the homogeneity and scaling properties of the iteratedfunctionals. For a fixed control η : [0,T ] 7→U , considered as a variation of the zeroreference control η∗t ≡ 0, introduce an (m+1)-parameter family of scaled controlsas follows, compare figure 7. For (δ ,ε) ∈ [0,1]1+m and 0 ≤ t ≤ (1− δ )T define
Contents 39
η iε,δ (t) = 0 and for 1≤ i≤ m and t ∈ [0,T ) define
ηiε,δ (T −δ t) = εiη
i(T − t) (66)
Proposition 3.11 Suppose (k, `) = (k, `1, . . . `m) ∈ (Z+0 )
m+1, and w ∈ A(k,`)(Z), andηε,δ is a (m+1)-parameter family of control variations as above, then
ϒw(ηε,δ )(T ) = ε
`δ
k+|`|ϒ
w(η1,1)(T ) (67)
where ε` = ε`11 · . . . · ε`m
m . Recall that w ∈ A(k,`)(Z) means that w is a word (linearcombination of words each of) which contains k times the letter a0, and `i times theletter ai for 1≤ i≤m. The proof is immediate by induction and using the chain rule(substituting for the time variables of integration).
To generate a single tangent vector ζ to the reachable set q0R(T ), define a suffi-ciently smooth curve s 7→ (ε(s),δ (s)) and evaluate the first nonvanishing derivative
dr
dsr
∣∣∣s=0
ξ (T ;ηε(s),δ (s)). (68)
Using the Chen-Fliess series (27) to expand the endpoint map, the iterated integralsare polynomial in (ε(s),δ (s)). Commonly one employs weighting s 7→ (ε(s),δ (s))in which each component is a nonnegative power of s. This induces a partial order onthe words in Z∗ which descends to a partial order of the iterated integral functionals.A higher order Lie bracket is a tangent vector to the reachable sets if all lowerorder terms (in the induced partial order) either vanish because the iterated integralsvanish by construction of the control variation, or the corresponding iterated Liebracket vanishes at the initial point q0, or because certain linear combinations ofiterated Lie brackets and their associated iterated integrals vanish. The rewriting ofthe higher order partial differential operators in the Chen-Fliess series in terms offirst order differential operators, that is, as iterated Lie brackets, is subject of thenext section.
Some commonly used weightings include the following:
• If s 7→ (ε(s),δ (s))= (s, . . . ,s,1) then the support of the control variations is fixed,and only their amplitude ‖ηε‖∞ goes to zero as s↘ 0. This corresponds to theabove described variations familiar from the classical calculus of variations, andthis is instrumental in the proof of e.g. the Hermes condition in theorem 3.2.
• The choice s 7→ (ε(s),δ (s))= (1, . . . ,1,s) corresponds to the original needle vari-ations, whose amplitude is fixed. As s↘ 0 only the measure of the support of thecontrol variations goes to zero, and hence ‖ηε‖1↘ 0 as s↘ 0. All control vectorfields and the drift vector field are weighted equally. This scheme was recentlyemployed for a necessary and sufficient condition for systems that are homoge-neous in a very narrow sense [4].
• If s 7→ (ε(s),δ (s)) = (s1−θ , . . . ,s1−θ ,sθ ) then for w ∈ A(k,`)(Z) the iterated inte-gral satisfies
ϒw(ηε(s),δ (s)) = s(1−θ)∑
mi=1 `i · sθ(k+∑
mi=1 `i)ϒ
w(η1,1) = sθk+∑mi=1 `iϒ
w(η1,1) (69)
40 Contents
This balancing of the rates at which the amplitude and the support of the controlvariations vanish as s↘ 0 corresponds to the scaling in the version of Sussmann’sgeneral theorem stated in theorem 3.4.
6
6
6
-
-
-
T
T
T
t
t
t
u1
u1/2
u1/4
Fig. 8 More complex family of control variations
Common in all the above constructions and conditions is that they only dependon the finely homogeneous degree of the iterated Lie bracket or iterated integralfunctional. The next section demonstrates how the Chen series may be rewritten interms of first order differential operators only. It will become apparent that the com-binatorial structure of iterated integral functionals and formal iterated Lie bracketsis much more delicate than what can be captured by their finely homogeneous de-grees (the number of times each letter appears in the corresponding word). Indeed,already in [46] an example of a simple polynomial single-input system on R4 waspresented and shown to be small-time locally controllable, but for which none ofthe above weightings and scalings suffice to establish controllability. This exampleuses families of control variations that effectively are switching faster and faster asthe parameter s↘ 0 goes to zero. It was shown in [46] that such strategy is nec-essary: If there is a bound on the number of changes of sign of the antiderivativet 7→
∫ Tt0 u(s)ds of the control, then for sufficiently small time and bound on the mag-
nitude ‖u‖ ≤ u0 of the control, the system is not controllable with such controls.The example is built about the two functionals
Contents 41
u 7→T∫
0
(
t∫0
(
s∫0
u(σ)dσ)3 ds)2dt and u 7→T∫
0
(
t∫0
s∫0
u(σ)dσ ds)7dt (70)
which correspond to the formal iterated Lie brackets (binary trees)
(((b,(b,(b,a)))),(((b,(b,(b,a)))),a)) and((a,b),((a,b),((a,b),((a,b),((a,b),((a,b),((a,b),a))))))) (71)
whose multi-degrees in (a,b) are (6,3) and (8,7), respectively. Whereas the firstiterated integral functional clearly is sign definite, it was shown that by judiciouschoice of the fast switching control variations its order could be made larger thanthat of the second, sign-indefinite integral functional – even though the first corre-sponds to both fewer integrations (lower order in time) and it is of lower order inthe control. However, as s↘ 0, and both the support and the control amplitude goto zero, the latter can be made to dominate the first.
In the context of these lecture notes, this example demonstrates the need to useadvanced combinatorial tools to investigate more sophisticated partial orders on Z∗,or rather, on suitably chosen bases for L(Z) that correspond to meaningful construc-tions of families of control variations, and thereby yield stronger sufficient condi-tions for controllability or equivalently, stronger necessary conditions for optimality.
4 Product expansions and realizations
Solving a differential equation by iteration of the equivalent integral equation as insections 2.4 and 2.5 yields an infinite series expansion of the solution. An alternativeis to iterate the method of variation of parameters and obtain an infinite (directed)product expansion for the solution. Using bases for free Lie algebras such exponen-tial product expansions are better suited as tools for analysis and design in control.A further alternative is to write solutions as the exponential of a Lie series. In thesetting of matrix differential equations the corresponding expansions are known asthe classical Fer [21] and Magnus expansions [60]. In a formal algebraic frameworktheir close relationship was demonstrated in [19], which uses dendriform and pre-Lie algebras to provide a uniform approach to both expansions. This section aims atproviding a bridge between the classical dynamical systems setting and the formalframework of [19], with an emphasis on the mappings and correspondences betweenformal combinatorial objects and geometric and dynamic objects in control.
42 Contents
4.1 Variation of parameters and exponential products
First recall the method of variation of parameters for solving differential equations.A discussion of possible meanings of the objects and symbols will be postponeduntil after the statement of the proposition and the formal outline of a proof.
Proposition 4.1 (Formal variation of parameters) If qt and pt formally satisfyq0 = p0 = 1, qt = qt( ft +gt) and pt = pt ft , and qt = rt pt then rt satisfies
rt = rt
∞
∑k=0
1k! (adk ft ,gt). (72)
Proof. In a completely formal way using only the Leibniz rule and distributivity andassuming that p is invertible, the equations q = q( f +g), p = p f , and q = rp yieldrp( f +g) = rp+ rp f , and hence r = rpgp−1.Using chronological exponential notation, one may write pt = e
∫ t0 fs ds. When differ-
entiated the Ansatz qt = rte∫ t
0 fs ds yields
rte∫ t
0 fs ds( ft +gt) = rte∫ t
0 fs ds + rte∫ t
0 fs ds ft (73)
After canceling two terms, the result is easily solved to yield a differential equationfor the correction term rt . Using the ad-formula (18) (and using that the exponential(the flow of ft ) is invertible) this simplifies to
rt = rte∫ t
0 fs dsgte−∫ t
0 fs ds = rte(ad∫ t
0 fs ds ,·)gt = rt
∞
∑ν=0
1ν! (adν
∫ t0 fs ds, gt). (74)
Solving differential equations by variation of parameters is most familiar in thecontext of time-invariant linear differential equations x = Ax+ b where A ∈ Rn×n
and b∈Rn. The solution above is recovered by interpreting ft←→ Ax, and gt←→ b(which may be time varying), yielding x(t) = etAx0+
∫ t0 e(t−s)Ab(s)ds. Analogous is
the case of time-varying linear systems x = A(t)x+B(t)x where B(t) is frequentlyinterpreted as a perturbation of A(t).
In the setting of affine control systems (3), of particular interest is the case whenthe right hand side splits into time varying controls and fixed geometric vector fieldsft ↔ u1
t f1 and gt ↔ u2t f2. The previous linear case is included by combining the
components in the form
A(t)x←→n
∑i, j=1
aij(t)x
j ∂
∂xi and b(t)←→n
∑j=1
b j(t) ∂
∂x j . (75)
The main difference between the Magnus and Fer expansions and the formulas pre-ferred in control is that in the former there are n2 many unrelated terms ai
j(t) playingthe role of the inputs, whereas in control there are typically much fewer controls and
Contents 43
states, i.e. the main interest is in matrices A(t) =m∑
i=1ui(t)Ai that have a very rigid
structure with m << n2.
Further abstracting the setting to the algebra A(Z) of formal power series over thealphabet Z, the solution qt is a curve of group-like elements qt ∈ A(Z) characterizedin terms of the copoduct by ∆qt = qt ⊗qt . The vector fields ft and gt correspond togenerators a ∈ Z, or more generally Lie elements `t ∈ L(Z) characterized by ∆`t =`t ⊗1+1⊗ `t . In the case of general time-varying vector fields ft as in Agrachev’schronological calculus the exponentials also become directed as in general fs ft 6=ft fs, i.e., the fields do not commute with themselves at different times.
In the case that ft = u1t f and ft = u2
t g split into time-dependent scalar controlsand constant geometric vector fields, the differential equation (74) for the correctionterm also splits
rt = rt∞
∑ν=0
1ν!
(∫ t0 u1
s ds)
νu2t (adν f , g). (76)
The corresponding formal analogue identifies qt , ft , pt with elements in A(Z) or inA(Z)⊗ A(Z). E.g., for a∈ Z, the vector a⊗a is mapped to the product ua
t fa, whereasqt corresponds to the series (ϒ ⊗ Id)(CF)(ut) = ∑
w∈Z∗ϒ w(ut)⊗w of the control ut .
The variation of parameters has resulted in the new differential equation (76)which is of the same form as the original equation qt = qt( ft +gt). Its right hand sideis again a linear combination of constant vector fields with time varying coefficients.Hence one may iterate this procedure using the Ansatz rk
t = rk+1t pk
t with r0t = qt . This
yields the expansion qt = q0rkt pk−1
t · · · p1t where each pi
t is of the form of a knownsimple exponential
qt = rst · e
∫ t0 uhk (s)ds fhk · · · · e
∫ t0 uh2 (s)ds fh2 · e
∫ t0 uh1 (s)ds fh1 (77)
The vector fields fh j are selected from the differential equations for the j-th correc-tion term r j
t and the new controls uhi are calculated from the controls at the previousstage according to the formula (76). At first sight it may appear that at every stageone replaces one differential equation by one whose right hand side has infinitelymany more terms each, and each of these is of higher complexity. But a clever con-struction assures this process converges when continued ad infinitum. The objectiveis to obtain simple recursive formulas for the new vector fields and new controls(iterated integrals) and to prove convergence. The resulting formulas were first pre-sented by Schutzenberger in [69] and then rediscovered in various different forms.This presentation follows the proof given by Sussmann [81], but suppresses most ofthe differential equations notation and emphasizes a more combinatorial symbolism.For the construction to work it is critical to work with formal iterated brackets, orequivalently, rooted binary trees labeled by the alphabet Z. Technically they are ele-ments of the free magma M (Z) [10] which is defined recursively by M 0(Z) = {e}(the set containing the empty word or empty tree), and
44 Contents
M 1(Z) = Z, M k(Z) =k⋃
j=0
M k− j(Z) ×M j(Z), and M (Z) =∞⋃
k=0
M k(Z). (78)
For two trees τ, τ ′ ∈M (Z) denote their noncommutative, nonassociative productby (τ, τ ′), and write (τ,e) = τ = (e, τ). It is convenient to also recursively introducenotation for iterated left products λ 0
τ(τ ′) = τ ′ and λ
k+1τ
(τ ′) = (τ,λ kτ(τ ′)).
Denote the natural deparenthesation map which sends each tree to its foliage bydropping the tilde: e.g. for a tree τ = (τ ′, τ ′′) ∈M (Z) denote by τ = τ ′τ ′′ ∈ Z∗
its image under the foliage map. (This notation allows that τ 6= τ ′ while τ = τ ′
– but this will not be a problem.) Similarly, denote the natural map which sendseach binary tree λτ(τ
′) = (τ, τ ′) to the commutator (ad [τ], [τ ′]) = [[τ], [τ ′]] ∈ L(Z)by using unary brackets. This includes the natural identification of a letter a ∈ Zwith the Lie polynomial [a] = a ∈ L(Z)⊆ A(Z) and the rooted labeled tree consist-ing of a single leaf (which is labeled by a). Extend both maps algebra homomor-phisms from the R vector space spanned by M (Z) to the algebra A(Z), and theLie algebra L(Z) ⊆ A(Z). Note that, for example, if a 6= b ∈ Z, then the two treesτ = (b,(a,(a,b))) and τ ′ = (a,(b,(a,b))) are distinct, but because of the Jacobiidentity their images in the Lie algebra L({a,b})⊆ A({a,b}) are identical
[τ] = [b, [a, [a,b]]] = bbaa−2baba+2abab−aabb = [a, [b, [a,b]]] = [τ ′]. (79)
Using this formalism, the following iteration yields a converging infinite analogueof (77), and it provides explicit formulas for the exponents. The construction suc-cessively generates two sequences {Ek}∞
k=1,{Fk}∞k=1 ⊆M (Z) of ordered subsets of
labeled binary trees (formal brackets) and recursive formulas for the iterated inte-grals and the iterated Lie brackets that appear in the exponents of the infinite productexpansion.
Start with the empty set E0 = /0 and F1 = Z, both considered as subsets of M (Z).At each stage choose a well-ordering of the set Fk which must satisfy some mildconditions to be discussed later. Select the smallest element hk ∈ Fk and adjoin itto the set Ek to form the new set Ek+1 = Ek ∪{hk}. At each stage use variation ofparameters to formally solve the formal differential equation
rkt = rk
t
(uhk
t [hk]+ ∑B∈Fk
uBs [B]
)(80)
by making the Ansatz
rkt = rk+1
t · e∫ t
0 uhks ds [hk]. (81)
Analogous to the derivation of (76) this determines a new formal differential equa-tion for rk+1
t
rk+1t = rk+1
t ∑B∈Fk+1
uBs [B] (82)
where the set Fk+1 and the new controls uB for B ∈ Fk+1 are of the form
Contents 45
Fk+1 = {λ ν
hk(B) : ν ≥ 0, B ∈ Fk \{hk} }, and (83)
uλ ν
hk(B)
t = 1ν!
(∫ t
0uhk
s ds)ν uB
t . (84)
Upon well ordering the new set Fk+1 and singling out its smallest element hk+1 onthe right hand side of the differential equation (82), the iteration may continue.While at each stage infinitely many new trees are added to Fk to form Fk+1 ⊇ Fk \{hk}, all of the new trees are larger than the one removed from the previous set.Since the original set F0 = Z was finite, for every k, the set Fk+1 \ Fk of newlygenerated trees only contains trees with more leaves than hk, and for each r ∈ Z+,each set Fk+1 contains only a finite number of trees with r leaves. Given the uniformstructure on A(Z) introduced in section 2.5, any well-orderings of each of the sets Fkyields convergence as long as it guarantees that for every k≥ 0 after a finite numberof steps all trees with at most k leaves will have appeared as the smallest elementof some set Fj. A traditional choice is to choose the ordering of each set Fk to becompatible with the length of the trees (number of leaves), i.e., if |τ| < |τ ′| thenτ ≺ τ ′. It is a common (but not required) to choose the ordering such that if τ, τ ′ ∈Fk+1 ∩Fk and τ ≺k τ ′ in Fk then also τ ≺k+1 τ ′ in Fk+1. The formal considerationsabove and convergence in A(Z) translate directly to corresponding results in termsof asymptotic expansions on the analytic side for controlled dynamical systems onsmooth manifolds. For a detailed discussion and a proof that works as much asfeasible on the dynamic systems side see [81]. For analytic details on the variationsof parameters procedure in the case of time-varying vector fields see [3].
4.2 Computations using Zinbiel products
This section establishes basic algebraic properties of the map ϒ from combinato-rial objects to dynamical systems. These allow one to simplify the calculations forPicard iteration presented in section 2.4, and those for iterating the variation of pa-rameters procedure as in section 4.1.
The calculations in the preceding sections demonstrated that in control a ubiqui-tous operation on absolutely continuous functions U,V : [0,T ] 7→ R (think of theseas primitives of controls, i.e., U ′ = u) is the product
(U ?V )(t) =∫ t
0U(s) ·V ′(s)ds. (85)
Analogous, for essentially bounded integrable functions functions u, v : [0,T ] 7→ Rdefine
(u∗ v)(t) =∫ t
0u(s)ds · v(t). (86)
It is an elementary exercise (using integration by parts) to show that both theseproducts satisfy the right Zinbiel identity
46 Contents
r ∗ (s∗ t) = (r ∗ s)∗ t +(s∗ r)∗ t. (87)
The name Zinbiel is Leibniz spelled backwards reflecting the Koszul duality ofthe corresponding algebras, compare [27, 57]. This suggests one to consider onthe combinatorial level an analogous product. Inductively define a binary operation∗ : Z∗×Z+ on words by setting for a ∈ Z and w,z ∈ Z∗
e∗w = w, w∗a = wa, and w∗ (za) = (w∗ z)a+(z∗w)a (88)
and bilinearly extend the product to A(Z)×A+(Z). The empty word is a left iden-tity, but right multiplication by e remains undefined. (For some application it mightis convenient to extend the product by defining w∗e= 0 for w∈ Z+.) It is immediatethat this product on its domain again satisfies the right Zinbiel identity (87). Withthe above defined products, the spaces of absolutely continuous functions, integrablefunctions, iterated integral functionals, and the algebra A(Z) are right Zinbiel alge-bras. Using (28) and (35) it is immediate to verify that the map ϒ is a right Zinbielalgebra homomorphism, e.g., when evaluated on an admissible control u, for wordsw ∈ Z∗ and z ∈ Z+
ϒw∗z(u) =ϒ
w(u)∗ϒz(u). (89)
The symmetrization of the product ∗ on A(Z) yields the familiar shuffle product,e.g. for words w ∈ A+(Z)
wxz = w∗ z+ z∗w (90)
This product is associative and indeed globally defined on all of A(Z)×A(Z), us-ing the convention wxe = exw = w. The corresponding product on integrable orabsolutely continuous functions is pointwise multiplication, e.g. for w, z ∈ Z∗
ϒwxz(u)(t) =ϒ
w(u)(t) ·ϒ z(u)(t). (91)
This may be restated as the fact that the coefficients of the Chen series satisfy theshuffle relations, and consequently the Chen series is an exponential Lie series. Thisis known as Ree’s theorem [66, 67].
To illustrate how much working on the combinatorial level simplifies the calcu-lations and how it provides better insights, revisit the Picard iteration of section 2.4.For a set of admissible controls {ua : a ∈ Z} and their primitives Ua(t) =
∫ t0 ua(s)ds
consider the formal universal control system on A(Z) initialized at S(0) = 1
dSdt
= S(t) ·∑a∈Z
ua(t)a. (92)
Abbreviating U = ∑a∈Z Ua(t)a rewrite this as the equivalent integral equation
S = 1+S∗U. (93)
This may be solved by iteration using the Zinbiel product. The combinatorial ana-logue of Picard iteration yields the Chen-Fliess series written in the form
Contents 47
S = 1+U +(U ∗U)+((U ∗U)∗U)+ . . .=∞
∑k=1
U∗k. (94)
Here, the Zinbiel powers U∗k correspond to successive products taken on the right.Formally, for w∈A+(Z) define w∗1 = λ ∗1(w) =wx1 =w, and inductively for n≥ 1
λ∗(n+1)(w) = w∗λ
∗n(w), w∗(n+1) = w∗n ∗w, and wx(n+1) = w x wxn. (95)
It is straightforward to verify the following identities for w ∈ A+(Z) and n ∈ Z+.
w∗w∗(n−1) = (n−1) ·w∗n, λ∗n(w) = (n−1)! ·w∗n, wxn = n! ·w∗n. (96)
Note that by rewriting a power using a different product one may make factorialsappear or disappear, and thus make e.g. the Taylor expansions of the exponentiallook like a geometric series, the Taylor expansion of the reciprocal 1
1−x , compare[49, 50], and also [19]. This appearance and disappearance of the factorials can alsobe explained in terms of the formal Laplace-Borel transform as described in [56].This is a familiar phenomenon in control engineering.
With these identities the Chen series as obtained by formal Picard iteration at thecombinatorial level, is immediately rewritten as
S =∞
∑k=1
U∗k = 1+U +12!
Ux2 +13!
Ux3 . . .= exp(U) (97)
To further illustrate the simplifications at the combinatorial level, one may jus-tify inverting the universal control system (93) using a product with the formal leftZinbiel inverse
U = λ∗(−1)(S)∗ (S−1) (98)
in terms of left Zinbiel powers, which is a preview of the logarithm of the Chen-Fliess series elaborated in section 4.4. Using the abbreviation S+ = (S−1),
U = S+−S+ ∗S++S+ ∗ (S+ ∗S+)+ . . .= ∑k>0
(−)k+1λ∗k(S+)
= ∑k>0
(−)k+1
k (S−1)xk = log(S−1). (99)
4.3 Exponential products and normal forms for nilpotent systems
As noted in the previous section, since ϒ is an associative homomorphism (91) ofthe shuffle algebra, the logarithm of the abstract Chen series (33) is a Lie series. Inother words, the Chen series may be expressed as the exponential of a Lie series,or as an infinite product of exponentials of Lie polynomials. This section exhibitsthe combinatorial analogue of the corresponding result obtained in section 4.1 that
48 Contents
used the standard dynamical systems technique of variation of parameters to obtainan exponential product expansion of the Chen-Fliess series. First this requires abrief review of bases of free Lie algebra. The section concludes by rewriting theexpansion in a form that immediately gives a coordinate representation of a universalnilpotent system that covers every nilpotent control system.
As a consequence of Ree’s theorem, for every basis B = {[b] : b ∈ B} (suitablyindexed by a set B⊆ Z∗) of the free Lie algebra L(Z) there exist functions ζ ,ξ : B 7→A(Z) such that
CF = ∑w∈Z∗
w⊗w= exp
(∑b∈B
ζb⊗ [b]
)=−→
∏b∈B
exp(
ξb⊗ [b]
)∈A(Z)⊗ A(Z). (100)
In agreement with established language from Lie groups and Lie algebras, refer tothe functions ζ h and ξ h as coordinates of the first and second kind, respectively. Ofparticular importance for applications in the analysis and design of control systemsare explicit formulas for the iterated integral functionals ϒ ζ h
and ϒ ξ hthat are the
images of noncommutative polynomials under the Zinbiel homomorphism (89).Numerous constructions of bases for free Lie algebras have been proposed in the
literature, (compare e.g. [67] for historic remarks, see also [35, 70, 71]). The gener-alization of Hall bases as presented in the unifying work by Viennot [87] has provento be particularly useful for applications in control theory. It is distinguished by be-ing intimately related to the dynamical systems technique of iterating the method ofvariations of parameters as presented in section 4.1. The common underlying algo-rithm is the Lazard elimination [10, 54, 55] (for recent related work see also [58])which is known to generate bases for the free Lie algebras [10]. It is based on thefundamental observation that if L(Z) is the Lie free algebra generated by the set Zand a0 ∈ Z, then L(Z) is the direct sum of the one dimensional subspace spanned bya0 and the Lie algebra that is freely generated by the set
{(adν a0,a) : ν ≥ 0, a ∈ Z \{a0}}. (101)
The construction of bases for free Lie algebras relies on being able to identify leftand right factors, and thus uses the language of rooted labeled binary trees, makinguse of the free magma M (Z) introduced in (78).
Definition 4.1 A Hall-Viennot set over the set Z is a strictly ordered subset H ⊆M (Z) that satisfies:
(i) Z ⊆ H(ii) Suppose a ∈ Z. Then (t,a) ∈ H iff t ∈ H , t ≺ a, and a≺ (t,a).(iii) Suppose t ′, t ′′, t ′′′,(t ′, t ′′) ∈ H .
Then (t ′,(t ′′, t ′′′)) ∈ H iff t ′′ ≤ t ′ � (t ′′, t ′′′) and t ′ ≺ (t ′,(t ′′, t ′′′)).
The bases originally proposed by P. Hall (compare Bourbaki [10]) require thatthe ordering be compatible with the length, i.e. if t ′ and t ′′ are Hall trees, and t ′ hasfewer leaves than t ′′, then t ′ ≺ t ′′. Viennot replaced this condition (and minor otherparts) by condition (ii) in definition 4.1.
Contents 49
The image of a Hall-Viennot set under the map h 7→ [h] is an ordered basis forL(Z) [10, 35, 86]. Hall-Viennot sets have the unique factorization property that insome sense makes these bases optimal [62, 61, 67, 87].
Theorem 4.2 Suppose H ⊆M (Z) is a Hall Viennot set and H = {h : h ∈ H } ⊆Z+ is the corresponding set of Hall words with the induced ordering. Then everyword w ∈ Z+ factors uniquely into a nonincreasing product of Hall words, i. e.,there exist unique r ≥ 0, h j ∈H , such that
w = h1h2 . . .hs and h1 � h2 � . . .� hr (102)
In particular, the restriction of the deparenthesation map M (Z) 7→ Z∗ to any Hall-Viennot set H ⊆M (Z) is injective, justifying the notation [h] for Hall trees h.
This now allows one to obtain concise formulas for the coefficients ξ b in the ex-ponential product expansion (100), which map to the iterated integral functions ϒ ξ b
.Note, that the injectivity of the foliage map (when restricted to a Hall-Viennot set)allows one to index these coefficients by Hall words rather than the more cumber-some Hall trees. Algebraically, these coefficients (up to scaling) form a dual basis tothe Poincare-Birkhoff-Witt basis for A(Z) considered as the universal (associative)enveloping algebra of the free Lie algebra L(Z).
Theorem 4.3 (compare [62, 67, 69, 81]) Suppose H ⊆M (Z) is a Hall-Viennotset, and H ⊆ A(Z) and its canonical image under the deparenthesation map thatsends each binary labeled tree to its foliage. Then the coefficients ξ h for h ∈H inthe exponential product expansion (100) satisfy the following: If hrh′ ∈H is suchthat h is not the left factor in the Hall Viennot factorizations of h′ then
ξhrh′ = 1
r! (ξh)xr ∗ξ
h′ . (103)
For applications in control it is convenient to rewrite the recursive equations (103)in the form of an infinite dimensional analog of the system (3). It is helpful to in-troduce for a ∈ Z the adjoints of the left and right shift maps w 7→ wa and w 7→ awfrom on A(Z) defined by
for all z ∈ Z∗ 〈wa−1,z〉= 〈w,za〉 and 〈a−1w,z〉= 〈w,az〉. (104)
A critical property is that on the Zinbiel algebra the first is a linear map whereas thesecond one is a derivation [50]: For all a ∈ Z and all w,z ∈ Z∗
(w∗ z)a−1 = w∗ (za−1) and a−1(w∗ z) = (a−1w)∗ z+(a−1z)∗w. (105)
Related properties hold for Lie polynomials, and one may define w−1 for generalA(Z) but care needs to be taken to use notation that identifies the map that sendsa word a1a2 . . .ar to the reversed word an . . .a1, which is intimately related to theantipode in the Hopf algebra. This section does not need these extensions from a−1
to w−1.
50 Contents
Theorem 4.4 Fix r≥ 1, m≥ 2 and a Hall-Viennot set H ⊆Z∗ over Z = {a1, . . .am}.Denote by H ≤r ⊆H the set of all words in H that have length strictly less thanr. Assume that the ordering of H is compatible with the length of the words and letN be the cardinality of H <r. For a ∈ Z define the vector fields
fa = ∑h∈H <r
qha−1∂
∂qh (106)
with polynomial components on RN with its standard coordinates indexed by H ≤r.Then the Lie algebra L({ fa1 , . . . fam}) that is free nilpotent of step r.
This theorem effectively gives normal forms in coordinates for affine control sys-tems whose fields generate free nilpotent Lie algebras [50], see also [31]. Togetherwith a theorem from Sussmann[79] that states that the isotropy subalgebra at a pointcompletely locally characterizes the control system, it follows as a corollary that thisconstruction yields normal forms for all nilpotent systems (not necessarily free).
Corollary 4.5 If g1, . . .gm are analytic vector fields that generate a nilpotent Liealgebra. Then there exists r ∈ Z+ and a linear map Φ : RN 7→ Rn such that theimages of the fields fi (both fi and N as in the previous theorem) generate a Liealgebra that is isomorphic to L(g1, . . . ,gn)
The following example illustrates how straightforward it is to recursively gen-erate an explicit version of the normal form of a free nilpotent system with twovector fields. The key is to index coordinate functions (or states) by Hall wordsrather than by natural numbers. The vector fields in the following system of theform q = q(u0 f0 +u1 f1) generate a maximally free nilpotent Lie algebra of rank 5.
q0 = u0
q1 = u1
q01 = q0q1 = q0 u1
q001 = q0q01 = (q0)2 u1 using the factorization (0(01))q101 = q1q01 = q1q0 u1 using the factorization (1(01))q0001 = q0q001 = (q0)3 u1 using the factorization (0(0(01)))q1001 = q1q001 = q1(q0)2 u1 using the factorization (1(0(01)))q1101 = q1q101 = (q1)2q0 u1 using the factorization (1(1(01)))q00001 = q0q001 = (q0)4 u1 using the factorization (0(0(0(01))))q10001 = q1q0001 = q1(q0)3 u1 using the factorization (1(0(0(01))))q11001 = q1q1001 = (q1)2(q0)2 u1 using the factorization (1(1(0(01))))q01001 = q01q001 = q01(q0)3 u1 using the factorization ((01)(0(01)))q01101 = q01q101 = q01(q1)2q0 u1 using the factorization ((01)(1(01))).
(107)
Contents 51
4.4 Logarithm of the Chen series
The exponential product expansion is useful for theoretical analysis of controllabil-ity and optimality properties, and for proving that certain feedback schemes indeedstabilize a system. However, the computational cost of numerically computing eachexponential suggests for other applications to combine all terms in a series and com-pute a single exponential. This applies in particular to path planning and trackingalgorithms, as well as numerical calculations of system trajectories. This sectiondeals with such alternative way to simplify the Chen-Fliess series, by expressing itas single exponential of an infinite Lie series, compare (100).
Again, the existence of coordinates of the first kind ζ b for any basis B of L(Z)is a priori clear. Of practical value are efficiently computable explicit formulas forthese. There is a long history of efforts to derive such continuous Baker-Campbell-Hausdorff formulas, compare e.g. [6, 63, 78] for classical results, and [64] for recentwork in the context of numerical integration. While the latter employs Hall sets, theolder formulas generally do not use bases (only spanning sets). This section demon-strates how to use tools from combinatorial Hopf algebras to efficiently obtain ex-plicit formulas for the ζ h from those for the ξ h for Hall words h. This presentationfollows the approach taken in [25] and [26]. For a detailed discussion of elementsof combinatorial Hopf algebras see the corresponding chapter of this volume [20].This section briefly reviews some maps and properties that are directly used for thecontrol relevant results, and fixes the notation to match the presentation throughoutthis article, while staying close to established symbols [67].
The algebra A(Z) is endowed with two Hopf algebra structures whose productsare the concatenation and the shuffle product, respectively. They share the sameantipode, denoted here by α : A 7→ A, which is the linear map that maps any wordw = a1a2 . . .ak ∈ Zk to its signed reversal
α(w) = (−1)k ak . . .a2a1. (108)
The copoducts will be denoted by ∆ ,∆ ′ : A(Z) 7→ A(Z)⊗A(Z) and are defined ongenerators a ∈ Z by the same formula
∆(a) = ∆′(a) = e⊗a+a⊗ e (109)
They are associative algebra homomorphisms with respect to the concatenation andthe shuffle product, respectively. For example, for letters a,b ∈ Z, one has
∆(ab) = e⊗ab+a⊗b+b⊗a+ab⊗ e and∆′(ab) = e⊗ab+a⊗b +ab⊗ e. (110)
The coproduct ∆ is the transpose of the shuffle product, i.e., for all v,w,z ∈ Z∗,
〈vxw,z〉= 〈v⊗w,∆(z)〉. (111)
52 Contents
A useful characterization of the second copoduct ∆ ′ : A(Z) 7→ A(Z)⊗A(Z) is as theshuffle-algebra homomorphism that satisfies w ∈ Z∗
∆′(w) = ∑
u,v∈Z∗〈w,uv〉 u⊗ v. (112)
Correspondingly, these two Hopf algebra structures endow the algebra of linear en-domorphism of the algebra A(Z) with two different associative convolution productswhich here are denoted ? and ?′. For linear endomorphisms f ,g : A(Z) 7→A(Z), theyare defined by (using conc to denote the concatenation product)
f ?g = conc◦ ( f ⊗g)◦∆ , andf ?′ g = shu◦ ( f ⊗g)◦∆
′, (113)
i. e., for any word w ∈ A(Z)
( f ?g)(w) = ∑u,v∈Z+
〈w,uxv〉 f (u)g(v) and (114)
( f ?′ g)(w) = ∑u,v∈Z+
〈w,uv〉 f (u)xg(v) = ∑uv=w
u,v∈Z+
f (u)xg(v). (115)
The formulas for the ζ h are obtained from those for the ξ h by the adjoint of pro-jections from A(Z) onto the Lie algebra L(Z) along subspaces spanned by symmet-ric products of Lie polynomials. The key is to identify the kernel of the projection.For this consider a composition of the algebra A(Z) as a direct sum of fundamentalsubspaces Uk to be defined in the sequel with associated projection maps.
A(Z) =∞⊕
k=1
Uk and πk : A(Z) 7→Uk. (116)
Note that the projection maps πk are not an orthogonal projection with respect tothe standard inner product on A(Z). The first fundamental subspace is the free Liealgebra U1 = L(Z)⊆A(Z). The higher fundamental subspaces Uk, k > 1 are spannedby symmetric products of k Lie polynomials. Here, the symmetric product of anyfinite sequence p1, . . . pk ∈ A(Z) of polynomials in A(Z) is the multi-linear mapdefined by
Sym(w1, . . .wk) =1k! ∑
σ∈Sk
wσ(1)wσ(2) . . .wσ(k) (117)
where the sum is taken over the symmetric group of order k.As already observed in the formal calculation (99), the natural expansion of the
logarithm as a formal power series involves the projection I : A(Z) 7→ A(Z)+ thatmaps the empty word to zero and that is the identity on Z+. In terms of this map,one has a simple series expansion for the projection π1 : A(Z) 7→U1 in terms of theiterated convolution product.
Contents 53
π1 = ∑k≥1
(−1)k−1 1k I?k. (118)
For practical computations, using (114), one may expand these convolution productsin terms of higher k-ary analogues of the coproduct and concatenation product.
π1 = ∑k≥1
(−1)k−1 1k conck ◦ I⊗k ◦∆
k. (119)
The projections πk : A(Z) 7→ Uk onto the higher order fundamental subspaces arenormalized k-fold convolutions of π1 with itself.
πk =1k! π
?k1 . (120)
From the projections πk on the side that maps to vector fields and higher order partialdifferential operators, one obtains the corresponding projections for the dual space.These map to the iterated integral functionals ζ h. Define the map π ′1 : A(Z) 7→ A(Z)as the adjoint of π1, i.e., the linear map that satisfies for all w,z ∈ Z
〈w,π1(z)〉= 〈π ′1(w),z〉. (121)
Using that I is selfadjoint, and that conc and ∆ ′ are adjoints of each other, as arex and ∆ , the definition (118) of π1 in terms of the convolution product ?, gives thecorresponding formula for π ′1 in terms of the other convolution product ?′.
π′1 = ∑
k≥1(−1)k−1 1
k I?′k. (122)
For practical computations one may expand this formula for words w ∈ Z in theform
π′1(w) = ∑
k≥1
1k (−1)k−1
∑u1...uk=w
u1x . . .xuk.
The most simple nontrivial example is
π′1(ξ
ab) = π′1(ab) = ab− 1
2 axb = 12 (ab−ba). (123)
The main result [25] is that for Hall words h ∈ H the dual projection π ′1 maps{ξ h : h ∈H } elementwise onto {ζ h : h ∈H }.
Theorem 4.6 (Gehrig [25]) For every Hall set H ⊆ Z∗, for all h ∈H
ζh = π
′1 (ξ
h). (124)
Employing function notation this reads as ζ = π ′1 ◦ξ . Using the unique factorizationproperty characteristic for Hall words (theorem 4.2) one may extend the domains ofξ and ζ to all of A(Z) and one obtains explicit formulas for the dual bases forPoincare-Birkhoff-Witt bases of A(Z) when regarded as the enveloping algebra ofL(Z). If w = hr1
1 hr22 · · ·h
rkk is the factorization of the word w ∈ A(Z) with h1 � h2 �
54 Contents
. . . � hk a strictly decreasing sequence of Hall words h j and r j ∈ Z+, then the dualbasis elements are
ξw = 1
r1!···rk! (ξh1)xr1 x(ξ h2)xr2 x . . .x(ξ hk)xrk (125)
In general the maps π ′k do not commute with the shuffle product, but they do onproducts of this special form: As shown in [25], if |r| = r1 + . . .+ rk with r j asabove, then
ζw = π
′|r|(ξ
w) = 1r1!···rk! (ζ
h1)xr1 x . . .x(ζ hk)xrk . (126)
With the characterization (121) of π ′1 and formulas for it, it now is a straightfor-ward calculation to obtain explicit formulas for the ζ h, compare [26]. As an examplefirst consider the projection of an individual word
π′1(aaabb) = aaabb− 1
2 (aaabxb+aaaxbb+aaxaab+axaabb)
+ 13 (aaaxbxb+aaxabxb+2aaxaxbb+axaabxb+axaxabb)
− 14 (aaxaxbxb+axaaxbxb+axaxabxb+axaxaxbb)
+ 15 axaxaxbxb.
For the projection of a polynomial ξ abaab use ξ abaab = 12 (abxaxa)b = 3aaabb+
2aabab+abaab, and then calculate ζ abaab
ζabaab = π
′1(ξ
abaab) = π′1(3aaabb+2aabab+abaab)
= 110 aaabb+ 1
15 aabab+ 115 aabba+ 1
15 abaab− 110 ababa (127)
+ 115 abbaa− 1
10 baaab+ 115 baaba+ 1
15 babaa− 110 bbaaa.
The following system explicitly gives the polynomials ζ h for the first 14 Hall words(compare [49, 26].
Contents 55
ζa = a
ζb = b
ζab = 1
2 (ab−ba) = 12 [a,b]
ζaab = 1
6 ( aab−2aba+baa) = 16 [a, [a,b]]
ζaab = 1
6 (−abb+2bab−bba) = 16 [b, [a,b]]
ζaaab = 1
6 (abaa−aaba)
ζbaab = 1
6 (abab−aabb+bbaa−baba)
ζbbab = 1
6 (bbab−babb)
ζaaaab =− 1
30 (aaaab+aaaba−4aabaa+abaaa+baaaa) (128)
ζbaaab = 1
30 (−2aaabb+3aabab+3aabba−2abaab−2ababa
+3abbaa−2baaab−2baaba+3babaa−2bbaaa)
ζabaab = 1
30 (−3aaabb+2aabab+2aabba+2abaab−3ababa
+2abbaa−3baaab+2baaba+2babaa−3bbaaa)
ζbbaab = 1
30 (2aabbb−3ababb+2abbab+2abbba−3baabb
+2babab+2babba−3bbaba−3bbaab+2bbbaa)
ζabbab = 1
30 (−aabbb−ababb+4abbab−abbba−baabb
−babab+4babba−bbaba−bbaab−bbbaa)
ζbbbab = 1
30 (abbbb+babbb−4bbabb+bbbab+bbbba)
The above system expresses the ζ h for h∈H as linear combinations of the standardbasis vectors w ∈ Z∗. For control applications, and, potentially, for numerical inte-gration algorithms it is more useful to rewrite these formulas in a recursive formatusing the Zinbiel product, i.e. as a cascade nilpotent control system. The followingmay again be seen as an alternate normal form for a free nilpotent control system.
56 Contents
za=ua
zb=ub
zab=− 16 zbua + 1
6 zaub
zaab=(− 1
2 zab− 112 zazb
)ua + 1
12 (za)2ub
zbab=− 112 (z
b)2ua +(− 1
2 zab + 112 zazb
)ub
zaaab=(− 1
2 zaab− 112 zabza
)ua
zbaab=(− 1
2 zbab− 112 zabzb
)ua−
(12 zaab + 1
12 zabza)
ub
zbbab=(− 1
2 zbab− 112 zabzb
)ub
zaaaab=(− 1
2 zaaab− 112 zazaab− 1
720 (za)3zb
)ua− 1
720 (za)4ub
zbbbab= 1720 (z
b)4ua−(
12 zbbab + 1
12 zbzbab + 1720 za(zb)3
)ub (129)
zbaaab=(
1240 (z
a)2(zb)2− 12 zbaab− 1
12 zbzaab− 112 zazbab
)ua
−(
12 zaaab + 1
12 zazaab + 1240 (z
a)3zb)
ub
zabaab=(− 1
2 zbaab + 112 zbzaab− 1
12 zazbab− 112 (z
ab)2 + 1360 (z
a)2(zb)2)
ua
+(− 1
6 zazaab− 1360 (z
a)3zb)
ub
zabbab=(− 1
2 zbbab + 112 zbzbab + 1
720 za(zb)3)
ua
−(
16 zazbab + 1
12 (zab)2 + 1
720 (za)2(zb)2
)ub
zbbaab=(− 1
2 zbbab− 112 zbzbab + 1
240 za(zb)3)
ua
−(
12 zbaab + 1
12 zbzaab + 112 zazbab + 1
240 (za)2(za)2
)ub
This system (129) for the ζ h does not have the elegant algebraic form of the sys-tem (107), which immediately lends itself to studying geometric features, control-lability and optimality properties. However, these formulas for coefficients are ex-plicit and easily computable by machine and well-suited for direct implementationfor solving problems of tracking, path planning, and for geometric numerical inte-gration.
Acknowledgments
The author would like the anonymous referees for their very careful reading of theoriginal manuscript and for many valuable suggestions to improve it.
References 57
References
1. A. A. AGRACEV AND R. V. GAMKRELIDZE, Exponential representation of flows and achronological enumeration, Mat. Sb. (N.S.), 107(149) (1978), pp. 467–532, 639.
2. , Chronological algebras and nonstationary vector fields, in Problems in geometry,Vol. 11 (Russian), Akad. Nauk SSSR, Vsesoyuz. Inst. Nauchn. i Tekhn. Informatsii, Moscow,1980, pp. 135–176, 243.
3. A. A. AGRACHEV AND Y. L. SACHKOV, Control theory from the geometric viewpoint, vol. 87of Encyclopaedia of Mathematical Sciences, Springer-Verlag, Berlin, 2004. Control Theoryand Optimization, II.
4. C. O. AGUILAR AND A. D. LEWIS, Small-time local controllability of homogeneous systems,SIAM J. Control Optim., ((to appear)).
5. A. BELLAICHE, The tangent space in sub-Riemannian geometry, in Sub-Riemannian geome-try, vol. 144 of Progr. Math., Birkhauser, Basel, 1996, pp. 1–78.
6. I. BIALYNICKI-BIRULA, B. MIELNIK, AND J. F. PLEBANSKI, Explicit solution of the contin-uous baker-campbell-hausdorff problem and a new expression for the phase operator, Annalsof Physics, 51 (1969), pp. 187 – 200.
7. R.-M. BIANCHINI AND M. KAWSKI, Needle variations that cannot be summed, SIAM J.Control Optim., 42 (2003), pp. 218–238 (electronic).
8. R. M. BIANCHINI AND G. STEFANI, Normal local controllability of order one, Internat. J.Control, 39 (1984), pp. 701–714.
9. A. M. BLOCH, Nonholonomic mechanics and control, vol. 24 of Interdisciplinary AppliedMathematics, Springer-Verlag, New York, 2003. With the collaboration of J. Baillieul, P.Crouch and J. Marsden, With scientific input from P. S. Krishnaprasad, R. M. Murray and D.Zenkov, Systems and Control.
10. N. BOURBAKI, Lie groups and Lie algebras. Chapters 1–3, Elements of Mathematics(Berlin), Springer-Verlag, Berlin, 1998. Translated from the French, Reprint of the 1989 En-glish translation.
11. A. BRESSAN AND B. PICCOLI, Introduction to the mathematical theory of control, vol. 2 ofAIMS Series on Applied Mathematics, American Institute of Mathematical Sciences (AIMS),Springfield, MO, 2007.
12. F. BULLO AND A. D. LEWIS, Geometric control of mechanical systems, vol. 49 of Texts inApplied Mathematics, Springer-Verlag, New York, 2005. Modeling, analysis, and design forsimple mechanical control systems.
13. J. E. CAMPBELL, On a law of combination of operators bearing on the theory of continuoustransformation groups, Proc. London Math. Soc., 28 (1897), pp. 381–390.
14. K.-T. CHEN, Iterated integrals and exponential homomorphisms, Proc. London Math. Soc.(3), 4 (1954), pp. 502–512.
15. , Integration of paths, geometric invariants and a generalized Baker-Hausdorff formula,Ann. of Math. (2), 65 (1957), pp. 163–178.
16. , Integration of paths—a faithful representation of paths by non-commutative formalpower series, Trans. Amer. Math. Soc., 89 (1958), pp. 395–407.
17. F. H. CLARKE, Optimization and nonsmooth analysis, vol. 5 of Classics in Applied Mathe-matics, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, second ed.,1990.
18. J.-M. CORON, Control and nonlinearity, vol. 136 of Mathematical Surveys and Monographs,American Mathematical Society, Providence, RI, 2007.
19. K. EBRAHIMI-FARD AND D. MANCHON, A Magnus- and Fer-type formula in dendriformalgebras, Found. Comput. Math., 9 (2009), pp. 295–316.
20. K. EBRAHIMI-FARD, F. PATRAS, D. M. DE DIEGO, AND E. MARTINEZ, this volume,(2013).
21. F. FER, Resolution de l’equation matricielle dU/dt = pU par produit infini d’exponentiellesmatricielles, Acad. Roy. Belg. Bull. Cl. Sci. (5), 44 (1958), pp. 818–829.
58 Contents
22. M. FLIESS, Realizations of nonlinear systems and abstract transitive Lie algebras, Bull.Amer. Math. Soc. (N.S.), 2 (1980), pp. 444–446.
23. , Fonctionnelles causales non lineaires et indeterminees non commutatives, Bull. Soc.Math. France, 109 (1981), pp. 3–40.
24. H. FRANKOWSKA, Theoreme d’application ouverte pour les correspondances, C. R. Acad.Sci. Paris Ser. I Math., 302 (1986), pp. 559–562.
25. E. GEHRIG, Hopf algebras, projections, and coordinates of the first kind in control theory,PhD thesis, Arizona State University, 2007.
26. E. GEHRIG AND M. KAWSKI, A hopf-algebraic formula for compositions of noncommutingflows, in Proceedings IEEE Conference Decision and Cointrol, Cancun, 2008, pp. 156–1574.
27. V. GINZBURG AND M. KAPRANOV, Koszul duality for operads, Duke Math. J., 76 (1994),pp. 203–272.
28. W. S. GRAY AND L. A. DUFFAUT ESPINOSA, A Faa di Bruno Hopf algebra for a group ofFliess operators with applications to feedback, Systems Control Lett., 60 (2011), pp. 441–449.
29. W. S. GRAY AND Y. LI, Generating series for interconnected analytic nonlinear systems,SIAM J. Control Optim., 44 (2005), pp. 646–672.
30. W. S. GRAY AND Y. WANG, Fliess operators on Lp spaces: convergence and continuity,Systems Control Lett., 46 (2002), pp. 67–74.
31. M. GRAYSON AND R. GROSSMAN, Models for free nilpotent Lie algebras, J. Algebra, 135(1990), pp. 177–191.
32. R. GROSSMAN AND R. G. LARSON, Hopf-algebraic structure of families of trees, J. Algebra,126 (1989), pp. 184–210.
33. , The realization of input-output maps using bialgebras, Forum Math., 4 (1992),pp. 109–121.
34. R. L. GROSSMAN AND R. G. LARSON, Bialgebras and realizations, in Hopf algebras,vol. 237 of Lecture Notes in Pure and Appl. Math., Dekker, New York, 2004, pp. 157–166.
35. M. HALL, JR., A basis for free Lie rings and higher commutators in free groups, Proc. Amer.Math. Soc., 1 (1950), pp. 575–581.
36. B. HAMBLY AND T. LYONS, Uniqueness for the signature of a path of bounded variation andthe reduced path group, Ann. of Math. (2), 171 (2010), pp. 109–167.
37. J. HAUSER, On the controllability of the pendubot, 8th IFAC Symposium on Nonlinear Con-trol Systems, (2010).
38. H. G. HERMES, Controlled stability, Ann. Mat. Pura Appl. (4), 114 (1977), pp. 103–119.39. , On local controllability, SIAM J. Control Optim., 20 (1982), pp. 211–220.40. , Nilpotent approximations of control systems and distributions, SIAM J. Control Op-
tim., 24 (1986), pp. 731–736.41. , Nilpotent and high-order approximations of vector field systems, SIAM Rev., 33
(1991), pp. 238–264.42. F. JEAN, Uniform estimation of sub-Riemannian balls, J. Dynam. Control Systems, 7 (2001),
pp. 473–500.43. V. JURDJEVIC, Geometric control theory, vol. 52 of Cambridge Studies in Advanced Mathe-
matics, Cambridge University Press, Cambridge, 1997.44. M. KAWSKI, A necessary condition for local controllability, in Differential geometry: the in-
terface between pure and applied mathematics (San Antonio, Tex., 1986), vol. 68 of Contemp.Math., Amer. Math. Soc., Providence, RI, 1987, pp. 143–155.
45. , An angular open mapping theorem, in Analysis and optimization of systems (Antibes,1988), vol. 111 of Lecture Notes in Control and Inform. Sci., Springer, Berlin, 1988, pp. 361–371.
46. , Control variations with an increasing number of switchings, Bull. Amer. Math. Soc.(N.S.), 18 (1988), pp. 149–152.
47. , Nilpotent Lie algebras of vectorfields, J. Reine Angew. Math., 388 (1988), pp. 1–17.48. , High-order small-time local controllability, in Nonlinear controllability and optimal
control, vol. 133 of Monogr. Textbooks Pure Appl. Math., Dekker, New York, 1990, pp. 431–467.
References 59
49. , Calculating the logarithm of the chen-fliess series, in Proceedings MTNS, Perpignan,2000, pp. 111–128.
50. , The combinatorics of nonlinear controllability and noncommuting flows, in Mathe-matical control theory, Part 1, 2 (Trieste, 2001), ICTP Lect. Notes, VIII, Abdus Salam Int.Cent. Theoret. Phys., Trieste, 2002, pp. 223–311 (electronic).
51. M. KAWSKI AND H. J. SUSSMANN, Noncommutative power series and formal Lie-algebraictechniques in nonlinear control theory, in Operators, systems, and linear algebra (Kaiser-slautern, 1997), European Consort. Math. Indust., Teubner, Stuttgart, 1997, pp. 111–128.
52. H.-W. KNOBLOCH, Higher order necessary conditions in optimal control theory, vol. 34 ofLecture Notes in Computer Science, Springer-Verlag, Berlin, 1981.
53. A. J. KRENER, The high order maximal principle and its application to singular extremals,SIAM J. Control Optimization, 15 (1977), pp. 256–293.
54. M. LAZARD, Sur les algebres enveloppantes universelles de certaines algebres de Lie, Publ.Sci. Univ. Alger. Ser. A., 1 (1954), pp. 281–294 (1955).
55. , Groupes, anneaux de lie et probleme de burnside, tech. report, Istituto Matematicodell’ Universia di Roma, 1960.
56. Y. LI AND W. S. GRAY, The formal Laplace-Borel transform of Fliess operators and thecomposition product, Int. J. Math. Math. Sci., (2006), pp. Art. ID 34217, 14.
57. J.-L. LODAY, Une version non commutative des algebres de Lie: les algebres de Leibniz, inR.C.P. 25, Vol. 44 (French) (Strasbourg, 1992), vol. 1993/41 of Prepubl. Inst. Rech. Math.Av., Univ. Louis Pasteur, Strasbourg, 1993, pp. 127–151.
58. J.-G. LUQUE AND J.-Y. THIBON, Noncommutative symmetric functions associated with acode, Lazard elimination, and Witt vectors, Discrete Math. Theor. Comput. Sci., 9 (2007),pp. 59–72 (electronic).
59. T. LYONS AND Z. QIAN, System control and rough paths, Oxford Mathematical Monographs,Oxford University Press, Oxford, 2002. Oxford Science Publications.
60. W. MAGNUS, On the exponential solution of differential equations for a linear operator,Comm. Pure Appl. Math., 7 (1954), pp. 649–673.
61. G. MELANCON, Combinatorics of Hall trees and Hall words, J. Combin. Theory Ser. A, 59(1992), pp. 285–308.
62. G. MELANCON AND C. REUTENAUER, Lyndon words, free algebras and shuffles, Canad. J.Math., 41 (1989), pp. 577–591.
63. B. MIELNIK AND J. PLEBANSKI, Combinatorial approach to Baker-Campbell-Hausdorff ex-ponents, Ann. Inst. H. Poincare Sect. A (N.S.), 12 (1970), pp. 215–254.
64. A. MURUA, The Hopf algebra of rooted trees, free Lie algebras, and Lie series, Found. Com-put. Math., 6 (2006), pp. 387–426.
65. T. NAGANO, Linear differential systems with singularities and an application to transitive Liealgebras, J. Math. Soc. Japan, 18 (1966), pp. 398–404.
66. R. REE, Lie elements and an algebra associated with shuffles, Ann. of Math. (2), 68 (1958),pp. 210–220.
67. C. REUTENAUER, Free Lie algebras, vol. 7 of London Mathematical Society Monographs.New Series, The Clarendon Press Oxford University Press, New York, 1993. Oxford SciencePublications.
68. A. V. SARYCHEV, Lie- and chronologico-algebraic tools for studying stability of time-varyingsystems, Systems Control Lett., 43 (2001), pp. 59–76. Lie theory and its applications in control(Wurzburg, 1999).
69. M.-P. SCHUTZENBERGER, Sur une propriete combinatoire des algebres de lie libres pouvantetre utilisee dans un probleme de mathematiques appliquees,, Seminaire P. Dubreil, Algebreset Theorie des Nombres, (1958).
70. A. I. SIRSOV, On free Lie rings, Mat. Sb. N.S., 45(87) (1958), pp. 113–122.71. , On the bases of a free Lie algebra, Algebra i Logika Sem., 1 (1962), pp. 14–19.72. G. SKLYAR AND S. IGNATOVICH, Fliess series, a generalization of the Ree’s theorem, and an
algebraic approach to a homogeneous approximation problem, Internat. J. Control, 81 (2008),pp. 369–378.
60 Contents
73. G. M. SKLYAR AND S. Y. IGNATOVICH, Representations of control systems in the Fliessalgebra and in the algebra of nonlinear power moments, Systems Control Lett., 47 (2002),pp. 227–235.
74. , Description of all privileged coordinates in the homogeneous approximation problemfor nonlinear control systems, C. R. Math. Acad. Sci. Paris, 344 (2007), pp. 109–114.
75. E. D. SONTAG, Mathematical control theory, vol. 6 of Texts in Applied Mathematics, Springer-Verlag, New York, second ed., 1998. Deterministic finite-dimensional systems.
76. G. STEFANI, Local controllability of nonlinear systems: an example, Systems Control Lett., 6(1985), pp. 123–125.
77. , On the local controllability of a scalar-input control system, in Theory and appli-cations of nonlinear control systems (Stockholm, 1985), North-Holland, Amsterdam, 1986,pp. 167–179.
78. R. S. STRICHARTZ, The Campbell-Baker-Hausdorff-Dynkin formula and solutions of differ-ential equations, J. Funct. Anal., 72 (1987), pp. 320–345.
79. H. J. SUSSMANN, An extension of a theorem of Nagano on transitive Lie algebras, Proc.Amer. Math. Soc., 45 (1974), pp. 349–356.
80. , Lie brackets and local controllability: a sufficient condition for scalar-input systems,SIAM J. Control Optim., 21 (1983), pp. 686–713.
81. , A product expansion for the Chen series, in Theory and applications of nonlinearcontrol systems (Stockholm, 1985), North-Holland, Amsterdam, 1986, pp. 323–335.
82. , A general theorem on local controllability, SIAM J. Control Optim., 25 (1987),pp. 158–194.
83. , New theories of set-valued differentials and new versions of the maximum principleof optimal control theory, in Nonlinear control in the year 2000, Vol. 2 (Paris), vol. 259 ofLecture Notes in Control and Inform. Sci., Springer, London, 2001, pp. 487–526.
84. , Set separation, approximating multicones, and the Lipschitz maximum principle, J.Differential Equations, 243 (2007), pp. 448–488.
85. M. THITSA AND W. S. GRAY, On the radius of convergence of interconnected analytic non-linear input-output systems, SIAM J. Control Optim., 50 (2012), pp. 2786–2813.
86. G. VIENNOT, Une generalisation des ensembles de Hall, C. R. Acad. Sci. Paris Ser. A-B, 276(1973), pp. A599–A602.
87. , Algebres de Lie libres et monoıdes libres, vol. 691 of Lecture Notes in Mathematics,Springer, Berlin, 1978. Bases des algebres de Lie libres et factorisations des monoıdes libres.