backwards reachable set

Hybrid Systems Controller Synthesis Examples

EE291E Tomlin

Backwards Reachable Set

All states for which, for all possible control actions, there is a disturbance action which can drive the

system state into a region G(0) in time t

Backwards Reachable Set

Reachability as game: disturbance attempts to force system into unsafe region, control attempts to stay safe

Reachable Set Propagation

[Mitchell, Bayen, Tomlin 2005]

Theorem [Computing ]:

where is the unique Crandall-Evans-Lions viscosity solution to:

Backwards Reachable Set: Safety

unsafe

Backwards Reachable Set

On boundary, apply control to stay out of red

In red, system may become

unsafe

In blue, system will stay safe

Safety Property can be encoded as a condition on the system’s reachable set of states

Example 1: Aircraft Collision Avoidance

Two identical aircraft at fixed altitude & speed:

‘evader’ (control) ‘pursuer’ (disturbance)

x

y

uv

d

v

Continuous Reachable Set

x

y

Collision Avoidance Filter

Simple demonstration– Pursuer: turn to head toward evader– Evader: turn to head right

pursuer

safety filter’s input modification

pursuer’s inputevader’s desired input

evader

evader’s actual input

unsafe setcollision set

Movies…

Collision Avoidance Control

http://www.cs.ubc.ca/~mitchell/ToolboxLS/

Overapproximating Reachable Sets

[Khrustalev, Varaiya, Kurzhanski]

Overapproximative reachable set:

Exact:

Approximate:

~1 sec on 700MHz Pentium III (vs 4 minutes for exact)

• Polytopic overapproximations for nonlinear games• Subsystem level set functions• “Norm-like” functions with identical strategies to exact

[Hwang, Stipanović, Tomlin]

1 2 3 K

modes

1

2

3

n

itera

tion

s

unsafeunsafe

safe

Computing Reach Sets for Hybrid Systems

Reach Sets: uncontrollable predecessor

1 2 3 K

modes

1

2

3

n

itera

tion

s

uncontrolledtransition unsafe

“safe”

Reach Sets: controllable predecessor

1 2 3 K

modes

1

2

3

n

itera

tion

s

safe

controlled transition

“safe”

Reach Sets: Variational Inequality

1 2 3 K

modes

1

2

3

n

itera

tion

s

States which reach G without hitting E first:

where

subject to

Reach Sets: Iterate

1 2 3 K

modes

1

2

3

n

itera

tion

s

Can separation assurance be automated?

Requires provably safe protocols for aircraft interaction

Must take into account:• Uncertainties in sensed information, in actions of the other vehicle• Potential loss of communication• Intent, or non-intent

unsafe set with choiceto maneuver or not?

Example 2: Protocol design

unsafe set with maneuver

unsafe set without maneuver

?

unsafe

safe

Protocol Safety Analysis• Ability to choose maneuver start time further reduces unsafe set

safe without switchunsafe to switch

safe with switch

unsafe with or without switch

Implementation: a finite automaton• It can be easier to analyze discrete systems than continuous:

use reachable set information to abstract away continuous details

q1

safe at presentwill become unsafe

unsafe to 1

q5

safe at presentalways safesafe to 1

q3

safe at presentwill become unsafe

safe to 1

q4

safe at presentalways safeunsafe to 1

q2

unsafe at presentwill become unsafe

unsafe to 1

qs

SAFE

qu

UNSAFE

forced transitioncontrolled transition (1)

q1

q5

q3

qu

q4 q2

San Francisco Airport750 ft separation

Example 2: Closely Spaced Parallel Approaches

Example 3: Closely Spaced Approaches

evader

EEM Maneuver 1: accelerateEEM Maneuver 2: turn 45 deg, accelerate

EEM Maneuver 3: turn 60 deg

[Rodney Teo]

Sample Trajectories

Segment 1

Segment 2

Segment 3

Dragonfly 3Dragonfly 2

Ground Station

Tested on the Stanford DragonFly UAVs

EEM alert

Sep

arat

ion

dist

anc

e (m

)N

orth

(m

)

East (m)

time (s)

Above threshold

Accelerate and turn EEM

Put video here

Tested at Moffett Federal Airfield

EEM alert

Sep

arat

ion

dist

anc

e (m

)N

orth

(m

)

East (m)

time (s)

Above threshold

Put video here

Coast and turn EEM

Tested at Moffett Federal Airfield

Tested at Edwards Air Force Base

T-33 Cockpit

[DARPA/Boeing SEC Final Demonstration:F-15 (blunderer), T-33 (evader)]

Photo courtesy of Sharon Houck;Tests conducted with Chad Jennings

Implementation: Display design courtesy of

Chad Jennings, Andy Barrows, David Powell

R. Teo’s Blunder Zone is shown by the yellow contour

Red Zone in the green tunnel is the intersection of the BZ with approach path.

The Red Zone corresponds to an assumed 2 second pilot delay. The Yellow Zone corresponds to an 8 second pilot delay

R. Teo’s Blunder Zone is shown by the yellow contour

Red Zone in the green tunnel is the intersection of the BZ with approach path.

The Red Zone corresponds to an assumed 2 second pilot delay. The Yellow Zone corresponds to an 8 second pilot delay

Map View showing a blunder

The BZ calculations are performed in real time (40Hz) so that the contour is updated with each video frame.

Map View with Color Strips

The pilots only need to know which portion of their tunnel is off limits. The color strips are more efficient method of communicating the relevant extent of the Blunder zone

Experimental Platform: STARMAC

The Stanford Testbed of Autonomous Rotorcraft for Multi-Agent Control

Example 4: Collision Avoidance

Pilots instructed to attempt to collide vehicles

Aircraft must stay within safe flight envelope during landing:– Bounds on velocity ( ), flight path angle (), height ( )– Control over engine thrust ( ), angle of attack (), flap settings– Model flap settings as discrete modes – Terms in continuous dynamics depend on flap setting

Example 5: Aircraft Autolander

inertial frame

wind frame

body frame

Autolander: Synthesizing Control

For states at the boundary of the safe set, results of reach-avoid computation determine– What continuous inputs (if any) maintain safety– What discrete jumps (if any) are safe to perform– Level set values and gradients provide all relevant data

Application to Autoland Interface• Controllable flight envelopes for landing and Take Off / Go

Around (TOGA) maneuvers may not be the same• Pilot’s cockpit display may not contain sufficient information to

distinguish whether TOGA can be initiated

flareflaps extendedminimum thrust

rolloutflaps extendedreverse thrust

slow TOGAflaps extended

maximum thrust

TOGAflaps retracted

maximum thrust

flareflaps extendedminimum thrust

rolloutflaps extendedreverse thrust

TOGAflaps retracted

maximum thrust

revised interface

existing interface

controllable flare envelope

controllable TOGA envelopeintersection

Aircraft Simulator Tests• Setup

– Commercial flight simulator, B767 pilot– Digital video of primary flight display

• Maneuver– Go-around at low speed, high descent rate

• Goal– Determine whether problematic behavior predicted by our

model is possible in aircraft flight simulator

Aircraft Simulator Results

Produced unexpected behaviorNon-standard procedure; Unable to duplicate

Validated types of problems addressed by this method

Backwards Reachable Set: Safety

unsafe

Backwards Reachable Set

On boundary, apply control to stay out of red

In red, system may become

unsafe

In blue, system will stay safe

Safety Property can be encoded as a condition on the system’s reachable set of states

Backwards Reachable Set: Capture

desired

Backwards Reachable Set

Capture property can also be encoded as a condition on the system’s reachable set of states

Maneuver sequencing, “Reachavoid”

Target Set

Maneuver sequencing is accomplished by stringing together capture sets, starting from the target set and working backwards

Avoid sets can be combined with capture sets to guarantee safety

Unsafe Set

Example 5: Quadrotor Back-Flip

• Divide flip into three modes• Difficult problem:

– Hitting some target sets while avoiding some unsafe sets

• Solution:– Analyze rotational dynamics and vertical dynamics separately

ImpulseDriftRecovery

Back-flip: Method (1)

Recovery Drift Impulse• Identify target region in

rotational state space for each mode

• Use reachable sets to calculate capture basin for each target– Dynamic game

formulation accounts for worst-case disturbances

• Verify that target of each mode is contained by capture basin of next mode

Back-flip: Method (2)• Identify unsafe region in

vertical state space for final mode

• Use reachable sets to propagate unsafe set for each mode– Dynamic game

formulation accounts for worst-case disturbances

• Verify that control keeps state out of unsafe set

Back-Flip: Results

Back-Flip: Results• Assumptions Validated

• Safety Guaranteed

• Reachability Demonstrated

18 20 22 24 26 28 30 32 34-15

-10

-5

0

5

10

time (seconds)

Pitc

h (

de

gre

es)

Pitch vs Time

Ground

Climb

ImpulseDrift

Recovery

Example 6: Automated aerial refueling

Desired Target Set

Capture Set and Unsafe Set Computation Result

Example 7: Teaming up humans and robots

http://www.goforyourlife.vic.gov.au/hav/articles.nsf/pages/Capture_the_Flag

Multiple playersAdversarial gameLimited InformationMultiple objectives

Quadrotor UAVs

GPS-enabled Smartphones

3G Wireless

Computing

Flag Capture Only

Flag Return Only

Full Game

“Capture the Flag”

Action Support For Human Agents

Undergraduate Team

Scott HoagAndrew Sy

The computed solution can be used to guide and assist human agents.

attacker

defender

Supporting Complex Actions

Reachable sets also assist and enable more complex actions and strategic decision making.

attacker

defender

In this case reachability information helps the attacker mislead the defender to win from a losing initial configuration.

Reachability-Guided UAV Search

UAV

UAVVisibility Attacker

Defender

Attacker Goal

Defender Winning Region

Attacker Winning Region

AttackerVisibility

Possible Defender Locations