autonomous truck supervisory system design overview by ray burns

76
Autonomous Truck Supervisory System Design Overview by Ray Burns

Upload: daisy-newman

Post on 19-Jan-2016

213 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Autonomous Truck Supervisory System Design Overview by Ray Burns

Autonomous Truck Supervisory System

Design Overviewby Ray Burns

Page 2: Autonomous Truck Supervisory System Design Overview by Ray Burns

Supervisory System Functions Destination Selection Route Selection Load & Dump Points Traffic Control & Rules Course Database Autonomous Interaction Manned Vehicle Interaction Fail-safe Operation System-wide Security Central user interface

RouteSystem

GUI

TrafficControl

DISPATCH(Select destination)

(Control all Systems)

Locations(Load / Dump)

Course DataSystem

Course DataSystem

InteractionSystem

(Auto / Manual)

InteractionSystem

(Auto / Manual)

StatusSystem

StatusSystem

Speed Control GPS

Fail-S

afe System

s

Radio Network

CAN Bus

Page 3: Autonomous Truck Supervisory System Design Overview by Ray Burns

Destination Selection

Using DISPATCH® Mine Management System Optimizes overall productivity

Best Path(BP)

LinearProgramming

(LP)

DynamicProgramming

(DP)

EstimatedTravel Times

Path FlowRates

Min

eT

opog

raph

y

Con

stra

ints

Tru

ckS

tatu

s

TruckAssignments

Updated Travel Times

DISPATCH Optimization Algorithms

Page 4: Autonomous Truck Supervisory System Design Overview by Ray Burns

Destination Selection - Example (BP) Estimate travel time

between each pair of locations. (BP Algorithm) Use a combination of

distance, lift up/down, and history.

DiggingWaste

Crusher

Shovel 1

Shovel 2

Shovel 3

Waste Dump

DiggingOre

DiggingOre

3

5

9

11

14

14

13

20

18

Page 5: Autonomous Truck Supervisory System Design Overview by Ray Burns

Destination Selection - Example (LP) Estimate travel time

between each pair of locations. (BP Algorithm)

Compute flow rates for each path. (LP) Use shovel capacities,

material types, travel times, and crusher capacity.

DiggingWaste

Crusher

Shovel 1

Shovel 2

Shovel 3

Waste Dump

DiggingOre

DiggingOre

1000empty

1000empty

500empty/ore

1000waste

1000ore

Page 6: Autonomous Truck Supervisory System Design Overview by Ray Burns

Destination Selection - Example (DP) Estimate travel time

between each pair of locations. (BP Algorithm)

Compute flow rates for each path. (LP)

When a truck finishes at a load/dump site, pick a new destination. (DP) Consider truck location &

capabilities, and attempt to maintain flow rates.

DiggingWaste

Crusher

Shovel 1

Shovel 2

Shovel 3

Waste Dump

DiggingOre

DiggingOre

1000empty

1000empty

500empty/ore

1000waste

1000ore

Just finisheddumping

“T6 finished dumping”“Assign to Shovel 3”

Page 7: Autonomous Truck Supervisory System Design Overview by Ray Burns

Route Selection Given available courses, find

best route from here to there.

“Here” “There”

Page 8: Autonomous Truck Supervisory System Design Overview by Ray Burns

Route Selection - Location concept Location: A logical destination a truck can be assigned to. Can be a point in space, an area, a set of points, etc. Many different kinds (Crusher, Fuel, Parking, Load, Dump) Represented by Java Object

LocationsLocations

Page 9: Autonomous Truck Supervisory System Design Overview by Ray Burns

Route Selection - Location entry points

RouteSystem

TrafficControl

DISPATCH(Select destination)

LocationDatabase

StatusSystem

Assign “T6” to “Bench 3”1

2 34

5

6

7

Where is Truck “T6” now?

Find location “Bench 3”

Get entry pointsfor truck T6

Parked at (x,y,heading)

Bench 3 returned

Bench 3

Bench 2

Bench 1C

rush

er

Dump 3

Dump 1Dump 2

Par

king

are

a

Fuelbay

Entry points are (x,y,heading){cost=5} & (x,y,heading){cost=272}

9

8 Route system computes best path

Extend route for T6 byadding course

10T6 route changed

Page 10: Autonomous Truck Supervisory System Design Overview by Ray Burns

Route Selection - Location types - Crushers

Simple Crusher locations Single entry point Upon arrival, truck dumps load. When done, it is marked “empty” and given a new

assignment. More complex Crusher locations can implement:

Special parking rules Interfaces to Crusher monitoring systems Multiple crushers as single destination

(Crusher is chosen after truck arrives)

Page 11: Autonomous Truck Supervisory System Design Overview by Ray Burns

Route Selection - Location types - Benches

Manually spotted Benches Entry/exit points computed using bounding polygon

Entry pointsExit points

Page 12: Autonomous Truck Supervisory System Design Overview by Ray Burns

Route Selection - Location types - Benches

Manually spotted Benches Entry/exit points computed using bounding polygon Upon arrival, truck switches to manual control

Truck parks at entry point

Page 13: Autonomous Truck Supervisory System Design Overview by Ray Burns

Route Selection - Location types - Benches

Manually spotted Benches Entry/exit points computed using bounding polygon Upon arrival, truck switches to manual control When done:

Bench generates a simple course from truck’s current position to each “safe exit point”

Note: Generated paths must meet some bench-specific criteria to be considered safe.

Page 14: Autonomous Truck Supervisory System Design Overview by Ray Burns

Route Selection - Location types - Benches

Manually spotted Benches Entry/exit points computed using bounding polygon Upon arrival, truck switches to manual control When done:

Bench generates a simple course from truck’s current position to each “safe exit point”

Bench asks DISPATCH to assign loaded truck

Page 15: Autonomous Truck Supervisory System Design Overview by Ray Burns

Route Selection - Location types - Benches

Automatically Spotted Benches Entry/exit points computed using bounding polygon Just before arrival:

Bench decides which shovel to send the truck to Bench generates a course to that shovel Bench extends truck’s route

with generated course

Exit like Manual

Optionally, the Bench may park the truck somewhere to wait for a ready shovel.

Page 16: Autonomous Truck Supervisory System Design Overview by Ray Burns

Route Selection - Location types - Dumps

Dumps and Stockpiles Entry/exit points computed using bounding polygon Just before arrival:

Dump chooses dumping location(Using slot grid or other structure)

Dump generates course to that location* and out Dump extends truck’s route

When done: Mark empty & assign

* Courses may be pre-generated

Page 17: Autonomous Truck Supervisory System Design Overview by Ray Burns

Route Selection - Location types - Fuel / Park

Fuel bays Single point, like simple crusher When person indicates refueling is complete, Fuel bay

requests a new assignment for the truck Parking spots

Single point Truck stays until externally reassigned

Parking areas Auto-generate courses to multiple parking spots Just before truck arrives, choose parking spot

Fuel Bay

Parking Spot

Parking Area

Page 18: Autonomous Truck Supervisory System Design Overview by Ray Burns

Course Data - Course concept Course: A specific continuous smooth trajectory from one point

to another, along with associated data and attributes.

A course consists of A start position and an end position A continuous smooth curve connecting them Course Attributes on various pieces of curve Associated data

Not a course A single position (x,y,heading) A list of positions A curve discontinuous in (x,y) or heading at any point

Portrait of a Course

Page 19: Autonomous Truck Supervisory System Design Overview by Ray Burns

Course Data - Course concept

A piece of cheese

Break it into two pieces

You get two pieces of cheese

Put it back together

The pieces fit perfectly

A course

Break it into two pieces

You get two courses

Put it back together

The pieces fit perfectly

Page 20: Autonomous Truck Supervisory System Design Overview by Ray Burns

Course Data - Course concept

Where can a course be cut? In theory, a course could be cut anywhere In practice, it is limited by the use of integer indices, creating

atomic (indivisible) courses we call “steps” We try to keep the steps less than 10cm long

When can two courses be joined end to end? Any time the (x,y,heading) matches exactly In practice, this means joining either:

Two matching pieces of a previously cut course, or A recorded course with a course generated to match it

Atomic “steps” (indivisible)

1 meter

Yes

Yes

No -- these never match EXACTLY at any point

Page 21: Autonomous Truck Supervisory System Design Overview by Ray Burns

Course Data - Course concept

Trajectory Immutability A course object’s trajectory can never be changed. When you cut a course or join courses you get a new course

object.

Associated data A course may also store Speed & other Safety data Retrieved using custom interfaces (not always supported)

Course Attributes Objects attached to sections of course, used to store:

Human readable name Teaching history (who,when,...) Generation authority

Validation info (who, when, …) Enabling conditions Anything else

Page 22: Autonomous Truck Supervisory System Design Overview by Ray Burns

Course Data - Course Attribute concept Course Attribute: A Java object attached to a course.

Can apply to whole length of course or any part

Attribute “Color(Blue)” applies to this whole course Attribute “Style(Dotted)” applies to first ¾ of this course Attribute “Wide” applies to first and last ¼ of this course

Attributes apply to pieces of course, not points Concept “The attributes at this point” is meaningless Think of each step as having a set of attributes

Attributes are attached to underlying data If you cut a course and apply an attribute to one of the pieces, the

attribute will be applied to that part of the original course.“in common”

Page 23: Autonomous Truck Supervisory System Design Overview by Ray Burns

Course Data - Course Attribute concept

Example: Joe & Tom create courses by recording teaching data:

Recording code adds attributes to each course: TeachingHistory(“Joe”, 1:30pm, …) added to whole length of Course 1 TeachingHistory(“Tom”, 1:45pm, …) added to whole length of Course 2

Susan unconditionally enables both courses: UnconditionalEnable(“Susan”) added to whole length of Course 1 UnconditionalEnable(“Susan”) added to whole length of Course 2

Routing code generates a connecting segment:

Page 24: Autonomous Truck Supervisory System Design Overview by Ray Burns

Course Data - Course Attribute concept

Example, continued: Routing code signs & enables generated course:

GeneratedBy(SwitchPoint,Authority=81232435352) added to Course 3 SwitchSegmentEnable() added to Course 3

When truck “T5” is assigned to “Bench 3”: Route Selector computes best route:

And constructs it:

Route Selector updates the “LastUsed(…)” attribute on the new course:

LastUsed(4:32pm) added to [ join(course1.cut(…),course3,course2.cut(…)) ]

“Best route is: join(course1.cut(…), course3, course2.cut(…))”

Page 25: Autonomous Truck Supervisory System Design Overview by Ray Burns

Course Data - Course Attribute concept

Example, continued: myRoute.getAttributes(TeachingHistory):

TeachingHistory(“Joe”, 1:30pm, …) TeachingHistory(“Tom”, 1:45pm, …)

myRoute.getAttributes(*): TeachingHistory(“Joe”, 1:30pm, …) TeachingHistory(“Tom”, 1:45pm, …) UnconditionalEnable(“Susan”) GeneratedBy(SwitchPoint,Authority=81232435352) SwitchSegmentEnable() LastUsed(4:32pm)

myRoute.cut(...).getAttributes(*): TeachingHistory(“Joe”, 1:30pm, …) UnconditionalEnable(“Susan”) LastUsed(4:32pm)

Page 26: Autonomous Truck Supervisory System Design Overview by Ray Burns

Course Data - Course Attribute concept

Example, continued: course1.getAttributes(*):

TeachingHistory(“Joe”, 1:30pm, …) UnconditionalEnable(“Susan”) LastUsed(4:32pm)

course2.getAttributes(*): TeachingHistory(“Tom”, 1:45pm, …) UnconditionalEnable(“Susan”) LastUsed(4:32pm)

Use Course Cursor to scan course, retrieving trajectory and attributes.

Page 27: Autonomous Truck Supervisory System Design Overview by Ray Burns

Course Data - Mechanisms

How is a course stored? Different courses can be stored in different ways:

Some courses may be stored as a sequence of points Some courses may be stored as B-Spline curves Some courses may be stored as arc and line segments

All of these objects implement the Course interface

CourseInterface

Point SequenceCourse

B-SplineCourse

Arc & LineCourse

Page 28: Autonomous Truck Supervisory System Design Overview by Ray Burns

Course Data - Mechanisms

How is a course sent over the radio? Each class (Points, B-Spline, Arc/Line, etc.) uses it’s own

encoding and compression mechanism The Course Data Sender is responsible for making sure the

embedded devices have all necessary course data: To drive their assigned routes To interact with other trucks

Most messages refer to course by CourseName, a small opaque value identifying the course

Embedded devices can use the CourseName to look up the actual Course object

“Hi, I’d like you to drive course 07 F3 26 5C 82 4A 84 please”

CourseName

Page 29: Autonomous Truck Supervisory System Design Overview by Ray Burns

Route Selection - Switch Points

Need to splice recorded courses together Need to splice in generated courses Automatic splicing

Page 30: Autonomous Truck Supervisory System Design Overview by Ray Burns

Route Selection - Switch Point concept Switch Point: A place where two courses are sufficiently close

that a truck can transfer from one to the other.

We generate two connecting courses for most switch points:

Not all possible switch points are used

Page 31: Autonomous Truck Supervisory System Design Overview by Ray Burns

Route Selection - Best Path calculation

RouteSystem

CourseData

System

Update route

12

3

4

5

6

7

98

An assignment is made byDISPATCH (see other slide)

10

Create Switch Segment Model

Created

Switch Segment Model

Get courses & attributes

ReturnAdd courses

to model

Report new & deletedswitch segments

Best Path Network

StatusSystem

Compute best path to (x,y,h)

Best path is ___Once perentry point

Steps 1-3: At system startupSteps 4-6: As courses changeSteps 7-10: At assignment time

Page 32: Autonomous Truck Supervisory System Design Overview by Ray Burns

Route Selection - Best Path calculation

Use cost based on estimated travel time Rough estimate from “teaching” speed Historical database indexed on switch segment Future: Use congestion info from Traffic Control

Use entrance costs provided by locations

Join selected courses into a new route

The whole route is stored and transmitted as one big long course.

Page 33: Autonomous Truck Supervisory System Design Overview by Ray Burns

Route Selection - Updating the Route

If truck has no route, just add new route If truck already has a route:

Compute first branch point

Tell truck to cut route at branch point and append new course instead.

If truck is near branch point, compute alternate routes: Compute alternate route in case truck misses first branch point. Compute branch point for alternate route Repeat until branch point is “far enough” away.

AlternateBranch Point

Branch Point

Old Route

New Route Alternate

New Route

Page 34: Autonomous Truck Supervisory System Design Overview by Ray Burns

Collision Avoidance systems Autonomous interaction system

Used for interactions with things which move predictably (autonomous trucks, parked trucks)

Permission based Truck can only drive permitted courses

Manual interaction system Used for interactions with things which move unpredictably

(manned trucks, portable safety units)

Position message based Truck must check for potential collisions based on (x,y,h)

Obstacle detection system Used for interactions with all non-radio objects

Page 35: Autonomous Truck Supervisory System Design Overview by Ray Burns

Collision Avoidance - Safety Envelope concept Safety Envelope: A rectangle representing the actual size of

the truck enlarged somewhat to take into account navigation errors, steering errors, calculation errors, and so forth.

Safety Envelope

Nominal Truck Position

Page 36: Autonomous Truck Supervisory System Design Overview by Ray Burns

Autonomous interaction system Used for interactions with things which move predictably:

Autonomous trucks Parked & Locked pickup trucks

Permission based Autonomous truck can only drive permitted courses Parked pickup truck must stay in permitted location

Truck 3 may drive in this area: P832

Page 37: Autonomous Truck Supervisory System Design Overview by Ray Burns

Autonomous interaction - Permission concept Permission: An instruction allowing an autonomous truck to

drive or park in a specific area, as soon as the given conditions are met.

Permissions: Can only be created by central workstation Remain valid until the truck relinquishes them Two units’ active* permissions may never overlap

Redundant fail-safe systems double check all calculations

Permissions include: Permission ID Permitted area Zero or more Conditions

Non-overlapping permissions

* An active permission is one which has all of it’s conditions fulfilled

Page 38: Autonomous Truck Supervisory System Design Overview by Ray Burns

Autonomous interaction - ExampleCentral Truck T3 Truck T7

T3 may drive course ATell T7 when done

P832

A

T7 may drive course B after T3 relinquishes P832P544

B

T3 relinquishing P832T3 relinquishing P832

T7

T3

T3

T7

T3

T7

T7

T3

Page 39: Autonomous Truck Supervisory System Design Overview by Ray Burns

Autonomous interaction - Permissions

A truck may have several active permissions Permissions are additive

A truck must always have active permissions covering: It’s current position It’s entire “worst case stopping distance” along the

course, given it’s current speed

If necessary, slow down

Worst case stopping distance

Required permission area

Page 40: Autonomous Truck Supervisory System Design Overview by Ray Burns

Autonomous interaction - Permission area Options for describing permitted area:

Course with standard sliding safety envelope

Course with custom sliding safety envelope

Polygon or B-spline

Can combine using set Union, Intersection, Difference

Page 41: Autonomous Truck Supervisory System Design Overview by Ray Burns

Autonomous interaction - Permission area

Need to be careful to slide safety envelope properly

*Diagram is slightly exaggerated to make it easier to see the problem

Page 42: Autonomous Truck Supervisory System Design Overview by Ray Burns

In the future assign adjacent permissions on a course so they don’t overlap*:

For now we assign them like this:

This requires us to use mostly conditional permissions, resulting in extra radio traffic

Autonomous interaction - Overlaps

* Using {env(A), env(B)-env(A), ...}

Page 43: Autonomous Truck Supervisory System Design Overview by Ray Burns

Autonomous interaction - Partial relinquish

Can relinquish part of permitted area Used for smooth following Used for narrow roads

Relinquished area is subtracted from permission

Relinquished area can be any shape

Original permission area Retained permission areaRelinquished area

Page 44: Autonomous Truck Supervisory System Design Overview by Ray Burns

Autonomous interaction - Partial relinquishTruck T4 Truck T9

I relinquish portion “A{0,45}” of P832

I relinquish portion “A{0,86}” of P832

I relinquish portion “A{0,124}” of P832

I relinquish portion “A{0,171}” of P832

Only T4’s permission is shown

T9T4

T9T4

T9T4

T9

T4

T9

T4

Page 45: Autonomous Truck Supervisory System Design Overview by Ray Burns

Manual interaction system Used for interactions with things which move unpredictably:

Pickup trucks Shovels Loaders Manned haul trucks Autonomous haul trucks which aren’t “locked” into autonomous mode Men carrying portable safety units

Position message based Truck must check for potential collisions based on (x,y,h)

Except when parked & locked

At 7:52:19am, pickup 3 is at (284,982,27°), speed 17kph

I need to slow down

Page 46: Autonomous Truck Supervisory System Design Overview by Ray Burns

Manual interaction - Position Message Concept Position Message: A message giving a vehicle’s (x,y,heading)

and speed at a specific time.

Position messages contain: Source id Time stamp Position (x,y,heading) Speed

Sent between vehicles, also to the central computer

Sent more frequently when sender & receiver nearby

At 7:52:19am, pickup 3 is at (284,982,27°), speed 17kph

Page 47: Autonomous Truck Supervisory System Design Overview by Ray Burns

Manual interaction - Vicinity List Concept Vicinity List: A complete list of all unpredictably moving

vehicles near a given truck, meaning that the vehicle has the ability to enter the truck’s safety envelope within x seconds.

Vicinity List contains: Autonomous truck id Expiration time A list of nearby vehicles, each with:

Manual vehicle id Soonest possible contact time Capability & Promise information (not shown)

Generated by central computer & sent to each truck

Vicinity list for T2, expires 6:25:32pm:

Pickup 5 soonest contact @ 6:21:10pmLoader 2 soonest contact @ 6:21:48pmTruck M soonest contact @ 6:22:58pm

Page 48: Autonomous Truck Supervisory System Design Overview by Ray Burns

Manual interaction - Vicinity List Concept To be in motion, a truck must have:

A vicinity list (not expired) A position message from each vehicle on the list

All the vehicles on the list mustalso be far enough away.

Vicinity list for T2, expires 6:25:32pm:

Pickup 5 soonest contact @ 6:21:10pmLoader 2 soonest contact @ 6:21:48pmTruck M soonest contact @ 6:22:58pm

Pickup 5

Truck T2

Loader 2

Truck M Stopping distance

Page 49: Autonomous Truck Supervisory System Design Overview by Ray Burns

Manual interaction - Soonest contact Depends on:

Autonomous truck route Manual vehicle properties:

Acceleration abilities Maximum speed Turning radius (speed dependent)

Promises made by manual vehicle

Calculated by: Central - low resolution - for vicinity list Autonomous truck - high resolution

- for slow/stop decision

Pickup moving at 100km/h

1 second2 seconds

Pedestrian with PSU

Stopped pickup truck

Page 50: Autonomous Truck Supervisory System Design Overview by Ray Burns

Manual interaction - Soonest contact Use time-distance function

Time-distance function may be direction dependent

Pickup moving at 100km/h

1 second2 seconds3 seconds

Distance = vehicle.timeDist( Time, )

0

500

1000

1500

2000

2500

0 100 200 300 400 500

Time (seconds)

Dis

tanc

e (m

)

20km/h

30km/h

40km/h

Page 51: Autonomous Truck Supervisory System Design Overview by Ray Burns

Manual interaction - ExamplePickup M4 Truck T9

6:27:55 M4 at (253,647,27º), 14km/h

6:28:10 M4 at (260,645,28º), 16km/h

6:28:15 M4 at (266,643,32º), 13km/h

Loader 2

Pickup M4Truck T9

Stopping distance

Pickup M4Truck T9

Pickup M4 Truck T9

Page 52: Autonomous Truck Supervisory System Design Overview by Ray Burns

Manual interaction - Promise concept Promise: A contract made by a vehicle’s embedded controller to

guarantee (and enforce, if possible) specific behavior.

Promises: Made by the vehicle’s embedded controller May depend on manual input (ie. putting truck into park&lock mode) Specific expiration time Revocable only if all recipients agree Any broken promise immediately shuts down all trucks

Example promise types: Promise to stay within permissions Promise to keep speed below specific limit Promise to maintain other vehicle dynamics (eg. Turning radius speed) Promise not to cross specific boundaries Promise to stay within specific area Promise to stay out of specific area

I promise to limit my speed to 10km/h

Page 53: Autonomous Truck Supervisory System Design Overview by Ray Burns

Manual interaction - Promise types Promise to stay within permissions (Permission Lock)

Used for any/all vehicles in “autonomous” or “parking” mode Can’t be used if manual operator could override

Requirements: Truck must already have active permission Truck must stay within active permission

Enforcement options: Physical lockout on manual controls Physical lockout of access to cab Hardware control of brake & accelerator Button “agree not to touch the controls” Button “enter parking lock mode” Mine policy: Press “request unlock” after turning on truck

and before driving

I promise to follow all Autonomous Interaction rules and stay within my

permitted area at all times.

Including manual trucks

Page 54: Autonomous Truck Supervisory System Design Overview by Ray Burns

Manual interaction - Promise types Promise to keep speed below x

Used with manual vehicles and Portable Safety Units Enforcement options:

Hardware control of brake & accelerator Button “agree to limit speed to 20km/h” Mine policy: 100km/h mine-wide speed limit Mine policy: Press “request speed limit increase” before

exceeding 20km/h

Promise to maintain other vehicle dynamics Acceleration, turning radius, etc.

Promise not to turn left / right Short-term promise, used with turn signals

Speed Limit

50

No Left Turn

Page 55: Autonomous Truck Supervisory System Design Overview by Ray Burns

Manual interaction - Promise types Promise to not cross specific boundaries

Used with manual vehicles or Humans carrying PSUs Possible boundaries:

Pit walls, cliffs, lakes, buildings Physical divider in middle of the road Mine policy: Press “request entry to haul road” before entering haul road Mine policy: Wait for “green light” before entering marked intersection

Promise to stay out of specific area Used with manual vehicles or humans carrying PSUs Enforcement options:

Button “agree to stay off displayed course for T3” Mine policy: When truck on “right of way” & within 100m, stay out of path.

Do Not

Enter

Page 56: Autonomous Truck Supervisory System Design Overview by Ray Burns

Autonomous vs Manual interaction

Autonomous interaction Permission based Extremely efficient Low radio bandwidth Nearly optimal productivity Extremely reliable

-- BUT -- Only work if truck is

Permission Locked

Manual interaction Position message based Not very efficient High radio bandwidth Poor productivity Very reliable

-- BUT -- Works even if truck is not

Permission Locked

Use Autonomous Interaction as much as possible Use Manual Interaction only when necessary

Page 57: Autonomous Truck Supervisory System Design Overview by Ray Burns

Permission Lock: When a vehicle has promised to stay within it’s permissions at all times, thus allowing the use of the more efficient Autonomous interaction.

Autonomous trucks are not always permission locked: Can’t lock if an operator could wrest control at any time Use both systems during manual autonomous transition

Manual trucks may be permission locked when parked: Manual truck requests a rectangular permission Manual truck promises to stay within it

Before truck is driven, promise must be rescinded.

Autonomous vs Manual - Permission Lock

Permission assigned to manual truck

Page 58: Autonomous Truck Supervisory System Design Overview by Ray Burns

Obstacle detection system

Used for interactions with all non-radio objects

Implemented by Safety controller Not part of supervisory system Supplements supervisory collision avoidance

Manual interaction

Obstacle Detection

Autonomous Interaction

Page 59: Autonomous Truck Supervisory System Design Overview by Ray Burns

Traffic Control

Given: Locations of all vehicles Routes for each autonomous truck Loaded / Empty status Mine priorities Traffic rules

Optimize sequencing of trucks through intersections

Prevent deadlocks

Implement traffic rules

LocalPlanner

InteractionPlanner

TrafficNetwork

IntersectionModel

GlobalPlanner

TrafficRules

Algorithms

RoadsIntersectionsEnd points

Manual truck

Which truck should go first?

Page 60: Autonomous Truck Supervisory System Design Overview by Ray Burns

Traffic Control - Intersection Concept Intersection: Anywhere two or more active courses come close

enough together that the truck safety envelopes could overlap.

Basic Concepts: Intersections are based on courses, not roads

Courses need not actually cross to be an intersection

Only active courses are considered A course is active if it has recently been driven or

is currently in a truck’s route Saves bandwidth by ignoring unused turnoffs

Inactive

Page 61: Autonomous Truck Supervisory System Design Overview by Ray Burns

Traffic Control - Intersection Concept Intersections are dynamically recomputed at run time

Dynamically generated courses areanalyzed too

Page 62: Autonomous Truck Supervisory System Design Overview by Ray Burns

Traffic Control - Intersection Concept Special kinds of intersections:

Narrow roads Are intersections by definition Have overlapping permissions Can use special “pull off to side” algorithm:

• Generate two standard permissions for the course

• Generate one permission with “offset” safety envelope

• Link ‘em up right (left as an exercise for the reader)

• Now truck can pull off into “offset” permission to let other truck by

Bidirectional lanes Single lane roads with traffic going both ways Are exclusive intersections, but special optimization algorithms

apply

Page 63: Autonomous Truck Supervisory System Design Overview by Ray Burns

Traffic Control - Traffic Network

Simplify active courses into logical graph model Removes geometric considerations Makes global and local optimization algorithms easier “Intersection & Link” style

Intersection

Intersection

Intersection Intersection

Intersection

Links

Links

Links

Link

Link Link

Page 64: Autonomous Truck Supervisory System Design Overview by Ray Burns

Traffic Control - Planners

Global planner Considers whole network at once

Local planner Makes final sequencing decision

for each intersection Makes sure all traffic rules are applied

Interaction planner Translates sequences into autonomous permissions

and instructions to manual drivers (“stop”, “go”)

LocalPlanner

InteractionPlanner

TrafficNetwork

IntersectionModel

GlobalPlanner

TrafficRules

Algorithms

RoadsIntersectionsEnd points

Manual truck

Page 65: Autonomous Truck Supervisory System Design Overview by Ray Burns

Traffic Control - Traffic Rules Java objects, which can specify just about anything:

Speed limits Stop signs Loaded / empty priorities Special permission schemes Passing behavior etc.

Can be applied to any region of the mine: Polygonal area Named location Whole mine etc.

Can be conditional on arbitrary conditions (ie. Rain, Tuesday)

Raining

Slow to 10km/h within 100m of other truck

on entry

Page 66: Autonomous Truck Supervisory System Design Overview by Ray Burns

Control Constraints

Interface between interaction and speed control Controls truck’s progress along route Two basic kinds of constraint:

Stop point: Point at which truck must come to a rest

Speed limit Point after which we must slow down to a specified speed

The stop point or speed limit goes away when the constraint is removed

Speed Limit

Stop P

oint

Speed

x

Page 67: Autonomous Truck Supervisory System Design Overview by Ray Burns

Speed Control Intelligent optimization of speed:

Prevent “stutter” due to lost messages Reduce unnecessary acceleration Implement global cost policies

0

10

20

30

40

50

60

1000 1050 1100 1150 1200 1250 1300

Trajectory index

Vel

oci

ty (

km/h

)

Maximum speedSpecified speedSpecified-deceleration limitedCrystal ball specifiedOptimized targetTeaching speed

Page 68: Autonomous Truck Supervisory System Design Overview by Ray Burns

Fail-safe systems Autonomous interaction verifier Manual interaction verifier Emergency system Broken promise alert

Planner or Interaction code

1. Analyze options2. Select best possible action3. Generate proof that action is safe4. Pass proof & action to Verifier

8. Send signed action

Verifier

5. Check each proof step6. Generate signature7. Attach it to action

Proof-based Verification with Signatures

Page 69: Autonomous Truck Supervisory System Design Overview by Ray Burns

Fail-safe systems

Autonomous interaction verifier Proof-checks and signs permissions, simple algorithm:

A routine to check two permissions for non-overlap A routine to check if one permission is conditional on another A routine to check if two permissions are for the same truck Main routine calls above three routines on each pair of permissions

Checks signatures on incoming permissions Checks stopping distance Checks if that area is covered by active permissions

Whenever setting the Control Constraints Just before signing a relinquish message

Checks signatures on incoming relinquish messages

Page 70: Autonomous Truck Supervisory System Design Overview by Ray Burns

Fail-safe systems

Manual interaction verifier Proof-checks “soonest contact” time calculation Proof-checks and signs vicinity lists Checks and signs outgoing position reports Checks signatures on:

Incoming vicinity lists Incoming position reports

Page 71: Autonomous Truck Supervisory System Design Overview by Ray Burns

Fail-safe systems

Emergency collision detect Third level of redundancy Watches for imminent collision Only output is the “Emergency! Danger! Slam on the

brakes now! You’re about to crash!” signal Uses separate position report system Uses unrouted Spread Spectrum Radio for messages

Broken promise alert If a promise is ever broken:

The embedded controller broadcasts “Broken promise” alert All autonomous trucks stop immediately After broken promise is rescinded, originator shuts of alert

Page 72: Autonomous Truck Supervisory System Design Overview by Ray Burns

Handling of Dynamic changes

Recording teaching data Can be done at any time Possible to allow data to be enabled for use directly

from the field Converted to B-Spline or other format on truck before

sending over radio Making a course available or unavailable for routing

Can be done through central GUI Can be done directly from CGC (if we allow it) Can be set up to automatically happen based on

external conditions (eg. weather sensor, time of day)

Page 73: Autonomous Truck Supervisory System Design Overview by Ray Burns

Handling of Dynamic changes

How courses become active or inactive Activating courses:

Automatically when added to any truck’s route Can pre-activate courses we know we’ll need

Deactivating courses: Automatically after a certain number of days of disuse Can manually deactivate if not in any truck’s route Location may automatically deactivate generated courses

(useful for loading/dumping where courses always changing)

When active, course is part of intersection model

Page 74: Autonomous Truck Supervisory System Design Overview by Ray Burns

Coordinate systems

WGS-84 standard coordinates Used by GPS receiver Used by entire Java supervisory system Only coordinate system to go across the radio

Komatsu grid coordinates Used only by Nav/Drive & Safety controllers Local to a truck (may vary from truck to truck)

DISPATCH mine coordinates Used only by DISPATCH destination selection code Local to DISPATCH code in central computer

(In compressed form)

Page 75: Autonomous Truck Supervisory System Design Overview by Ray Burns

Coordinate system shifts

All coordinates in mine can shift: If base station is moved If GPS accuracy is improved or a GPS bug is fixed

This isn’t a problem, if we know about it All training courses are date stamped Attribute mechanism can automatically cause:

All courses recorded during specific date/time range are invalid But automatically generate a course with coordinates corrected

Page 76: Autonomous Truck Supervisory System Design Overview by Ray Burns

Graphical User Interface

Can edit: Course Data Course Attributes Traffic Rules Optimization parameters

Extensible