autonomous vehicles and validation - t&vs · validation tries to answer the question "are...

© Ricardo plc 2019

Autonomous Vehicles

and Validation

2June 2019© Ricardo plc 2019

Validation tries to answer the question

"Are you building the right thing?”

Barry Boehm, Software Engineering Economics, 1981

But…

for an autonomous vehicle what is the right thing?

In general driving isn’t defined - it’s taught.

The First Problem


We don’t know what we are trying to validate.

Therefore…


Driving is also taught within a context

The context is

350,000 years of human evolution

16+ years of dealing with the real world

AND

The real world is complex!

The Second Problem


Example – what is this?

By Beijing Traffic Management Bureau - PDF document GIF diagrams, Public Domain, https://commons.wikimedia.org/w/index.php?curid=34836136


Example – what about this?

By Beijing Traffic Management Bureau - PDF document GIF diagrams, Public Domain, https://commons.wikimedia.org/w/index.php?curid=34836136


The ODD is large…

Operational terrain

– Road surface, curvature, banking etc.

Environmental and weather

– Rain, wind, visibility, lighting, glare etc.

Infrastructure

– Signs, traffic lights, road markings, tool booths etc.

Rules of engagement

– Road rules and differences between countries/states/cities

In addition…

People (operator, other drivers, pedestrians)

Animals – kangaroos are a real problem…

Everything else!

Operational Design Domain (ODD)

Adapted from: Koopman & Fratrik, How Many

Operational Design Domains, Objects, and Events?


Some of everything else…

By William Warby from London, England - Traffic Light Tree, CC BY 2.0,

https://commons.wikimedia.org/w/index.php?curid=5224129

Smart Australia

https://www.adsoftheworld.com/media/outdoor/specsavers_bus_back_crash


Groups yet to (fully) deploy self driving cars…

Waymo (Google)

8 million real world miles The Verge July 2018

8 million miles/day simulation The Atlantic Aug 2017

16 billion miles simulated to date

Tesla

1 Jan 2019 estimated 1 billion miles & Lex Fridman Dec 2018

31 Dec 2019 estimated 2.3 billion miles ibid

GM/Cruise

Honda joined Cruise program at $2.5 billion for a 7% shareholding Reuters Oct 2018

How BIG is the Problem?


Going to the moon might be easier...

Image: SpaceX


Limiting the ODD

Geo-fencing

Excluding conditions (rain, snow, fog)

Works reasonably well – difficult to extend

Requiring the driver to be ready to take over

Mostly works at L2

Won’t work at L3 – L4: humans are bad at monotonous tasks

External human monitoring

Even more boring than being in the car…

Can only deal with one car at a time

Partial Solutions…


Define partial safety goals (specifications)

Stay in lane unless it is unambiguously safe to manoeuvre laterally

RSS - Responsibility-Sensitive Safety

1. Do not hit someone from behind.

2. Do not cut-in recklessly.

3. Right-of-way is given, not taken.

4. Be careful of areas with limited visibility

5. If you can avoid an accident without causing another one, you

must do it.

Top Down Solutions – Rules of Engagement

Adapted from: Nilsson, Safe Self-driving Cars: Challenges and

Some Solutions SSS18

Shalev-Shwartz Shammah, Shashua , On a Formal

Model of Safe and Scalable Self-driving Cars


Stay in lane unless it is unambiguously safe to manoeuvre laterally.

What do you mean by lane?

Stay in lane unless…


What is a lane?

https://imgur.com/a/hAeQI

https://electrek.co/2019/05/20/tesla-fly-guard-rail-funny-balancing-pack/

To see what the drivers sees there is a video at

https://electrek.co/2018/04/03/tesla-autopilot-crash-barrier-

markings-fatal-model-x-accident/

https://imgur.com/a/hAeQI

https://electrek.co/2019/05/20/tesla-fly-guard-rail-funny-balancing-pack/

https://electrek.co/2018/04/03/tesla-autopilot-crash-barrier-markings-fatal-model-x-accident/


Verification is difficult because we can’t define some basic

concepts.

That’s just the start…

Major Issue


Why is this so hard?

Perception Planning Execution Actuation

Planning Checker

Execution Checker

Actuation Checker

ACTUATORSSENSORS

Cross check with

safety envelope

Cross check redundant

information

Cross check Against

Feedback

Heurist planning

Algorithmic Software

Algorithmic Software

How do we know what we perceive is real?

ML Systems

Adapted from https://www.slideshare.net/PhilipKoopman1/edge-cases-and-

autonomous-vehicle-safety-sss-2019

Heuristic planning


The vehicle shall always come to a halt at a stop sign.

Consider the partial safety goal..


Humans

– When they see something they usually know what it is

• but not always…

– But they don’t always see the something

Autonomous Vehicles

- “see” everything

- have no “understanding”

- they “pattern match”

- no pattern, no match

Perception is a BIG problem

Gu, Doian-Gavit & Garg, BadNets: Identifying Vulnerabilities in

the Machine Learning Model Supply Chain arXiv 2019


More might be better…

N-version hardware

Multiple processing units homogeneous/heterogenous

M of N voting

N-version software

Value is debateable…

There is some gain, but not as much as expected – human bias

Ensemble of neural nets

Different models

Different data sets

Not always much better e.g. 94% → 96%

Probably difficult to scale…

Can we engineer a solutions?


When humans are uncertain they “often” take more care…

Can we use uncertainty to increase safety?

Can we measure uncertainty?

– Intrinsic uncertainty in a classification

– Extrinsic uncertainty over time (e.g. uncertainty in classification)

Extrinsic example - Uber crash in Phoenix Arizona, object classified as

unknown object,

then as a vehicle,

finally as a bicycle

Uncertainty

PRELIMINARY REPORT

HIGHWAY HWY18MH010


Invariants

For example - a person has certain proportions

but not on stilts

Adversarial testing

altering images

changing the focus

changing contrast

adding noise

Repeat until?

The “until” part will always be partly arbitrary.

Other ideas…


“perceiving” the stop sign is the problem

stopping at the stop sign isn’t the problem

Posted by u/ken3 on reddit


A successful test is a test that fails.

Final though…

autonomous vehicles and validation - t&vs · validation tries to answer the question "are...

Documents