Upload File

Download - Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Transcript
Page 1: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Confidence-aware motion prediction for real-time collision avoidance

Andrea Bajcsy

Long-term Human Motion Prediction Workshop

ICRA 2019

Work with Sylvia Herbert, Jaime Fisac, David Fridovich-Keil, Steven Wang, Sampada Deglurkar, Claire Tomlin and Anca Dragan

Page 2: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

When robots observe behavior that is not well explained by their predictive models, how do

they produce safe but efficient motions?

Page 3: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Connections to reachability analysis

Confidence-aware prediction & planning

Scaling up to multi-robot, multi-human scenarios

Page 4: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Confidence-aware prediction & planning

Scaling up to multi-robot, multi-human scenarios

Connections to reachability analysis

Page 5: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Low probabilityHigh probability

𝑃 𝑒𝐻 π‘₯𝐻; πœƒ, 𝛽) ∝ 𝑒𝛽𝑄(π‘₯𝐻,𝑒𝐻;πœƒ)

Noisily-Rational Human Motion Prediction

ሢπ‘₯𝐻 = 𝑓𝐻(π‘₯𝐻 , 𝑒𝐻)

[Luce, 1959]

[Baker et al., 2007][Ziebart et al., AAAI 2008]

[Ramachandran et al., IJCAI 2007]

[Finn et al., ICML 2016]

[Pfeiffer et al., IROS 2016]

[Herman et al., ICRA 2015]

[Schultz et al., ICRA 2017]

Page 6: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Robust Robot Planning with Human Predictions

π‘₯𝑅𝑔

ሢπ‘₯𝑅 = 𝑓𝑅(π‘₯𝑅 , 𝑒𝑅)

Page 7: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Robust Robot Planning with Human Predictions

π‘₯𝑅𝑔

ሢπ‘₯𝑅 = 𝑓𝑅(π‘₯𝑅 , 𝑒𝑅)

αˆΆπ‘ π‘… = αˆšπ‘“π‘…(𝑠𝑅 , π‘Žπ‘…)

vs.

[Herbert, 2017]

[Mitchell, 2005]

[Lygeros, 2005]

Page 8: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Hamilton-Jacobi Reachability Analysis

Robust Robot Planning with Human Predictions

π‘₯𝑅𝑔

ሢπ‘₯𝑅 = 𝑓𝑅(π‘₯𝑅 , 𝑒𝑅)

αˆΆπ‘ π‘… = αˆšπ‘“π‘…(𝑠𝑅 , π‘Žπ‘…)

vs.

[Herbert, 2017]

𝑉 π‘Ÿ, 𝑇 = supπ‘Ž[𝑒](βˆ™)

inf𝑒(βˆ™)

{ supπ‘‘βˆˆ[0,𝑇]

π‘π‘œπ‘ π‘‘(πœ‰π‘Ÿπ‘’,π‘Ž(𝑑))}

αˆΆπ‘Ÿ = 𝑓𝑅 π‘₯𝑅 , 𝑒𝑅 βˆ’ πœ™( αˆšπ‘“π‘… 𝑠𝑅 , π‘Žπ‘… )

[Mitchell, 2005]

[Lygeros, 2005]

[Herbert*, Chen*, Han, Bansal, Fisac, Tomlin. "FaSTrack: a Modular Framework for Fast and Guaranteed Safe Motion Planning." CDC, 2017.]

Page 9: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Robust Robot Planning with Human Predictions

π‘₯𝑅𝑔

Error bound𝑉 π‘Ÿ, 𝑇 = sup

π‘Ž[𝑒](βˆ™)inf𝑒(βˆ™)

{ supπ‘‘βˆˆ[0,𝑇]

π‘π‘œπ‘ π‘‘(πœ‰π‘Ÿπ‘’,π‘Ž(𝑑))}

αˆΆπ‘Ÿ = 𝑓𝑅 π‘₯𝑅 , 𝑒𝑅 βˆ’ πœ™( αˆšπ‘“π‘… 𝑠𝑅 , π‘Žπ‘… )

[Mitchell, 2005]

[Lygeros, 2005]

Hamilton-Jacobi Reachability Analysis

[Herbert*, Chen*, Han, Bansal, Fisac, Tomlin. "FaSTrack: a Modular Framework for Fast and Guaranteed Safe Motion Planning." CDC, 2017.]

Page 10: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Robust Robot Planning with Human Predictions

π‘₯𝑅𝑔

[Herbert, 2017]

[Fisac, 2018]

𝑉 π‘Ÿ, 𝑇 = supπ‘Ž[𝑒](βˆ™)

inf𝑒(βˆ™)

{ supπ‘‘βˆˆ[0,𝑇]

π‘π‘œπ‘ π‘‘(πœ‰π‘Ÿπ‘’,π‘Ž(𝑑))}

αˆΆπ‘Ÿ = 𝑓𝑅 π‘₯𝑅 , 𝑒𝑅 βˆ’ πœ™( αˆšπ‘“π‘… 𝑠𝑅 , π‘Žπ‘… )

[Mitchell, 2005]

[Lygeros, 2005]

Hamilton-Jacobi Reachability Analysis

Page 11: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

𝑃 Crash > π‘π‘œπ‘™π‘™π‘–π‘ π‘–π‘œπ‘›π‘‘β„Žπ‘Ÿπ‘’π‘ β„Ž

Robust Robot Planning with Human Predictions

π‘₯𝑅𝑔

[Herbert, 2017]

[Mitchell, 2005]

[Fisac, 2018]

[Lygeros, 2005]

Page 12: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Robust Robot Planning with Human Predictions

[Mitchell, 2008]

π‘₯𝑅𝑔

[Herbert, 2017]

[Fisac, 2018]

[Lygeros, 2005]

𝑃 Crash > π‘π‘œπ‘™π‘™π‘–π‘ π‘–π‘œπ‘›π‘‘β„Žπ‘Ÿπ‘’π‘ β„Ž

Page 13: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

What if the predictive model is wrong?

Modeled human goal

UnmodeledObstacle

Robot goal

Page 14: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

High Confidence Low Confidence

UnmodeledObstacle

UnmodeledGoal

human goal

human goal 1

human goal 2

robot goal

robot goal 1

robot goal 2

Page 15: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

𝑃 𝑒𝐻 π‘₯𝐻; πœƒ, 𝛽) ∝ 𝑒𝛽𝑄(π‘₯𝐻,𝑒𝐻;πœƒ)

Bayesian Model Confidence

ሢπ‘₯𝐻 = 𝑓𝐻(π‘₯𝐻 , 𝑒𝐻)

Page 16: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

𝑃 𝑒𝐻 π‘₯𝐻; πœƒ, 𝛽) ∝ 𝑒𝛽𝑄(π‘₯𝐻,𝑒𝐻;πœƒ)

Bayesian Model Confidence

ሢπ‘₯𝐻 = 𝑓𝐻(π‘₯𝐻 , 𝑒𝐻)

𝛽

Page 17: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

𝑃 𝑒𝐻 π‘₯𝐻; πœƒ, 𝛽) ∝ 𝑒𝛽𝑄(π‘₯𝐻,𝑒𝐻;πœƒ)

Bayesian Model Confidence

ሢπ‘₯𝐻 = 𝑓𝐻(π‘₯𝐻 , 𝑒𝐻)

Page 18: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

𝑃 𝑒𝐻 π‘₯𝐻; πœƒ, 𝛽) ∝ 𝑒𝛽𝑄(π‘₯𝐻,𝑒𝐻;πœƒ)

Bayesian Model Confidence

ሢπ‘₯𝐻 = 𝑓𝐻(π‘₯𝐻 , 𝑒𝐻)

Page 19: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Bayesian Model Confidence

ሢπ‘₯𝐻 = 𝑓𝐻(π‘₯𝐻 , 𝑒𝐻)

𝑏𝑑 𝛽 ∝ 𝑃 𝑒𝐻𝑑 π‘₯𝐻

𝑑 ; πœƒ, 𝛽)π‘π‘‘βˆ’1(𝛽)

Page 20: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Bayesian Model Confidence

𝑏𝑑 𝛽 ∝ 𝑃 𝑒𝐻𝑑 π‘₯𝐻

𝑑 ; πœƒ, 𝛽)π‘π‘‘βˆ’1(𝛽)

Page 21: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Fixed confidence Bayesian confidence𝛽

Page 22: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Less confident!

More confident!

Page 23: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Confidence-aware prediction Scaling up to multi-robot, multi-human scenarios

Connections to reachability analysis

Robust motion planning

Page 24: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Confidence-aware prediction Scaling up to multi-robot, multi-human scenarios

Connections to reachability analysis

Robust motion planning

Page 25: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Forward Reachable Set

ሢπ‘₯𝐻 = 𝑓𝐻(π‘₯𝐻 , 𝑒𝐻)

𝐹𝑅𝑆 π‘₯𝐻 , 𝑑 ≔ {π‘₯β€²: βˆƒπ‘’π»(βˆ™), π‘₯β€² = πœ‰(π‘₯𝐻 , 𝑑, 𝑒𝐻(βˆ™))}

πœ‰(π‘₯𝐻(0), 𝑑, 𝑒𝐻(βˆ™))

π‘₯𝐻(0)

Page 26: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Forward Reachable Set

αˆΆβ„Žπ‘₯ = 𝑣𝐻cos(𝑒𝐻)αˆΆβ„Žπ‘¦ = 𝑣𝐻sin(𝑒𝐻)

Δ𝑑 βˆ— 𝑣𝐻

Page 27: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Forward Reachable Set

αˆΆβ„Žπ‘₯ = 𝑣𝐻cos(𝑒𝐻)αˆΆβ„Žπ‘¦ = 𝑣𝐻sin(𝑒𝐻)

Δ𝑑 βˆ— 𝑣𝐻

Page 28: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Forward Reachable Set

αˆΆβ„Žπ‘₯ = 𝑣𝐻cos(𝑒𝐻)αˆΆβ„Žπ‘¦ = 𝑣𝐻sin(𝑒𝐻)

𝟏 𝒔𝒆𝒄 βˆ— 𝑣𝐻

Page 29: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Forward Reachable Set

αˆΆβ„Žπ‘₯ = 𝑣𝐻cos(𝑒𝐻)αˆΆβ„Žπ‘¦ = 𝑣𝐻sin(𝑒𝐻)

𝟏 𝒔𝒆𝒄 βˆ— 𝑣𝐻

π‘ƒπ‘π‘œπ‘™π‘™ ≫ 0

Page 30: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Forward Reachable Set

αˆΆβ„Žπ‘₯ = 𝑣𝐻cos(𝑒𝐻)αˆΆβ„Žπ‘¦ = 𝑣𝐻sin(𝑒𝐻)

𝟏 𝒔𝒆𝒄 βˆ— 𝑣𝐻

π‘ƒπ‘π‘œπ‘™π‘™ > 0

Page 31: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Forward Reachable Set

αˆΆβ„Žπ‘₯ = 𝑣𝐻cos(𝑒𝐻)αˆΆβ„Žπ‘¦ = 𝑣𝐻sin(𝑒𝐻)

𝟏 𝒔𝒆𝒄 βˆ— 𝑣𝐻

π‘ƒπ‘π‘œπ‘™π‘™ β†’ 0

Page 32: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Forward Reachable Set

αˆΆβ„Žπ‘₯ = 𝑣𝐻cos(𝑒𝐻)αˆΆβ„Žπ‘¦ = 𝑣𝐻sin(𝑒𝐻)

2 𝒔 βˆ— 𝑣𝐻 3 𝒔 βˆ— π‘£π»πŸ 𝒔𝒆𝒄 βˆ— 𝑣𝐻

Page 33: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Forward Reachable Set

αˆΆβ„Žπ‘₯ = 𝑣𝐻cos(𝑒𝐻)αˆΆβ„Žπ‘¦ = 𝑣𝐻sin(𝑒𝐻)

2 𝒔 βˆ— 𝑣𝐻 3 𝒔 βˆ— π‘£π»πŸ 𝒔𝒆𝒄 βˆ— 𝑣𝐻

modeled goal

Page 34: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Scaling up to multi-robot, multi-human scenarios

Connections between predictions and FRS

Confidence-aware prediction

Robust motion planning

Confidently determining subsets of the FRS to avoid

Page 35: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Scaling up to multi-robot, multi-human scenarios

Connections between predictions and FRS

Confidence-aware prediction

Robust motion planning

Determining subsets of the FRS to avoid

Page 36: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

1000 x 1000 x 1000 = 1B 1000 x 1000 x 1000 x 1000 = 1T

1000

1000 x 1000 = 1M

Page 37: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

1000

1000

[Bajcsy, 2019]

Page 38: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Hardware Demonstration

Page 39: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Hardware Demonstration

Page 40: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Confidence-aware predictions offer promising directions for scaling

Connections between predictions and FRS

Confidence-aware prediction

Robust motion planning

Determining subsets of the FRS to avoid

1000

1000

1000 x 1000 = 1M

1000 x 1000 x 1000 = 1B

1000 x 1000 x 1000 x 1000 = 1T

1000

Page 41: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Confidence-aware predictions offer promising directions for scaling

Connections between predictions and FRS

Confidence-aware prediction

Robust motion planning

Determining subsets of the FRS to avoid

1000

1000

1000 x 1000 = 1M

1000 x 1000 x 1000 = 1B

1000 x 1000 x 1000 x 1000 = 1T

1000

Page 42: Confidence-aware motion prediction for real-time collision ...prediction for real-time collision avoidance Andrea Bajcsy Long-term Human Motion Prediction Workshop ICRA 2019 Work with

Papers

Herbert*, Chen*, Han, Bansal, Fisac, Tomlin. "FaSTrack: a Modular Framework for Fast and Guaranteed Safe Motion Planning." CDC, 2017.

Fisac*, Bajcsy*, Herbert, Fridovich-Keil, Wang, Tomlin, and Dragan. "Probabilistically Safe Robot Planning with Confidence-Based Human Predictions." RSS, 2018.

Fridovich-Keil*, Bajcsy*, Fisac, Herbert, Wang, Dragan, and Tomlin. β€œConfidence-Aware Motion Prediction for Real-Time Collision Avoidance.” IJRR, 2019

Bajcsy*, Herbert*, Fridovich-Keil, Fisac, Deglurkar, Dragan, and Tomlin, β€œA Scalable Framework for Real-Time Multi-Robot, Multi-Human Collision Avoidance.” ICRA, 2019.

Multi-robot, multi-human planning: https://github.com/HJReachability/faSTPeople

Code

Fast and safe robot tracking: https://github.com/HJReachability/fastrack

Pedestrian prediction: https://github.com/shwang/pedestrian_prediction

ROS wrapper for pedestrian prediction: https://github.com/abajcsy/crazyflie_human


Top Related

PΒ²-ROTECT – Prediction, Protection & Reduction of Orbital Exposure to Collision Threats
PΒ²-ROTECT – Prediction, Protection & Reduction of Orbital Exposure to Collision Threats
1 Collision Prediction from UWB Range MeasurementsarXiv:2010.04313v1 [eess.SP] 9 Oct 2020 1 Collision Prediction from UWB Range Measurements Alemayehu Solomon Abrar, Anh Luong, Gregory
1 Collision Prediction from UWB Range MeasurementsarXiv:2010.04313v1 [eess.SP] 9 Oct 2020 1 Collision Prediction from UWB Range Measurements Alemayehu Solomon Abrar, Anh Luong, Gregory
Ground Motion Prediction Equations: Past, Present, and Futuredaveboore.com/presentations/2014_joyner_lecture_smip15_davis_22oc… · Ground‐Motion Prediction Equations: Past, Present,
Ground Motion Prediction Equations: Past, Present, and Futuredaveboore.com/presentations/2014_joyner_lecture_smip15_davis_22oc… · Ground‐Motion Prediction Equations: Past, Present,
Collision Detection with Relative Screw Motion
Collision Detection with Relative Screw Motion
Motion Detection and Prediction through Spike-Timing
Motion Detection and Prediction through Spike-Timing
HEART MOTION PREDICTION BASED ON ADAPTIVE ESTIMATION ...engr.case.edu/cavusoglu_cenk/papers/EserErdemTunaMS2011.pdfΒ Β· ABSTRACT HEART MOTION PREDICTION BASED ON ADAPTIVE ESTIMATION
HEART MOTION PREDICTION BASED ON ADAPTIVE ESTIMATION ...engr.case.edu/cavusoglu_cenk/papers/EserErdemTunaMS2011.pdfΒ Β· ABSTRACT HEART MOTION PREDICTION BASED ON ADAPTIVE ESTIMATION
Ground Motion Prediction Equations and Seismic …web.ics.purdue.edu/~ecalais/haiti/macrozonation/formation/5_GMPEs...Ground Motion Prediction Equations and Seismic Hazard Assessment
Ground Motion Prediction Equations and Seismic …web.ics.purdue.edu/~ecalais/haiti/macrozonation/formation/5_GMPEs...Ground Motion Prediction Equations and Seismic Hazard Assessment
Prediction of Respiratory Motion Using A Statistical 4D Mean Motion
Prediction of Respiratory Motion Using A Statistical 4D Mean Motion