high-level robot behavior control using pomdps joelle pineau and sebastian thrun carnegie mellon...
TRANSCRIPT
![Page 1: High-level robot behavior control using POMDPs Joelle Pineau and Sebastian Thrun Carnegie Mellon University](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d0a5503460f949dd5f7/html5/thumbnails/1.jpg)
High-level robot behavior control using POMDPsJoelle Pineau and Sebastian Thrun
Carnegie Mellon University
![Page 2: High-level robot behavior control using POMDPs Joelle Pineau and Sebastian Thrun Carnegie Mellon University](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d0a5503460f949dd5f7/html5/thumbnails/2.jpg)
CogRob2002 workshop Joelle Pineau
Introducing Pearl
• Pearl is a prototype nursing robot, providing assistance to both nurses and elderly people.
“thinkers”
eyes with cameras
sonar sensors
handlebars
wheeled base
carrying tray
LCD smile/frown
![Page 3: High-level robot behavior control using POMDPs Joelle Pineau and Sebastian Thrun Carnegie Mellon University](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d0a5503460f949dd5f7/html5/thumbnails/3.jpg)
CogRob2002 workshop Joelle Pineau
Profile of our aging population
450,000 more nurses needed by 2008 campaign to recruit and retain nurses
and other health care providers
![Page 4: High-level robot behavior control using POMDPs Joelle Pineau and Sebastian Thrun Carnegie Mellon University](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d0a5503460f949dd5f7/html5/thumbnails/4.jpg)
CogRob2002 workshop Joelle Pineau
Moving thingsaround
Moving thingsaround
Enabling use of remote
health services
Enabling use of remote
health services
Supportinginter-personal
communication
Supportinginter-personal
communication
Calling for helpin emergencies
Calling for helpin emergencies
Monitoring Rx adherence
& safety
Monitoring Rx adherence
& safety
Providinginformation
(TV, weather)
Providinginformation
(TV, weather)
Management support of
ADLs
Management support of
ADLsReminding
to eat, drink, & take meds
Reminding to eat, drink, & take meds
Providing physical
assistance
Providing physical
assistance
Linking the caregiver to resources
Linking the caregiver to resources
Our vision of robotic-assisted health-care
![Page 5: High-level robot behavior control using POMDPs Joelle Pineau and Sebastian Thrun Carnegie Mellon University](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d0a5503460f949dd5f7/html5/thumbnails/5.jpg)
CogRob2002 workshop Joelle Pineau
The Nursebot project in its early days
![Page 6: High-level robot behavior control using POMDPs Joelle Pineau and Sebastian Thrun Carnegie Mellon University](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d0a5503460f949dd5f7/html5/thumbnails/6.jpg)
CogRob2002 workshop Joelle Pineau
We need a high-level controller that can:
• prioritize goals from specialized modules
• trade-off goals with widely different costs/rewards
• trade-off between information-gathering and goal- satisfaction
• switch between tasks and share sensory information
• handle uncertainty
Speech recognition&synthesis(Ravishankar, 1996; Black et al., 1999)
Autonomous navigation(Burgard et al., 1999)
Autominder(Pollack et al., 2002)
People tracking/following(Montemerlo et al., 2002)
Top controller
![Page 7: High-level robot behavior control using POMDPs Joelle Pineau and Sebastian Thrun Carnegie Mellon University](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d0a5503460f949dd5f7/html5/thumbnails/7.jpg)
CogRob2002 workshop Joelle Pineau
High-level robot behavior control usingPartially Observable Markov Decision Processes
USER + WORLD + ROBOT
ACTIONS
OBSERVATIONSBELIEF STATE
STATE
![Page 8: High-level robot behavior control using POMDPs Joelle Pineau and Sebastian Thrun Carnegie Mellon University](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d0a5503460f949dd5f7/html5/thumbnails/8.jpg)
CogRob2002 workshop Joelle Pineau
What are POMDPs?
POMDP is n-tuple { S, A, , b, T, O, R }
POMDP task 1: State tracking:
After an action, what is the state of the world?
POMDP task 2: Computing a policy:
Which action should the controller apply next?
Very hard!
Not so hard.
Robot belief:
World state:
Control layer:
bt-1 ??
at-1
ot
st-1 st
...
...
??ot-1
...
...
rt-1 rt
![Page 9: High-level robot behavior control using POMDPs Joelle Pineau and Sebastian Thrun Carnegie Mellon University](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d0a5503460f949dd5f7/html5/thumbnails/9.jpg)
CogRob2002 workshop Joelle Pineau
Our approach: Hierarchical POMDPs
• Key Idea: Exploit hierarchical structure in the problem domain to break a problem into many “related” POMDPs.
Act
InvestigateHealth
Move
NavigateCheckPulse
AskWhere
North South East West
CheckMeds
subtask
abstract action
primitive action
![Page 10: High-level robot behavior control using POMDPs Joelle Pineau and Sebastian Thrun Carnegie Mellon University](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d0a5503460f949dd5f7/html5/thumbnails/10.jpg)
CogRob2002 workshop Joelle Pineau
Planning with Hierarchical POMDPs
• Given POMDP model M = { S, A, , b, T, O, R } and hierarchy H
• For each subtask h H:1) Set components
Ah children nodes
Sh S
h bh, Th, Oh, Rh
2) Minimize modelSh {zh(s0), …, zh(sn)}
h {yh(o0), …, yh(op)}
3) Solve subtask hh {bh, Th, Oh, Rh}
Move
Navigate AskWhere
South East WestNorth
ANav ={N,S,E,W}SNav ={X,Y}Nav ={o0,…,om}
AMove ={AskWhere,Navigate}SMove ={X’,Y’,Destination}Move ={o0,…,op}
![Page 11: High-level robot behavior control using POMDPs Joelle Pineau and Sebastian Thrun Carnegie Mellon University](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d0a5503460f949dd5f7/html5/thumbnails/11.jpg)
CogRob2002 workshop Joelle Pineau
Execution with Hierarchical POMDPs
• Step 1 - Update belief:
• Step 2 - Traversing hierarchy top-down, for each subtask:1) Get local belief.
2) Consult local policy.
3) If a is leaf node, terminate.
Else, go to that subtask.
Act
InvestigateHealth Move
NavigateCheckPulse
AskWhere
North South East West
CheckMeds
baoP
sbassPasoP
sbSs
jjii
ij
t
t
,|
,|,| 1
![Page 12: High-level robot behavior control using POMDPs Joelle Pineau and Sebastian Thrun Carnegie Mellon University](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d0a5503460f949dd5f7/html5/thumbnails/12.jpg)
CogRob2002 workshop Joelle Pineau
Experimental Setup
• Task: Robot provides reminders and guidance to elderly user.
• Action hierarchy:
![Page 13: High-level robot behavior control using POMDPs Joelle Pineau and Sebastian Thrun Carnegie Mellon University](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d0a5503460f949dd5f7/html5/thumbnails/13.jpg)
CogRob2002 workshop Joelle Pineau
Sample Scenario
![Page 14: High-level robot behavior control using POMDPs Joelle Pineau and Sebastian Thrun Carnegie Mellon University](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d0a5503460f949dd5f7/html5/thumbnails/14.jpg)
CogRob2002 workshop Joelle Pineau
Still unconvinced about the importance of uncertainty?
![Page 15: High-level robot behavior control using POMDPs Joelle Pineau and Sebastian Thrun Carnegie Mellon University](https://reader035.vdocuments.mx/reader035/viewer/2022062407/56649d0a5503460f949dd5f7/html5/thumbnails/15.jpg)
CogRob2002 workshop Joelle Pineau
On the question of numericanumerical vs logicallogical representations...
• We haven’t tried using logical representations to control the robot…
• But our experience tells us that:
Uncertainty is crucial when dealing with people.
Probabilistic techniques are necessary to reason about uncertainty.
Real belief tracking and planning really matters!