image from: disaster response

Image from: http://www.automotoportal.com/article/hummer-adds-to-american-red-cross-disaster-response-capability

Disaster Response

2

Outline Introduction

Team Facilities

Goals Background Current Work Conclusions

Introduction | Goals | Background | Current Work | Conclusions

3

Team Dr. M. Bernardine Dias (PI) Dr. Anthony Stentz (Co-PI) Ph.D. Student:

G. Ayorkor Mills-Tettey Staff

Imran Fanaswala Wael Ghazzawi Ameer Abdulsalam


4

Support Team Dr. Brett Browning (Robotics Lab) Dr. Majd Sakr (Robotics Lab) Ph.D. Student:

E. Gil Jones

Staff Dr. Balajee Kannan Freddie Dias David Galati Bryan Nagy


5

Webpage www.qatar.cmu.edu/disaster-response


6

Facilities Faculty Robotics Lab (Doha) Undergraduate Robotics Lab (Doha) rCommerce Lab (Pittsburgh) Field Robotics Center Highbay (Pittsburgh) Robots:

ER1s (Doha and Pittsburgh) P3DX (Pittsburgh and soon

in Doha) Many more…


7

Outline Introduction Goals

Overall project goals Efficient distributed team coordination Optimal coordination of small sub-teams Effective human-robot team coordination

Background Current Work Conclusions


8

Project Goals Develop algorithms and

tools to enhance disaster response activities Specifically in coordination

Design, implement, and validate these tools

Comprehensively survey the needs and existing technology relevant to disaster response


http://us.oneworld.net/files/images/16470.img_assist_custom.jpg

http://blog.wired.com/photos/uncategorized/2007/10/05/disaster.jpg

9

Key Research Areas


Efficient Large-Team Coordination for Complex Tasks in Dynamic and

Uncertain SettingsOptimal Planning and

Re-Planning for SmallerSub-Teams Performing

Critical Tasks

Effective Coordinationof Human-Robot-Agent

Teams Operating inDynamic and Uncertain

Environments

10

Key Research Areas


DistributedMarket-Based

Task Allocation:TraderBots Mathematical

Programming:Branch-and-Price

Capturing Situational Awareness:

AdjustableAutonomy

11

Key Research Areas


TraderBotsBranch

AndPrice

AdjustableAutonomy

12

Outline Introduction Goals Background

Market-Based Allocation Mathematical Programming Adjustable Autonomy

Current Work Conclusions


13

Market-Based Task Allocation Robots are organized as an

economy with virtual money Team mission is to maximize

production and minimize costs Robots exchange money for tasks

to maximize individual profit System is designed to align local

and global profit maximization Auctions enable task allocation


14

Trader Interactions

OpTrader

Revenue paid

Information

Operator(GUI)

Robots(RoboTraders)

Operator(GUI)


15

Mathematical Programming Team coordination problems can be

formulated mathematically as integer programming problems A well-known, but difficult class of problems


16

Integer Programming Optimal solution approaches:

Branch-and-bound Branch-and-price Branch-and-cut … etc.


P

P1 P2

a

b c

01 x 11 x

P3 P4

b c02 x 12 x

17

Mathematical Programming for Team Coordination

Complexity of integer programming restricts the size of problems that can be solved Best suited for small team coordination


18

Adjustable Autonomy Robots can ask humans for help in difficult situations Robots in the disaster response arena usually rely on

more human control


Sliding Autonomy

19

Outline Introduction Goals Background Current Work

Survey Efficient Distributed Coordination Optimal Centralized Coordination Evaluation Human-Robot Teaming

Conclusions


20

Survey of Disaster Response Tech A survey of the needs and existing technology relevant to disaster response:

Disaster scenarios Disaster management Software systems and their applications Hardware systems and their applications


21

TraderBots

BranchAnd

Price

AdjustableAutonomy

TraderBots


22

Motivation Traditional approach – Ad hoc, protocol

based. We need an approach that is:

Efficient Distributed Robust Dynamic Scalable


23

Operator

OpTrader

Robot 1

Robot 2

Robot 3

24

Operator

Robot 1

Robot 2

Robot 3

Task 1 Task 2 Task 3

Task 4

OpTrader

25

Operator

OpTrader

Robot 1

Robot 2

Robot 3


Task 4

OpTraderAuction

Announce and clear auction

26

Operator

OpTrader

Robot 1

Robot 2

Robot 3


Task 4

OpTraderAuction


Bids

Bids

Bids

27

Operator

OpTrader

Robot 1

Robot 2

Robot 3


Task 4

OpTrader

28Introduction | Goals | Background | Current Work | Conclusions

Centralized Allocation

29

Operator

OpTrader

Robot 1

Robot 2

Robot 3


Task 4

RoboTraderAuction


OpTrader

30

Operator

OpTrader

Robot 1

Robot 2

Robot 3


Task 4

RoboTraderAuction


Bids

Bids

OpTrader

31

Operator

OpTrader

Robot 1

Robot 2

Robot 3


Task 4


Distributed Allocation


Comparison

34

Future Work and Challenges Deployment on robots

and hand-held devices Designing domain-

relevant cost functions Multi-threaded

asynchronous implementation

Improved visualization Integrated with simulator


35

Branch-and-Price

TraderBots

AdjustableAutonomy

BranchAnd

Price


36

Mathematical Programming Set-partitioning formulation

23

1 4

5 A

B

r0 r1

r2

r3

r4


A specific type of integer programming formulation, useful when assigning multiple items (e.g. tasks) to each agent

37

Mathematical Programming:Set-Partitioning Example

r0 r1 r2 r3 r4

A: 1

B: 1

Each agent assigned to at most 1 route:

r0 r1 r2 r3 r4

1: = 1

2: = 1… … … … … …

Each task assigned to exactly 1 route:

Arx1

Arx2

Arx3

Brx0

Brx4

Brx0

Brx4

Arx1

Br

Br

Ar

Ar

Ar

Ar

Ar

Ar

Br

Br xcxcxcxcxc

4433221100

23

1 4

5 A

B

r0 r1

r2

r3

r4

Minimize: subject to constraints:


38

Mathematical Programming: Branch-and-Bound Algorithm Steps to finding an optimal solution

Ignore some constraints to solve a simpler problem Solution may be invalid for original problem

“Branch” and insert a new constraint in each branch Repeat until best solution is valid for original problem

P

P1P2

20.67

22 21

),,,,0( 32

31

31

31x

),,,1,0( 31

31

31x)1,1,0,0,0(x

01A

rx 11A

rx

P3 P4

23 21.5)0,0,1,1,0(x),,0,1,0( 3

231x

00 Brx 1

0B

rx

39

Mathematical Programming: Branch-and-Price When feasible routes are too many to enumerate

Start off with a subset of known routes

Master Problem: Tries to assign known routes to agents

Sub problem:Generates additional useful routes to consider

P

P1 P2

P11 P12

r0 r1 r2.

5.45.24.7

r3.5.3 5.04.8 r4.4.9 r5.


….

40

Mathematics Programming for Optimal Centralized Coordination

Novel set-partitioning formulation considering many real-world requirements: Multi-step tasks Time constraints Precedence constraints Simultaneity constraints

Location choice Proximity constraints Mutual-exclusion

constraints


41

Optimal Centralized Coordination:Set-Partitioning Formulation

(Task rewards) – (travel cost) – (waiting/idle cost)

1 route per agent 1 route per task

Valid start time for taskValid arrival time for agent for task

Valid idle time for agent for task

Simultaneity constraints

Proximity constraints

Precedence constraints Mutual exclusion constraints

Location capacity constraints

Maximize:

Subject to:

Sid

e co

nstr

aint

sS

tand

ard

set-

part

ition

ing

form

ulat

ion


42

Optimal Centralized Coordination: Implementation Status

First version of solution method: Branch-and-bound algorithm for coordination

problem with precedence constraints Example


43

Optimal Centralized Coordination:Branch-and-Price Algorithm Created mathematical formulation of sub-problem

Constrained route-planning problem Next steps

Implement solution algorithm for sub-problem Integrate master & sub problem algorithms


Master Problem

Sub problem

44

Adjustable Autonomy

TraderBotsBranch

AndPrice

AdjustableAutonomy


45

Robots in Disaster Response

NISThttp://sciencelinks.jphttp://www.sintef.no

Anna KondaACM-R5 Robot Snake

Versatrax 100


46

Remote Operations Evolution Robotics ER1 robot enhanced in sensing and robustness Highlights challenges faced by first responders when remotely operating robots Camera is the only source of information for the human controller Difficult to maintain accurate situational awareness due to:

Limited “tunnel” vision Lag during communication Inherent sensor noise


47

Hand-held Tools - Motivation

Human first responders need to interface with the task allocation algorithms: To be assigned tasks To intervene in operations

Need portable interface device carried by first responders

Chosen option: A mobile-phone platform running Android OS, an operating system created by the Open Handset Alliance, a group of companies including Google


htc.com

48

Hand-held Tools - System

Next Steps: Purchase two

Android Development Phones

Create a basic software tool to inform human of tasks allocation and allow intervention in operations


49

Evaluation

TraderBotsBranch

AndPrice

AdjustableAutonomy



Simulation System

Visualization World Modeling Physics Engine Server & Client app

Sensor Modeling / Input output feeds Robot Modeling Basic Robot Control Application Interfacing

Artificial Intelligence Autonomous Control Path Planning Sensor Data Analysis & Modeling


Unreal Tournament 2004

Visualization World Modeling Physics Engine Server & Client app


USARSim

Sensor Modeling Robot Modeling Basic Robot Control Application Interfacing


MOAST

Motion ControlMotion Control

Reactive NavigationReactive Navigation

Path PlanningPath Planning

Interface to our AlgorithmInterface to our Algorithm


Virtual Human Robot Interaction

Provides a human-robot interaction training environment

Help fine-tune HRI interface to preview necessary information


Evaluation

Unreal Tournament 2004 Unreal Tournament 2004

USARSimUSARSim

MOAST++MOAST++

High Level AlgorithmHigh Level Algorithm

Modeling real world scenarios in realistic Virtual Environments

Simulating sensor errors Modeling specific robots Testing algorithms in different

deployment scenarios

56

New Robots


Pioneer 3 AT: Outdoors Robot Proposed Sensors

Range Finder Camera Sonar Wireless Communication GPS

Pioneer 3 DX: Outdoors Robot Proposed Sensors

Range Finder Camera Sonar Wireless Communication

57

Outline Introduction Goals Background Current Work Conclusions


58

Conclusions Our goal is to build

effective tools for team coordination in disaster response

We have built significant infrastructure and made good progress in our first year

We are now well-positioned to make good progress in all 3 key areas of research


image from: disaster response

Documents

majd sakr robotics labph

brett browning robotics

complex tasks

support teamdr

smaller subteams

er1s doha

existing technology

pittsburghp3dx pittsburgh