II-NEW: Multi-Robot Lab For Quality of Life Applications

proposal for CISE Computing Research Infrastructure (CRI)

http://www.nsf.gov/pubs/2011/nsf11536/nsf11536.htm

1 Introduction

NSF funds will support the purchase of ?? small humanoid robots to establish a multi-robot research and

education facility. Our vision of the future is on in which humans interact with a large number of devices and

robots in a human-robot community. In order to develop the research and educational foundation for such

a future, we propose establishing a multi-robot research and education facility at CMU. We will focus on

research and education about how robots can help everyone, with additional emphasis on helping children

with developmental disorders and older adults.

To work towards a future of human-robot community, we need to develop techniques for multi-robot per-

ception, control, learning from experience, and human-robot interaction. We need to explore how multiple

robots can help each other perform better, and exchange information effectively to achieve useful tasks. We

have extensively explored how individual robots operate in human environments and have investigated and

developed multi-robot systems in customized environments. We are now ready to experiment with multiple

robots in our daily human environments. Faculty who have expressed an interest in such a research facil-

ity include faculty from the Robotics Institute (Atkeson, Kanade, Likhachev, Rybski, Siegel, Thorpe), CSD

(Touretzky, Veloso), HCII (Forlizzi, Cassell), MLD (Gordon), LTI (Rudnicky), and Mechanical Engineering

(Messner). Choset will participate in his role as Director of the Robotics Undergraduate Minor. Example re-

search projects include studies on how the sensors on all robots can be most effectively combined to monitor

a large space, how to allocate resources to handle errors and emergencies (a human or robot accident such

as a spill), and how such a system can be used to present a consistent interface to humans traveling through

the space. We also contemplate experiments in which the robots are installed in other facilities such as a

nursing home, where we would do studies as to how a team of robots could make residents happier, monitor

residents, and reduce the burden on caregivers. We expect the underlying foundational issues of multi-robot

perception, control, and interaction to be very similar at our multiple study sites.

We also expect the multi-robot facility to play a major role in education. We have several courses that

already use small numbers of simple robots to work towards these goals (including 15-491: CMRoboBits:

Creating Intelligent Robots, 16-264: Humanoids, and 16-362: Mobile Robot Programming Laboratory).

With the availability of this multi-robot facility, we will see inclusion of hands on robotics in courses on

AI, machine learning, HCI, speech (LTI), optimization, mechanical engineering, and programming. We

also expect extensive involvement in Andrew’s Leap, the SAMS program, and other K-12 and minority

outreach. We believe two possible applications will be wildly motivating for undergraduates. The first is

team sports (Figure 1). We find that the element of competition is highly motivating for students. We also

find that interactive games and theater is a rich realm for developing and exploring human-robot interaction.

With a set of robots, video games can become physical games. One proposed educational project is a robot

play or musical. We have found that story-based projects are especially attractive and engaging for women

students. This is a link to a class project with an early NAO robot in the CMRoboBits class of 2009:

www.youtube.com/watch?v=hdX498wBftM. This is a link from some of the Creative Technology Night for

Girls work: www.youtube.com/watch?v=0DTJ14hTG3I.

We are focusing on humanoid robots for two reasons. The first is that there are complete humanoid

robots now available in the market, which are aiming to serve both research and education, and for which

we will be able to negotiate a significantly reduced price. The second reason is that the humanoid robot

artifact offers compelling research and educational features, including a complex and motivating articulated

1

body that allows for a rich walking motion, body poses, and gestures, as well as on board sensing, cognitive,

and actuation capabilities that permit sophisticated interaction with humans.

2 Humanoid Robots And Teaching

Veloso talks about her course.

Atkeson talks about his course. Platform for teaching programming, sensing, reasoning, planning, con-

trol, ...

3 Multiple Humanoid Robots Helping Everyone

Talk about Cobots.

Cobots is a focus of National Robotics Initiative.

4 Multiple Humanoid Robots in a Community

We will establish a multi-robot research and education facility at CMU. This facility would include the

Gates-Hillman Complex, Newell Simon Hall, and Wean Hall. These buildings are connected by robot-

navigable bridges and have been used extensively in the past for individual robot studies. Robots would be

able to work in the Wean and NSH 4th floor hallways and GHC floors 3-5 without opening doors, and with

assistance or additional technology will be able to pass through doors and ride elevators to go further.

5 Multiple Humanoid Robots And Human-Robot Interaction

Yaser Sheikh: I’d prospectively be interested in using your robot community to see if we can make the

robots display socially appropriate behavior (look in the right area at the right time by taking cues from

what humans are looking at). Of course, I’m interested in the realtime perception and scene modeling part

of it, and less equipped to do the realtime planning and control of the robots.

Robots are becoming increasingly functional in carrying out everyday tasks. As they begin to enter our

social environments, we will expect them to seamlessly collaborate with not only individuals but groups

of people, as well. In many scenarios, such as disaster recovery, logistical planning, and support at large

events (sports, concerts, etc.), robots will have to co-habit a space with a group of humans and directly

interact with them to cooperatively accomplish tasks. To achieve this, robots must first understand the social

dynamics of humans in a variety situations and environments. The goal of this research is to understand the

social dynamics of humans from cameras directly mounted on members of a social group (e.g., workers in

a factory, friends at a party, employees in an office space, journalists at an event). This will allow co-robots

in a social environment to gain an empathic understanding of the dynamics in the group, by being aware

of where each member is, what they are cognitively attending to, and what, as a group, they are trying to

accomplish. We aim to develop predictive models of social dynamics and discover algorithms to estimate a

time-varying social importance map that associates social importance to each location in 3D space.

Aaron Steinfeld: Steinfeld is currently exploring how robot behaviors and characteristics impact human

trust in robots (IIS-0905148). This work is focused on real-time, task-oriented interactions rather than

social interaction. The research is currently utilizing non-humanoid rovers to examine trust during remote

navigation. Research in other aspects of human-robot interaction strongly suggest that humanoid robot

forms lead to differences in human interpretation of robot performance and acceptance. Therefore, there are

2

opportunities to examine how the humanoid form alters operator trust and use of robots during task-oriented

activities.

6 Multiple Humanoid Robots Helping ChildrenWith Developmental Disor-

ders

Justine Cassell: My current work with humanoid graphical agents suggests that they are effective in helping

children with high-functioning autism to learn contingency, reciprocity and other social skills in communi-

cation. Recent results from my lab even suggest transfer such that children who learn some social skills with

the virtual humans later do better at those social skills with other children. Other work in the field suggests

that robots can also help children with autism but transfer has not been shown. I intend to compare graphical

agents (virtual humans) to humanoid robots in their ability to scaffold the core social communication skills

of children with autism.

7 Multiple Humanoid Robots Helping Older Adults

Atkeson writes this.

Dan Ding: I am interested in integrating the humanoid robots into smart environments. We’d like to

investigate how the smart environment and robots can compensate each other to promote maximum human

function, i.e., to what extent a smart environment equipped with embedded sensors, actuators, networked

appliances and objects can facilitate the operation and reliability of the robots? We would like to use the

robots in the cueing kitchen which can help monitor user actions (which can complement the fixed cameras

in the environment) and deliver prompts and cues with different modalities (e.g., voice or any sound, a

display attached to it, or projecting the information onto any surface) possible at the most opportune moment

and location to assist people with cognitive impairment to complete their daily tasks. I also think the other

two projects in HCHW (health kiosk and dwellsense) could use the robots to provide the health monitoring

feedback and functional assessment feedback to users, essentially serving as a 24/7 caregiver, which could

be more acceptable to the end-users. We could also deploy a robot to the wounded warrior concept homes

which can be designed as a mobile reminder for the residents, or an assessment center for stress and anxiety

which are common in service members (could be REU student projects).

8 Humanoid Robots and Understanding The Brain

Andy Schwartz: I am interested in looking at embodiment, using humanoid robots. We are especially

interested in haptic exploration and tool use. In our brain activity/ robotic research, we have found that

the motor areas of the brain respond when anthropomorphic robot arms/hands perform natural-appearing

movement. This further validates finding that there is an robust observation/mimicry network in the primate

brain and that this is a primary learning mechanism. Anthropomorphic robots could be “taught” using the

same principles and we could derive behavioral principles from how they learn to interact with objects in

the environment. We could then use these principles to build better, brain-controlled interfaces for high-

performance humanoid effectors, in our effort to build prosthetic systems for paralyzed individuals.

9 Modular Robots

Jodi Forlizzi: I would be interested in developing a general modular robot that we could modify in a number

of iterations for the purposes of conducting studies in the lab and in the field. I can supply more information

3

if needed.

10 Impact

Impact: Describe impact.

What is transformative? One transformative element of our proposed research is ...

Education and outreach: Revise this to fit this proposal: We will create an environment for training

of undergraduates and graduate students by: 1) involving them in the research, 2) developing and requiring

an appropriate curriculum, and 3) disseminating educational and research materials. In addition to his

undergraduate teaching of a course on humans and humanoid robots, The CMU Robotics Institute already

has an aggressive outreach program at the K-12 level [?], and we will participate in that program.

Dan Ding: The robots would also be useful for many of the QoLT outreach activities including the

QoLT Ambassador program, TechLink, BodyScott activities, and Creative Tech night for girls etc., for the

K-12 students and general public to explore how robots can be programmed to assist people with reduced

capabilities.

Reid Simmons: Yes, I’d be interested. I am discussing with Anne Mundell and Michael Chemers in

Drama about doing outreach to K-12 students using humanoid robots to teach them about the connections

between art and technology. The robots would do an interactive presentation with human actors and (per-

haps) answer questions from the students. We did a preliminary version of such a presentation recently to

drama students and got very good feedback.

Diversity: Revise this to fit this proposal: One of the co-investigators is female. We will make sure

to include women and minority students. CMU is fortunate in being successful in attracting an unusually

high percentage of female undergraduates in Computer Science. In terms of graduate students, we have had

success working with each of our respective graduate admissions programs.

Management plan:

11 Key Personnel

4

Figure 1: Prof. Manuela Veloso currently uses NAO robots in her studies of teams of intelligent agents in

complex, dynamic, and uncertain environments, in particular adversarial environments. We will build on

this experience base.

5