assistance for the elderly in robocare

Planning & Scheduling TeamIstc-Cnr

Assistance for the Elderly in RoboCare

Riccardo Rasconi

ISTC-CNR [PST]

Institute for Cognitive Science and Technology

National Research Council of Italy

Planning and Scheduling Team

http://pst.istc.cnr.it

Project “Project “RoboCareRoboCare: A multi-agent system with fixed and robotic : A multi-agent system with fixed and robotic intelligent components” MIUR Law 449/97(yr 00) – 2003-2006intelligent components” MIUR Law 449/97(yr 00) – 2003-2006

Joint work withAmedeo Cesta,Gabriella Cortellessa, Federico Pecora

Workshop on Telematics and Robotics

for the Quality of Life of the Elderly

28/09/2009


• A distributed system– Software Agents– Robotic Agents– Human Agents

• All cooperate to provide services for human assistance

http://robocare.istc.cnr.it

The RoboCare Project’s Goal


The Different Research Aspects Involved

Active Supervision Framework

Human-RobotInteraction

Personalized Intelligent Assistance

Acceptability Issues

Distributed H/S Infrastructure

Robust MobileRobotic Skills

http://robocare.istc.cnr.it


The RoboCare Domestic Environment

People Localization & TrackingPosture Recognition

Robot Mobility+

User Interaction

PDA

ADL Monitoring


Multiple Intelligent Systems

A number of base services provide the building blocks for higher-level assistive behavior

• Posture recognition

• Person localization

• Mobile robotic platform

• User interaction front-end

• PDA interface

• Daily activity monitor


The robotic platform evolution

2004 2005 2006


Vision Sensors: People Tracking System

Luca Iocchi and G. Riccardo Leone work from Univ. Rome “La Sapienza”

[Bahadori et al, Applied AI, 2007]


Caregiver vs. Active SupervisionFramework interface

Contextual Knowledge Component: Non-Intrusive Activity Supervision

Caregiver specifications compiled into scheduling problems

(a temporal constraint network)

compilation

T-REXscheduling problem

physician

family members

behavioral requirements(in terms of daily activities)

[Pecora et al, ISSEJ, 2006]


Representing temporal prescriptions as a schedule

• Activities and their mutual temporal constraints represented as a Simple Temporal Network

• Dispatched for execution and monitored

• Constraint violation triggers interaction


Schedule Execution Monitoring

• Data are continuously retrieved from the stereo cameras (more about this issue later in the talk);

• Activity status is updated at each execution step;• All constraint violations are detected;

behavioral pattern

time nowtime nowtime nowtime nowtime nowtime nowtime nowtime nowtime nowtime now

tmaximum allowed distance

time nowtime nowtime nowtime now


Using the Scheduler’s Temporal Knowledge to generate contextualized dialogues

breakfast cooking lunch

time now

t

medicine

You should hurry up taking your after-lunch medicine!

executed executed executed



time now

t

medicine

You should wait a little longer before having lunch!

executed executed




time now

t

medicine

Maybe you should cook yourself something warm to

eat!

executed NOT executed



The Proactive Interactor

Talking head

Speechrecognition

Interaction Manager

• What is behind the interaction?

A set of active services ….

Simple I/O Engine

Input/Output Channels

Knowledge for Interaction


The User Interaction Agent (1/2)

Speech Recognition


The User Interaction Agent (2/2)

Verbalizations Synthesis

• Simple synthesis of Speech Acts is performed by analyzing the information contained inthe Constraint Violation DB and in the Environment Status DB


Generating environment-coherent behavior

• Coordination of multiple services is achieved by solving a Multi-Agent Coordination (MAC) problem

• The MAC problem is cast as a Distributed Constraint Optimization Problem (DCOP)

• The DCOP is solved by the ADOPT-N algorithm, an extension of the ADOPT (Asynchronous Distributed Optimization) algorithm for dealing with n-ary constraints


Agents, variables and soft constraints

• Through cost functions, soft constraints are used to prefer (for the monitored person) healthy states and avoid dangerous states

• Cost functions are modeled so as to reflect the desiderata of system behavior

• Detailed description in [Pecora & Cesta, Comp. Int. 2007]


Possible Assistant/assisted interactions

• On-demand interaction(Person takes initiative)– Question / answering

• Proactive interaction(RoboCare takes initiative)– Danger– Warning

The RoboCare Environment as a Mixed-Initiative System


Managing interaction in RoboCare

ProactiveOn-demand


Some examples of interaction


Proactive Warning

• Feedback from sensors is a key activator• Explanation triggered by T-REX temporal knowledge


Proactive Alarm for Danger

• A reactive routine is activated with a precompiled plan– (go-to-place; try-interaction; call-emergency)


On Demand Question-Answering

• Query to the temporalized knowledge in T-REX

• Very simple additional internal query capabilities


Related work

• Intelligent assistants– Same domain and similar technologies:

• Autominder [Pollack et-al, 2003], PEAT [Levinson 1997], PEARL [Pineau et al. 2003; Pollack 2005], I.L.S.A. [Haigh, Kiff, & Ho 2006]

– Capability integration: • Similar problems addressed with CALO, CMradar,

etc. … although project scale quite different!


Conclusions

• RoboCare has addressed (among others) – the open challenge of integrating diversified intelligent capabilities to create a

proactive monitoring assistant for everyday life in a domestic environment

• Highlighted in this work– particular use of the internal knowledge of a constraint-based scheduler (the

temporal constraint network) as well as its capability of reasoning on changes in the environment

– constraint violations determine when the system has to interact. The analysis and interpretation of the violation contribute to determine how to interact with the user

– the use of a distributed coordination algorithm to create a coherent behavior of multiple “active agents”


THANK YOU!QUESTIONS?

assistance for the elderly in robocare

Documents