integrating cloud services in behaviour programming for autonomous robots

Integrating Cloud Services in Behaviour Programming for Autonomous Robots

Integrating Cloud Services in BehaviourProgramming for Autonomous Robots

Fabrizio Messina, Giuseppe Pappalardo, Corrado Santoro

ARSLAB - Autonomous and Robotic Systems LaboratoryDipartimento di Matematica e Informatica

Universita di Catania, Italy{messina, pappalardo, santoro}@dmi.unict.it

C-SmartCPS @ ICA3PP 2013 - Vietri sul Mare, 20 Dicembre 2013

December 19, 2013


Outline

1 Motivation

2 PROFETABasic EntitiesSensors and ActionsExecution Semantics

3 CLEPTAArchitectureASync Sensors andActionsA Simple Case-Study

4 Conclusions


Motivation

Motivation (I)

Robots are autonomous systems living in a pysicalenvironment.

We expect to have home robots in the next few years.

Computing platforms for home robots could not possess theadequate amount of computing power — i.e. CPU Powerand/or Storage capacity — to perform certaincomputation-intensive tasks, e.g.:

Speech to text, text to speechSpeaker recognitionNLP processingImage processingPattern recognition (in general)...


Motivation

Motivation (II)

But nowadays home environments are “on the net”

And “computation intensive” tasks could be performed“in the cloud”

Hence the necessity to include, in robot programmingenvironment/platforms, a proper support forCloud Computing


PROFETA

Basic Entities

PROFETA Basics

PROFETA (Python RObotic Framework for dEsigningsTrAtegies) is Python tool for programming autonomoussystems (agents or robots) using the Belief-Desire-Intention(BDI) model.

Designed at ARSLAB @ UNICT in 2010 and available athttp://github.com/corradosantoro/profeta

PROFETA provides a declarative language as a dialect ofAgentSpeak(L).

PROFETA is an “all-in-one” environment supporting bothimperative (algorithmical part) and declarative (behaviouralpart) programming models.

http://github.com/corradosantoro/profeta


PROFETA

Basic Entities

PROFETA Basics

PROFETA provides

A set of classes to represent basic BDI entities, i.e.

Beliefs ⇒ KnowledgeGoals ⇒ States to be achievedActions ⇒ Things to do to reach the Goals

A declarative language—dialect of AgentSpeak(L)—to expressagent’s behaviour

An engine executing that behaviour

A Knowledge base storing the beliefs


PROFETA

Basic Entities

PROFETA Declarative Syntax

A PROFETA behaviour is a set of rules in the formEvent ′′/′′Condition′′ >>′′ set of Actionwhere:

Event can be: belief assert or retract, goal achievementrequest, goal failure.Condition refers to a certain state of the knowledge base.Action can be: belief assert or retract, goal achievementrequest, Python expressions, user defined atomic action.

Example:

+object_at("X","Y") / object_got("no") >>

[ move_to("X", "Y"), pick_object() ]

Such expressions are managed thanks to operatoroverloading


PROFETA

Sensors and Actions

Interface with environment: Actions

In order to interact with the environment, two abstractionsare provided:

ActionsSensors

An action is defined as a subclass of Action.

The execute() method has to implement the computationwhich concretely performs the action.

The associated computation is executed atomically.


PROFETA

Sensors and Actions

Sensors

A Sensor class is provided with the objective of allowing theprogrammer to write the proper code to “sense” theenvironment and generate the proper beliefs.

The programmer has to:subclass Sensoroverride the sense() methodinform the Engine that a new sensor has been added in theprogram.

Method sense() may return

None, nothing has been sensed

+bel(...), something has been sensed, a belief is added tothe KB and the “add event” is generated.

bel(...), something has been sensed, a belief is added tothe KB but the “add event” is NOT generated.


PROFETA

Execution Semantics

PROFETA (Informal) Execution Semantics

PROFETA Main Loop Execution:

1 Scan sensors and, for each of them, call the sense() methodand analyse the return value.

2 If the return value is an event, e.g.+bel(...), find therelevant rule, verify the condition part and then put the rule inexecution.

3 Executes all the actions of the selected rule and then go to 1.

4 Go to 1.

Execution is strictly synchronous!


PROFETA

Execution Semantics

PROFETA and the Cloud

A simple robot which responds to voice commands by using TTSand STT Cloud services (e.g. Google API).

1 ...

2 class utterance(Belief ): pass

34 class move_to(Action ):

5 def execute(self):

6 x = self [0]

7 y = self [1]

8 # ... perform movement to x,y

910 class hear(Sensor ):

11 def sense(self):

12 command = # sample audio data and invoke STT clould service

13 return +utterance(command)

1415 class say(Action ):


17 phrase = self [0]

18 # invoke the TTS could service with "phrase"

1920 +utterance("go") >> [ say("ok , i’m going"), move_to("X", "Y") ]


PROFETA

Execution Semantics

Synchronous Execution

The synchronous execution is a desired PROFETA feature,but it could cause some problems.

Sensors are polled in sequence only if there are no actions tobe executed

If actions/sensors are “slow” an agent/robot is blind, w.r.t.events which could happen in the environment

Action/Sensor dynamics/duration matters!!

Action/Sensor for cloud services are potentially “slow”.


CLEPTA

Architecture

Introducing CLEPTA

To solve the problem a decoupling layer is needed.

The layer has to introduce the needed asynchrony in cloudservice invocation.

CLEPTA (CLoud Extension for ProfeTA) provides adifferent domain for the asynchronous execution of Sensorsand Actions.


CLEPTA

Architecture

Introducing CLEPTA

ServiceInvokers: Entities—subclass of Service—performingthe concrete invocation of cloud services.

PROFETA/CLEPTA Interface: Layer which, using specialkind of Actions and Sensors, connects the ServiceInvokerswith the PROFETA domain.


CLEPTA

ASync Sensors and Actions

Sensors in CLEPTA

A PROFETA Sensor is decoupled by wrapping it with aAsyncSensorProxy provided by CLEPTA.

The AsyncSensorProxy:

Creates a new thread in which the real Sensor executes.Manages a queue to store the beliefs generated by theSensors (w/ configurable length).It is seens by PROFETA as a classical Sensor.


CLEPTA

ASync Sensors and Actions

Actions in CLEPTA

A PROFETA Action is decoupled by creating it as a subclass ofAsyncAction (provided by CLEPTA).

Asynchronous execution of the cloud service invocation is handledby an AsyncActionExecutor, it:

Creates a new thread in which the Invoker executes.Manages a method queue to store the IDs of methods to becalled asynchronously.


CLEPTA

A Simple Case-Study

PROFETA + CLEPTA and the Cloud

A simple robot which responds to voice commands by using TTS andSTT Cloud services (e.g. Google API).

1 ...

2 class utterance(Belief ): pass

34 class say(AsyncAction ):

56 def create_service(self , lang = ’en’):

7 return GoogleTextToSpeech(lang) # provided by CLEPTA


10 utterance = self [0]

11 self.async_invoke(self.get_service ().say , [utterance ])

1213 class hear(Sensor ):

14 def sense(self):

15 command = # sample audio data and invoke STT clould service

16 return +utterance(command)

1718 ...

1920 PROFETA.add_sensor(AsyncSensorProxy(hear ()))

2122 +utterance("go") >> [ say("ok , i’m going"), .... ]

23 +utterance("_") >> [ say("could you repeat please?") ]


Conclusions

Conclusions

PROFETA allows to define robot strategies in a flexible andintuitive way, by using the principles of the BDI model.

PROFETA combines the advantages of both imperative anddeclarative approach, while offering an all-in-one environment.

CLEPTA adds the ability to exploit, in robot’s behaviour,external services provided “in the cloud”.

PROFETA+CLEPA have been successfully applied in variousrobots built at ARSLab, thus proving the effectiveness of thetool:

Voice-driven robotsComplex image analysis (e.g. target recognition)


Conclusions

References


L. Fichera, D. Marletta, V. Nicosia, C. Santoro. Flexible Robot StrategyDesign using Belief-Desire-Intention Model. In Proc. of InternationalConference on Research and Education in Robotics (EUROBOT 2010),Spriger CCIS Series, Rappreswille, Swizerland), May 27-30, 2010.

L. Fichera, D. Marletta, V. Nicosia, C. Santoro. A Methodology toExtend Imperative Languages with AgentSpeak Declarative Constructs.In Proc. of Workshop on Objects and Agents (WOA2010), CEUR-WSPublisher, ISSN 1613-0073, Rimini, Italy, Sept. 5-7, 2010.

G. Fortino, W. Russo, C. Santoro. Translating Statecharts-based into BDIAgents: The DSC/PROFETA case. MAS&S Workshop @ MATES 2013.

F. Messina, G. Pappalardo, C. Santoro. Integrating Cloud Services inBehaviour Programming for Autonomous Robots. CSmart-CPSWorkshop, LNCS, 2013.



Conclusions

Integrating Cloud Services in BehaviourProgramming for Autonomous Robots

Fabrizio Messina, Giuseppe Pappalardo, Corrado Santoro

ARSLAB - Autonomous and Robotic Systems LaboratoryDipartimento di Matematica e Informatica

Universita di Catania, Italy{messina, pappalardo, santoro}@dmi.unict.it

C-SmartCPS @ ICA3PP 2013 - Vietri sul Mare, 20 Dicembre 2013

December 19, 2013