small talk with robots

Noury Bouraqadi, Luc Fabresse, Jannik Laval 1

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Pomodoro

2


Pomodoro

3


Pomodoro

3

pomodorotechnique.com

Small Talk with Robots


http://car.mines-douai.fr


Research on Software Engineering for

5



6



6

Very long-term goal !


Current robots

7


Robot at our lab

8


Infrastructure

9

Physical)Environment)

Robot)

Actuator)Actuator)Computer)

Sensor)Sensor)Sensor) Actuator)Actuator)Actuator)

So9ware)So9ware)So9ware)


Infrastructure

10


Robot)



So9ware)So9ware)


Multiple kinds of robots

11



12



13


Heterogeneity challenge

14



14

Different sensors and actuators


Different computing capabilities


14



Different drivers and protocols



14






14


Different low-level algorithms





14


Different shapes and sizes

Different low-level algorithms


implementations exist



Robot)



Algo)Algo)C,#C++#Python#

)))



Robot)



Algo)Algo)C,)C++)Python)

?" )))



Robot)



Middleware)


)))



Robot)




)))


Our work is PhaROS

20


Robot)




)))

(small) talking to Robots


Robot guide using

22

- Museum - Hospital - Shopping Mall - …


Robot guide using

23

https://vimeo.com/109125625


Laser scans > Map

24


Annotated map

25


Robot guide using

26

Motor%Odometer% Motor%Odometer%Rangefinder%Rangefinder%Rangefinder%

Proprietary%So4ware%


Robot guide using

27



Path%%Planner%

Localiza/on%System%


Robot guide using

28



Path%%Planner%

Localiza/on%System%

App#

Bridge#


Talking(to(

with%

visual programing for robotics


You know phratch !

33


You know phratch !

34

on top of Pharo

https://code.google.com/p/scat/


You know phratch !

34

on top of Pharo

phratch.com sponsored by

https://code.google.com/p/scat/


What is new ?

35

phratch.com

Jetstorm - robotics

Jenkins

smart icons


Add-ons

36


Code accessibility

37


Code accessibility

38


Code accessibility

39


Code accessibility

40


Code accessibility

41


Code accessibility

42


High extensibility !

43


High extensibility !

44


Used for teaching robotics!

45

https://vimeo.com/103126982

More challenges


Resource(Constraints((RAM,(CPU,(Ba5ery...)(

Changing'Condi*ons'

Hardware'+'So+ware'Integra0on'




Self%Adapta+on%

Changing'Condi*ons'



Self%Adapta+on%

Changing'Condi*ons'

Smalltalk'+'FPGA'



•Build a Pharo robotic application

•Identify critical parts•Project the critical parts on FPGA

•Evaluation of performance gain/loss

EXPERIMENT


52

http://www.youtube.com/watch?v=e1Hzt4soQIc


Camera RGB →HSV

HSV filtrer

Laser sensor

Object detector Motors


Camera RGB →HSV

HSV filtrer

Laser sensor

Object detector

Critical part !

Motors


Smalltalk application

HW/SW Partitioning

SW HW

Pharo VM FPGA

Interface


Self%Adapta+on%

Changing'Condi*ons'

Smalltalk'+'FPGA'

Resource(Constraints((RAM,(CPU,(Ba5ery...)( TDD#




Robot%

Actuator%Actuator%Computer%

Sensor%Sensor%Sensor% Actuator%Actuator%Actuator%

So2ware%So2ware%So2ware%



Robot%

Actuator%Actuator%Computer%

Sensor%Sensor%Sensor% Actuator%Actuator%Actuator%

So2ware%So2ware%So2ware% Safety)


Simula'on*

Safety*


http://www.youtube.com/watch?v=oNrvuaASwrU


37#

Simula*on#

Noise#

Resources#

Failure#

Asynchrony#

Precision#


Methodology for safe TDD

62

Ensure emergency stop working!



63


Order tests- Sensors before actuators- Controlled environments before open ones- Static environments before dynamic ones



64



Run test only if prerequisites pass



65



Run test only if prerequisites passBoTest'

Framework'


the$

Futureis

Robo-cs


10#000#000+#


Big players invest in robotics

68

Amazon drones for Parcel Delivery



69

Google self-driving car



70

Google self-driving car

Google&Bought&8&Robo.cs&Companies&in&

2013!&


the$

Futureis

Robo-cs


the$

Futureis

Robo-cs

the$

Futureis

Robo-csLet’s$$make$it$with$

Smalltalk

Douai loves Argentina !


Douai

Small Talk with Robots


http://car.mines-douai.fr

small talk with robots

Software

luc fabresse

jannik laval robot

jannik laval research

jannik laval pomodoro

jannik laval infrastructure

robots noury bouraqadi

jannik laval http

jannik laval different