small talk with robots

Noury Bouraqadi, Luc Fabresse, Jannik Laval 1

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Pomodoro

2

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Pomodoro

3

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Pomodoro

3

pomodorotechnique.com

Small Talk with Robots

Noury Bouraqadi, Luc Fabresse, Jannik Laval

http://car.mines-douai.fr

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Research on Software Engineering for

5

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Research on Software Engineering for

6

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Research on Software Engineering for

6

Very long-term goal !

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Current robots

7

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Robot at our lab

8

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Infrastructure

9

Physical)Environment)

Robot)

Actuator)Actuator)Computer)

Sensor)Sensor)Sensor) Actuator)Actuator)Actuator)

So9ware)So9ware)So9ware)

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Infrastructure

10

Physical)Environment)

Robot)

Actuator)Actuator)Computer)

Sensor)Sensor)Sensor) Actuator)Actuator)Actuator)

So9ware)So9ware)

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Multiple kinds of robots

11

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Multiple kinds of robots

12

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Multiple kinds of robots

13

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Heterogeneity challenge

14

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Heterogeneity challenge

14

Different sensors and actuators

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Different computing capabilities

Heterogeneity challenge

14

Different sensors and actuators

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Different drivers and protocols

Different computing capabilities

Heterogeneity challenge

14

Different sensors and actuators

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Different drivers and protocols

Different computing capabilities

Heterogeneity challenge

14

Different sensors and actuators

Different low-level algorithms

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Different drivers and protocols

Different computing capabilities

Heterogeneity challenge

14

Different sensors and actuators

Different shapes and sizes

Different low-level algorithms

Noury Bouraqadi, Luc Fabresse, Jannik Laval 15

implementations exist

Noury Bouraqadi, Luc Fabresse, Jannik Laval 16

Physical)Environment)

Robot)

Actuator)Actuator)Computer)

Sensor)Sensor)Sensor) Actuator)Actuator)Actuator)

Algo)Algo)C,#C++#Python#

)))

Noury Bouraqadi, Luc Fabresse, Jannik Laval 17

Physical)Environment)

Robot)

Actuator)Actuator)Computer)

Sensor)Sensor)Sensor) Actuator)Actuator)Actuator)

Algo)Algo)C,)C++)Python)

?" )))

Noury Bouraqadi, Luc Fabresse, Jannik Laval 18

Physical)Environment)

Robot)

Actuator)Actuator)Computer)

Sensor)Sensor)Sensor) Actuator)Actuator)Actuator)

Middleware)

Algo)Algo)C,)C++)Python)

)))

Noury Bouraqadi, Luc Fabresse, Jannik Laval 19

Physical)Environment)

Robot)

Actuator)Actuator)Computer)

Sensor)Sensor)Sensor) Actuator)Actuator)Actuator)

Algo)Algo)C,)C++)Python)

)))

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Our work is PhaROS

20

Physical)Environment)

Robot)

Actuator)Actuator)Computer)

Sensor)Sensor)Sensor) Actuator)Actuator)Actuator)

Algo)Algo)C,)C++)Python)

)))

(small) talking to Robots

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Robot guide using

22

- Museum - Hospital - Shopping Mall - …

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Robot guide using

23

https://vimeo.com/109125625

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Laser scans > Map

24

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Annotated map

25

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Robot guide using

26

Motor%Odometer% Motor%Odometer%Rangefinder%Rangefinder%Rangefinder%

Proprietary%So4ware%

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Robot guide using

27

Motor%Odometer% Motor%Odometer%Rangefinder%Rangefinder%Rangefinder%

Proprietary%So4ware%

Path%%Planner%

Localiza/on%System%

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Robot guide using

28

Motor%Odometer% Motor%Odometer%Rangefinder%Rangefinder%Rangefinder%

Proprietary%So4ware%

Path%%Planner%

Localiza/on%System%

App#

Bridge#

Noury Bouraqadi, Luc Fabresse, Jannik Laval 29

Talking(to(

with%

Noury Bouraqadi, Luc Fabresse, Jannik Laval 30

Talking(to(

with%

Noury Bouraqadi, Luc Fabresse, Jannik Laval 31

Talking(to(

with%

visual programing for robotics

Noury Bouraqadi, Luc Fabresse, Jannik Laval

You know phratch !

33

Noury Bouraqadi, Luc Fabresse, Jannik Laval

You know phratch !

34

on top of Pharo

https://code.google.com/p/scat/

Noury Bouraqadi, Luc Fabresse, Jannik Laval

You know phratch !

34

on top of Pharo

https://code.google.com/p/scat/

Noury Bouraqadi, Luc Fabresse, Jannik Laval

You know phratch !

34

on top of Pharo

phratch.com sponsored by

https://code.google.com/p/scat/

Noury Bouraqadi, Luc Fabresse, Jannik Laval

What is new ?

35

phratch.com

Jetstorm - robotics

Jenkins

smart icons

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Add-ons

36

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Code accessibility

37

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Code accessibility

38

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Code accessibility

39

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Code accessibility

39

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Code accessibility

40

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Code accessibility

41

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Code accessibility

42

Noury Bouraqadi, Luc Fabresse, Jannik Laval

High extensibility !

43

Noury Bouraqadi, Luc Fabresse, Jannik Laval

High extensibility !

44

Noury Bouraqadi, Luc Fabresse, Jannik Laval

High extensibility !

44

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Used for teaching robotics!

45

https://vimeo.com/103126982

More challenges

Noury Bouraqadi, Luc Fabresse, Jannik Laval 47

Noury Bouraqadi, Luc Fabresse, Jannik Laval 48

Resource(Constraints((RAM,(CPU,(Ba5ery...)(

Changing'Condi*ons'

Hardware'+'So+ware'Integra0on'

Noury Bouraqadi, Luc Fabresse, Jannik Laval 49

Resource(Constraints((RAM,(CPU,(Ba5ery...)(

Hardware'+'So+ware'Integra0on'

Self%Adapta+on%

Changing'Condi*ons'

Noury Bouraqadi, Luc Fabresse, Jannik Laval 50

Hardware'+'So+ware'Integra0on'

Self%Adapta+on%

Changing'Condi*ons'

Smalltalk'+'FPGA'

Resource(Constraints((RAM,(CPU,(Ba5ery...)(

Noury Bouraqadi, Luc Fabresse, Jannik Laval 51

•Build a Pharo robotic application

•Identify critical parts•Project the critical parts on FPGA

•Evaluation of performance gain/loss

EXPERIMENT

Noury Bouraqadi, Luc Fabresse, Jannik Laval

52

http://www.youtube.com/watch?v=e1Hzt4soQIc

Noury Bouraqadi, Luc Fabresse, Jannik Laval 53

Camera RGB →HSV

HSV filtrer

Laser sensor

Object detector Motors

Noury Bouraqadi, Luc Fabresse, Jannik Laval 54

Camera RGB →HSV

HSV filtrer

Laser sensor

Object detector

Critical part !

Motors

Noury Bouraqadi, Luc Fabresse, Jannik Laval 55

Smalltalk application

HW/SW Partitioning

SW HW

Pharo VM FPGA

Interface

Noury Bouraqadi, Luc Fabresse, Jannik Laval 56

Self%Adapta+on%

Changing'Condi*ons'

Smalltalk'+'FPGA'

Resource(Constraints((RAM,(CPU,(Ba5ery...)( TDD#

Hardware'+'So+ware'Integra0on'

Noury Bouraqadi, Luc Fabresse, Jannik Laval 57

Physical)Environment)

Robot%

Actuator%Actuator%Computer%

Sensor%Sensor%Sensor% Actuator%Actuator%Actuator%

So2ware%So2ware%So2ware%

Noury Bouraqadi, Luc Fabresse, Jannik Laval 58

Physical)Environment)

Robot%

Actuator%Actuator%Computer%

Sensor%Sensor%Sensor% Actuator%Actuator%Actuator%

So2ware%So2ware%So2ware% Safety)

Noury Bouraqadi, Luc Fabresse, Jannik Laval 59

Simula'on*

Safety*

Noury Bouraqadi, Luc Fabresse, Jannik Laval 60

http://www.youtube.com/watch?v=oNrvuaASwrU

Noury Bouraqadi, Luc Fabresse, Jannik Laval 61

37#

Simula*on#

Noise#

Resources#

Failure#

Asynchrony#

Precision#

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Methodology for safe TDD

62

Ensure emergency stop working!

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Methodology for safe TDD

63

Ensure emergency stop working!

Order tests- Sensors before actuators- Controlled environments before open ones- Static environments before dynamic ones

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Methodology for safe TDD

64

Ensure emergency stop working!

Order tests- Sensors before actuators- Controlled environments before open ones- Static environments before dynamic ones

Run test only if prerequisites pass

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Methodology for safe TDD

65

Ensure emergency stop working!

Order tests- Sensors before actuators- Controlled environments before open ones- Static environments before dynamic ones

Run test only if prerequisites passBoTest'

Framework'

Noury Bouraqadi, Luc Fabresse, Jannik Laval 66

the$

Futureis

Robo-cs

Noury Bouraqadi, Luc Fabresse, Jannik Laval 67

10#000#000+#

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Big players invest in robotics

68

Amazon drones for Parcel Delivery

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Big players invest in robotics

69

Google self-driving car

Noury Bouraqadi, Luc Fabresse, Jannik Laval

Big players invest in robotics

70

Google self-driving car

Google&Bought&8&Robo.cs&Companies&in&

2013!&

Noury Bouraqadi, Luc Fabresse, Jannik Laval 71

the$

Futureis

Robo-cs

Noury Bouraqadi, Luc Fabresse, Jannik Laval 72

the$

Futureis

Robo-cs

the$

Futureis

Robo-csLet’s$$make$it$with$

Smalltalk

Noury Bouraqadi, Luc Fabresse, Jannik Laval 72

the$

Futureis

Robo-cs

the$

Futureis

Robo-csLet’s$$make$it$with$

Smalltalk

Douai loves Argentina !

Noury Bouraqadi, Luc Fabresse, Jannik Laval 74

Noury Bouraqadi, Luc Fabresse, Jannik Laval 74

Noury Bouraqadi, Luc Fabresse, Jannik Laval 74

Noury Bouraqadi, Luc Fabresse, Jannik Laval 74

Douai

Small Talk with Robots

Noury Bouraqadi, Luc Fabresse, Jannik Laval

http://car.mines-douai.fr