From Vision to Actionstowards adaptive & autonomous robots

Juxi Leitneristituto dalle molle di studi sull’intelligenza artificiale

idsia / supsi / usi

adaptive

adapt |əәˈdapt|verb [ with obj. ]

[...]• [ no obj. ] become adjusted to new conditions: a large organization can be slow to adapt to change.

[...]

adaptive

autonomousautonomous |ɔːˈtɒnəәməәs|adjective

(of a region) having the freedom to govern itself or control its own affairs: the federation included sixteen autonomous republics.

• having the freedom to act independently: school governors are legally autonomous.

[...]

autonomous

robotics

thanks to G. Metta, IIT and U.Plymouth for their pictures

iCub[Metta et al., 2010]

manipulationaimed at

[Metta et al., 2010]

SOLVED” CHESS is“

object manipulationtowards learned

http://robotics.idsia.ch/

PhD Projectparts

perceptionvisual

robust objectdetection[Leitner et al, 2012a/b, Harding et al., 2013]

cv approachescurrent

cartesian genetic programming

+ min dilate avg INP INP INP

[Leitner et al, 2012a/b, Harding et al., 2013]

detection

icImage* BlueCupDetector::runFilter() { ! icImage* node0 = new GpImage(-8.2, ImgW, ImgH);! icImage* node1 = InputImages[4];! icImage* node2 = node0->avg(node1);! icImage* node4 = node2->threshold(64);! icImage* node6 = node4->erode(5);! icImage* node42 = node6->SmoothMedian(5);! icImage* node79 = node42->gauss(9);! icImage* node99 = node79->dilate(5);

//cleanup ...! return node99; }

visualhand detection

[Leitner et al, 2013]

handsdetecting

locate

localisation approachescurrent

[P2,1] * F * [P1,1]' = 0

transferringspatial perception

[Leitner et al, IROS2012]

trainingset

9DOF

iCubbounding box

6 per eye Carte

sian

Coor

dinate

s

..

.

spatial perception

...

9DO

FiC

ubbo

undi

ng b

ox6

per e

ye

Cart

esian

Coor

dina

tes

fully

con

nect

ed

fully

con

nect

ed

...

neural network[Leitner et al, IJCNN 2013]

resultsANN

-700

-600

-500

-400

-300

-200

-100

0

100

200

300

400

0 200 400 600 800 1000

Y (

mm

)

Sample Index

PredictedExpected

100

150

200

250

300

350

400

450

500

550

600

0 200 400 600 800 1000

X (

mm

)

Sample Index

-1300

-1200

-1100

-1000

-900

-800

-700

-600

-500

-400

-300

-200

0 200 400 600 800 1000

Z (

mm

)

Sample Index

-700

-600

-500

-400

-300

-200

-100

0

100

200

300

400

0 200 400 600 800 1000

Y (

mm

)

Sample Index

PredictedExpected

100

150

200

250

300

350

400

450

500

550

600

0 200 400 600 800 1000

X (

mm

)

Sample Index

-1300

-1200

-1100

-1000

-900

-800

-700

-600

-500

-400

-300

-200

0 200 400 600 800 1000

Z (

mm

)

Sample Index

[Leitner et al, IJCNN 2013]

model update[Leitner et al, IJARS 2012, BICA 2013]

motion

robot

motion

robot

[Stollenga et al, IROS 2013]

MoBeE v1[Frank et al., 2011,2012, 2013]

model

explorationroadmap

useful actionsgenerating

[Stollenga et al., IROS 2013]

repertoireaction

repertoireaction[Leitner et al, in prep]

q’ = J(q) ¹ v

hand/armop-space forcing

CSWorld

CSHand

CSR/CSL

[Leitner et al, in prep]

-

coordinationhand-eye

action perception loop

firstintegration

[Leitner et al, in prep]

workfuture

human-robot interaction

sensorimotor integration

improving online, continuous learning

active vision/learning

action outcome/prediction

post PhD

youthank

http://dilbert.com/strips/comic/2013-10-24/

requirementsUSI PhD

Teachingbreadth (15 ECTS)

publications

after 30 months:

publications

robotics conference papersspace robotics conference paper

robotics journal paper

outreach &

IROS, EpiRob

iSAIRAS

IJARS

AI/ML conference papersjournal publications

GP book chapter

IJCNN, CEC, BICA

BICA, Acta Futura

+ journal papers in preparation Frontiers, RAS, CSEd

publicationsoutreach &