incremental learning of full body...
TRANSCRIPT
Inc
rem
en
tal
Le
arn
ing
of
Fu
llB
od
yM
oti
on
s
Da
na
Ku
lic
Na
ka
mu
ra-Y
am
an
eL
ab
ora
tory
,U
niv
ers
ity
ofTo
kyo,Ja
pa
n
Incre
menta
lLearn
ing
of
Full
Body
Motions
–p.1/3
0
Le
arn
ing
fro
mO
bs
erv
ati
on
for
Hu
ma
no
ids
Le
arn
toa
cco
mp
lish
tasks
by
ob
se
rvin
ga
hu
ma
nte
ach
er,
rath
er
tha
np
rogra
mm
ing
or
tra
jecto
ryp
lan
nin
g
Ta
ke
ad
van
tag
eo
fsim
ilar
str
uctu
reb
etw
ee
nh
um
an
an
dro
bo
t
Su
ita
ble
for
no
n-e
xp
ert
de
mo
nstr
ato
rs
Incre
menta
lLearn
ing
of
Full
Body
Motions
–p.2/3
0
Re
late
dW
ork
[Calin
on
and
Bill
ard
2007]H
OA
Pat
EP
FL
[Ike
uchiet
al.
2004]H
RP
-2at
AIS
TIn
cre
menta
lLearn
ing
of
Full
Body
Motions
–p.3/3
0
Lim
ita
tio
ns
of
the
cu
rre
nt
ap
pro
ach
es
Mo
tio
ns
are
sp
ecifi
ed
ma
nu
ally
by
the
de
sig
ne
r
Inle
arn
ing
syste
ms,
mo
tio
ns
are
se
gm
en
ted
an
dclu
ste
red
a-p
rio
ri
Off
-lin
e,
on
e-s
ho
ttr
ain
ing
No
furt
he
rle
arn
ing
du
rin
gth
eexe
cu
tio
nsta
ge
Incre
menta
lLearn
ing
of
Full
Body
Motions
–p.4/3
0
De
sir
ed
Sy
ste
m
-A
rob
ot
tha
tco
ha
bits
with
hu
ma
ns,
an
dle
arn
sin
cre
me
nta
llyove
ra
life
tim
eo
fo
bse
rva
tio
ns
-A
rob
ot
tha
ta
ccu
mu
late
skn
ow
led
ge
an
dim
pro
ves
pe
rfo
rma
nce
ove
rtim
e
-F
ully
au
ton
om
ou
s,o
n-lin
e,
co
ntinu
ou
sle
arn
ing
Syste
mR
eq
uire
me
nts
:
Au
ton
om
ou
sM
otio
nS
eg
me
nta
tio
n
Au
ton
om
ou
s,O
n-lin
eM
otio
nC
luste
rin
g
Au
ton
om
ou
sK
now
led
ge
Ma
na
ge
me
ntw
ith
fastR
etr
ieva
l
Incre
menta
lLearn
ing
of
Full
Body
Motions
–p.5/3
0
Ta
lkO
utl
ine
Ro
bo
tL
ea
rnin
gfr
om
Ob
se
rva
tio
n
Re
pre
se
ntin
gfu
ll-b
od
yM
otio
n
On
-lin
eS
eg
me
nta
tio
n
On
-lin
eC
luste
rin
ga
nd
Org
an
iza
tio
n
Co
mb
inin
gS
eg
me
nta
tio
na
nd
Clu
ste
rin
g
Le
arn
ing
the
se
qu
en
cin
go
fm
otio
np
rim
itiv
es
On
-lin
eco
rre
ctio
no
fclu
ste
rin
ge
rro
rs
Co
nclu
sio
ns
an
dD
ire
ctio
ns
for
Fu
ture
Wo
rk
Incre
menta
lLearn
ing
of
Full
Body
Motions
–p.6/3
0
Le
arn
ing
fro
mO
bs
erv
ati
on
-M
irro
rN
eu
ron
s
Th
esa
me
ne
ura
lstr
uctu
reis
use
dfo
rb
oth
reco
gn
itio
na
nd
ge
ne
ratio
n[R
izzo
latt
ie
ta
l.2
00
1]
Incre
menta
lLearn
ing
of
Full
Body
Motions
–p.7/3
0
Mo
tio
nR
ep
res
en
tati
on
by
Hid
de
nM
ark
ov
Mo
de
ls
[In
am
ura
et
al.
20
04
]
Sto
ch
astic
mo
de
lca
ptu
rin
gb
oth
sp
atia
la
nd
tem
po
ralva
ria
bili
ty
Mo
de
ltr
ain
ing
(le
arn
ing
)is
imp
lem
en
ted
with
the
Ba
um
-We
lch
Alg
ori
thm
On
ce
the
mo
de
lis
tra
ine
d,th
esa
me
mo
de
lca
nb
eu
se
dfo
rb
oth
Re
co
gn
itio
n(F
orw
ard
Pro
ce
du
re)
Ge
ne
ratio
n(e
ith
er
sto
ch
astic
or
de
term
inis
tic)
Fa
cto
ria
lH
MM
sa
lso
use
dfo
rre
pre
se
ntin
gm
otio
ns
with
gre
ate
r
accu
racy
[Ku
lice
ta
l.2
00
7]
Incre
menta
lLearn
ing
of
Full
Body
Motions
–p.8/3
0
Sto
ch
as
tic
Se
gm
en
tati
on
[Ko
hlm
org
en
an
dL
em
m,
20
01
]
Layo
uta
se
to
fw
ind
ow
sove
rre
ce
ntly
ob
se
rve
dd
ata
Estim
ate
the
de
nsity
dis
trib
utio
nin
ea
ch
win
dow
Co
mp
ute
the
dis
tan
ce
be
twe
en
win
dow
sb
ase
do
nin
tegra
ted
sq
ua
ree
rro
r
De
fin
ea
nH
MM
ove
rth
ese
to
fw
ind
ow
s
Use
Vite
rbia
lgo
rith
mto
ge
ne
rate
the
op
tim
um
sta
tese
qu
en
ce
rep
rese
ntin
gth
e
se
gm
en
tatio
nre
su
lt
Ifkn
ow
nm
otio
ns
are
ava
ilable
,g
en
era
tekn
ow
nsta
tes
an
db
ias
the
mo
de
lto
wa
rds
the
m[K
ulic
an
dN
aka
mu
ra,2
00
8]
Incre
menta
lLearn
ing
of
Full
Body
Motions
–p.9/3
0
On
-lin
eclu
ste
rin
ga
nd
hie
rarc
hy
form
ati
on
Use
HM
Mre
pre
senta
tion
toabstr
act
motion
pattern
sas
they
are
perc
eiv
ed
Clu
ste
rin
div
idualm
otion
pattern
sin
cre
menta
lly,
based
on
intr
a-m
odel
dis
tances
Use
form
ed
clu
ste
rsto
form
gro
up
models
Auto
nom
ously
sele
ct
appro
pri
ate
modelty
pe,
based
on
modeldis
tances
in
the
consid
ere
dre
gio
nofth
em
otion
space
3
1
2
4
5
6
3
1
2
4
5
6
7
3
11
9
14
1
2
4
5
6
7
10
13
12
15
8
1
2
4
5
6
71
1
9
14
3
10
17
18
19
16
13
12
15
20
8
Incre
menta
lLearn
ing
of
Full
Body
Motions
–p.10/3
0
Alg
ori
thm
Ps
eu
do
-Co
de
Fo
llow
ing
ob
se
rva
tio
no
fe
ach
mo
tio
nse
qu
en
ce
:
Ste
p1
En
co
de
ob
se
rva
tio
nse
qu
en
ce
Oi
into
an
HM
Mλ
i
Ste
p2
Ca
lcu
late
the
dis
tan
ce
be
twe
en
λi
an
de
ach
exis
tin
gb
eh
avio
rgro
up
mo
de
lλ
Gj
Ste
p3
Pla
ce
λi
into
the
clo
se
stgro
up
Gc
Ste
p4
Clu
ste
ra
llexe
mp
lars
with
inG
c
Ste
p5
Ifa
su
b-g
rou
pfo
rms,
form
an
ew
no
de
Gn,
co
nta
inin
gth
eexe
mp
lars
ofth
eclu
ste
r
Ste
p6
Usin
gth
eo
bse
rva
tio
nse
qu
en
ce
sfr
om
the
exe
mp
lars
inG
n,
form
the
new
su
b-g
rou
p
mo
de
lλ
Gn
Incre
menta
lLearn
ing
of
Full
Body
Motions
–p.11/3
0
Co
mb
inin
gs
eg
me
nta
tio
na
nd
Clu
ste
rin
g
1:
pro
ced
ure
CO
MB
INE
DS
EG
ME
NT
AT
ION
AN
DC
LU
ST
ER
ING
2:
wh
ile
1d
o
3:
Observ
eD
ata
Poin
t
4:
call
ON
LIN
EV
ITE
RB
ISC
AF
FO
LD
ED
5:
ifS
egP
oin
tth
en
6:
ifIS
VA
LID
(Segm
ent)
then
7:
call
INC
RE
ME
NT
ALC
LU
ST
ER
8:
ifIS
VA
LID
(New
Motion)
then
9:
Add/R
epla
ce
new
motion
as
perm
anentsta
te
10:
en
dif
11:
en
dif
12:
en
dif
13:
en
dw
hile
14:
en
dp
roced
ure
Incre
menta
lLearn
ing
of
Full
Body
Motions
–p.12/3
0
Ex
pe
rim
en
ts
θ
θ
t
x y z
4m
inu
tes
of
co
ntinu
ou
sw
ho
leb
od
ym
otio
nd
ata
ofa
sin
gle
su
bje
ctfr
om
mo
tio
nca
ptu
red
ata
da
tais
co
nve
rte
dto
a2
0D
oF
hu
ma
no
idm
od
elby
on
line
inve
rse
kin
em
atics
First,
testth
eb
asic
se
gm
en
tatio
na
lgo
rith
m,w
ith
no
kn
ow
nsta
tes,
an
dco
mp
are
with
ma
nu
alse
gm
en
tatio
n
Incre
menta
lLearn
ing
of
Full
Body
Motions
–p.13/3
0
Te
sti
ng
the
Se
gm
en
tati
on
θ
θ
t
x y z Next,
test
the
impro
vem
ents
obta
ined
thro
ugh
addin
gknow
nm
otions
Pro
vid
em
anually
extr
acte
dpri
mitiv
es
as
exe
mpla
rs
Incre
menta
lLearn
ing
of
Full
Body
Motions
–p.14/3
0
Se
gm
en
tati
on
Re
su
lts
70
72
74
76
78
80
82
84
RA
RR
AL
SL
SR
Cu
rre
nt
Mo
tion
Se
gm
en
tatio
n
Re
su
lt
(with k
no
wn
mo
tion
s)
Ma
nu
al
Se
gm
en
tatio
n
Se
gm
en
tatio
n
Re
su
lt
(no
know
n m
otion
s)
RK
ER
KR
WL
RWM
ID WR
LL
AR
LA
L
Tim
e [
se
co
nds]
Alg
ori
thm
Co
rre
ct
Fa
lse
Po
sF
als
eN
eg
Ba
sic
12
86
54
3
Sca
ffo
lde
d(w
ith
Sq
ua
ta
nd
Kic
k)
13
95
93
2
Wo
rstp
erf
orm
an
ce
occu
rsa
tsw
itch
ing
po
ints
wh
ere
few
join
tsa
rem
ovin
g
Sa
mp
leV
ide
oIn
cre
menta
lLearn
ing
of
Full
Body
Motions
–p.15/3
0
Te
sti
ng
the
Co
mb
ine
dS
eg
me
nta
tio
na
nd
Clu
ste
rin
g
θ
θ
t
x y z
Pre
se
ntth
eco
mp
lete
4m
inse
qu
en
ce
an
da
pp
lyse
gm
en
tatio
n
Th
ele
afn
od
es
of
the
resu
ltin
gtr
ee
are
use
dto
sca
ffo
ldth
e
se
gm
en
tatio
n
To
facili
tate
an
aly
sis
,4
min
se
qu
en
ce
isp
rese
nte
dre
pe
ate
dly
(ep
och
s),
an
dn
ew
exe
mp
lars
are
ad
de
dto
the
se
gm
en
tatio
n
mo
du
lea
tth
ee
nd
ofe
ach
ep
och
Incre
menta
lLearn
ing
of
Full
Body
Motions
–p.16/3
0
Aft
er
Ep
och
1
LA
L/p
art
ial
RA
RS
LW
RS
WLS
WF
Epoch =
0
Leaf G
roups F
orm
ed
LA
L /
part
ial
WF
WR
S
WLS
RA
RS
L
Exa
mp
leE
xtr
acte
dM
otio
n:
Rig
htA
rmR
ais
e
Incre
menta
lLearn
ing
of
Full
Body
Motions
–p.17/3
0
Aft
er
Ep
och
2
LA
LR
AR
SL
KR
WR
SW
LS
WF
Epoch =
1
Leaf G
roups F
orm
ed
LA
L
WF
WR
S
WLS
RA
R
KR
SL
Exa
mp
leE
xtr
acte
dM
otio
n:
Le
ftA
rmL
ow
er
Incre
menta
lLearn
ing
of
Full
Body
Motions
–p.18/3
0
Aft
er
Ep
och
3
LA
LR
AR
KR
KE
SR
SL
MIS
CW
LS
WR
SW
FR
AL
LA
R
Epoch =
2
Leaf G
roups F
orm
ed
LA
L
RA
L
LA
R
WF
WR
S
WLS
RA
R
KR
KE
SR
MIS
C
SL
Exa
mp
leE
xtr
acte
dM
otio
n:
Kic
kE
xte
nd,
Sq
ua
tR
ais
e
Incre
menta
lLearn
ing
of
Full
Body
Motions
–p.19/3
0
Mo
tio
nP
rim
itiv
eG
rap
h
12
1.0
0
0.2
0
3 0
.50
4 0
.30
0.5
0
0.5
0
5
1.0
0 1
.00
Atth
esam
etim
eas
learn
ing
the
motion
pri
mitiv
es,
learn
the
transitio
n
rule
sbetw
een
pri
mitiv
es
Each
node
inth
em
otion
pri
mitiv
egra
ph
repre
sents
am
otion
pri
mitiv
e,
while
each
edge
repre
sents
an
observ
ed
transitio
nbetw
een
two
motion
pri
mitiv
es
Each
tim
ea
new
motion
pri
mitiv
eis
abstr
acte
dby
the
clu
ste
ring
alg
ori
thm
as
ale
af
node,
acorr
espondin
gnode
isadded
toth
em
otion
pri
mitiv
e
gra
ph.
Each
tim
ea
transitio
nis
observ
ed
betw
een
two
know
nm
otions,
the
edge
count
isupdate
d
The
motion
pri
mitiv
egra
ph
can
then
be
used
togenera
teva
lidsequences
Incre
menta
lLearn
ing
of
Full
Body
Motions
–p.20/3
0
Ex
pe
rim
en
tsw
ith
aH
um
an
oid
Ro
bo
t
Co
llecte
d1
6m
ino
fco
ntinu
ou
sw
ho
leb
od
y
mo
tio
nd
ata
(26
diffe
ren
tm
otio
nty
pe
s)
of
a
sin
gle
su
bje
ctfr
om
mo
tio
nca
ptu
red
ata
da
tais
co
nve
rte
dto
a3
2D
oF
hu
ma
no
idm
od
el
by
on
line
inve
rse
kin
em
atics
on
line
fee
dto
au
tom
ate
dse
gm
en
tatio
n,
clu
ste
rin
ga
nd
mo
tio
ngra
ph
extr
actio
n
Incre
menta
lLearn
ing
of
Full
Body
Motions
–p.21/3
0
Da
taF
low
Dia
gra
m
θ
θ
t
x y z
Incre
menta
lLearn
ing
of
Full
Body
Motions
–p.22/3
0
Ro
bo
tH
ard
wa
rea
nd
Co
ntr
ol
Sy
ste
m
Incre
menta
lLearn
ing
of
Full
Body
Motions
–p.23/3
0
Th
eE
xtr
ac
ted
Mo
tio
nP
rim
itiv
eTre
e
LK
RM
MID
MR
LB
AR
BA
DB
AU
RA
RL
AR
LA
LR
AL
BA
LA
RR
KR
LP
RS
QD
ML
RS
QR
Le
af
Gro
up
s F
orm
ed
Incre
menta
lLearn
ing
of
Full
Body
Motions
–p.24/3
0
Th
eE
xtr
ac
ted
Mo
tio
nP
rim
itiv
eG
rap
h
MR
L(3
)
BA
R(4
)
0.2
5
RA
R(6
)
0.4
2
BA
D(9
)
0.0
8
LA
R(1
0)
0.0
8
ML
R(2
1)
0.1
7
BA
L(1
5)
1.0
0
RA
L(1
1)
1.0
0
BA
U(1
7)
1.0
0
LA
L(1
4)
1.0
0
MM
D(1
3)
1.0
0
0.2
5
0.2
5
0.2
5
SQ
D(2
0)
0.2
5
0.1
4
0.2
7
0.1
4
0.1
4
0.0
5
0.2
7
RK
R(8
)
0.2
0
AR
(16
)
0.8
0
0.5
0
0.5
0
0.2
0
0.2
0
0.2
0
0.2
0
0.2
0
SQ
R(2
2)
1.0
0
1.0
0
0.2
5
0.2
5
0.2
5
0.2
5
Due
tocurr
enthard
ware
limitations
ofth
ero
bot,
motions
invo
lvin
gfo
ot
rais
ing
are
manually
rem
ove
dfr
om
the
gra
ph
Incre
menta
lLearn
ing
of
Full
Body
Motions
–p.25/3
0
Ro
bo
tM
oti
on
Ge
ne
rati
on
Vid
eo
ofE
xp
eri
me
nt
Incre
menta
lLearn
ing
of
Full
Body
Motions
–p.26/3
0
Su
mm
ary
on
Au
tom
ate
dS
eg
me
nta
tio
n
Au
ton
om
ou
s,o
n-lin
ese
gm
en
tatio
no
ffu
llb
od
ym
otio
nd
ata
,by
bu
ildin
ga
nH
MM
ove
ra
win
dow
ofp
revio
us
ob
se
rva
tio
ns,
an
d
fin
din
gth
eo
ptim
um
sta
tese
qu
en
ce
[Ko
hlm
org
en
an
dL
em
m]
Inp
utse
gm
en
tsin
toa
uto
ma
ted
incre
me
nta
lclu
ste
rin
ga
lgo
rith
mfo
r
mo
tio
np
rim
itiv
eextr
actio
ns
Imp
rove
se
gm
en
tatio
nre
su
lts
by
sca
ffo
ldin
gw
ith
kn
ow
nm
otio
n
pri
mitiv
es
ob
tain
ed
fro
mth
eclu
ste
rin
g
As
mo
rem
otio
ns
be
co
me
kn
ow
n,
mo
tio
nm
od
ela
nd
se
gm
en
tatio
n
resu
lts
be
co
me
mo
rea
ccu
rate
Atth
esa
me
tim
e,
lea
rnth
etr
an
sitio
nm
od
elo
fth
em
otio
n
pri
mitiv
es
by
co
nstr
uctin
ga
mo
tio
np
rim
itiv
egra
ph
Incre
menta
lLearn
ing
of
Full
Body
Motions
–p.27/3
0
Fu
ture
Wo
rk
Inco
rpo
ratin
gin
tera
ctio
nw
ith
the
enviro
nm
en
t
Se
lectin
gth
eco
rre
ct
task
rep
rese
nta
tio
n
Inclu
din
ga
dd
itio
na
lle
arn
ing
mo
da
litie
s:
lea
rnin
gfr
om
pra
ctice
an
d
inte
ractio
nw
ith
the
tea
ch
er
Le
arn
ing
co
mp
lex
be
havio
rsfr
om
the
mo
tio
np
rim
itiv
es
Lo
ng
term
au
ton
om
ou
sm
oto
rskill
me
mo
ryo
rga
niz
atio
n
Incre
menta
lLearn
ing
of
Full
Body
Motions
–p.28/3
0
Re
fere
nc
es
D.K
ulic
,W
.Takano
and
Y.N
akam
ura
,In
cre
menta
lLearn
ing,
Clu
ste
ring
and
Hie
rarc
hy
Form
ation
of
Whole
Body
Motion
Patt
ern
susin
gA
daptive
Hid
den
Mark
ov
Chain
s,
Inte
rnationalJourn
alof
Robotics
Researc
h,
Vol.
27,
No.
7,
pp.
761
-784,
2008.
D.K
ulic
,D
.Lee,
C.O
ttand
Y.N
akam
ura
,In
cre
menta
lLearn
ing
ofF
ull
Body
Motion
Pri
mitiv
es
for
Hum
anoid
Robots
,
IEE
EIn
tern
ationalC
onfe
rence
on
Hum
anoid
Robots
,2008,
Accepte
dfo
rP
ublic
ation.
D.K
ulic
and
Y.N
akam
ura
,S
caffold
ing
On-lin
eS
egm
enta
tion
of
Fully
Body
Hum
an
Motion
Patt
ern
s,
IEE
EIn
tern
ational
Confe
rence
on
Inte
lligent
Robots
and
Syste
ms,pp.
2860
-2866,
2008.
D.K
ulic
and
Y.N
akam
ura
,In
cre
menta
lLearn
ing
and
Mem
ory
Consolid
ation
of
Whole
Body
Motion
Patt
ern
s,
Inte
rnationalC
onfe
rence
on
Epig
enetic
Robotics,pp.
61
-68,
2008.
D.K
ulic
,W
.Takano
and
Y.N
akam
ura
,C
om
bin
ing
Auto
mate
dO
n-L
ine
Segm
enta
tion
and
Incre
menta
lC
luste
ring
for
Whole
Body
Motions,
IEE
EIn
tern
ationalC
onfe
rence
on
Robotics
and
Auto
mation,
pp.
2591
-2598,
2008.
D.K
ulic
,W
.Takano
and
Y.N
akam
ura
,Tow
ard
sLife
long
Learn
ing
and
Org
aniz
ation
of
Whole
Body
Motion
Patt
ern
s,
Inte
rnationalS
ym
posiu
mof
Robotics
Researc
h,
pp.
113
-124,
2007.
D.K
ulic
,W
.Takano
and
Y.N
akam
ura
,In
cre
menta
lLearn
ing
of
Full
Body
Motions
via
Adaptive
Facto
rialH
idden
Mark
ov
Models
,In
tern
ationalC
onfe
rence
on
Epig
enetic
Robotics,
pp.
69
-76,
2007.
D.K
ulic
,W
.Takano
and
Y.N
akam
ura
,R
epre
senta
bili
tyof
Hum
an
Motions
by
Facto
rialH
idden
Mark
ov
Models
,IE
EE
Inte
rnationalC
onfe
rence
on
Inte
lligent
Robots
and
Syste
ms,pp.
2388
-2393,
2007.
D.K
ulic
,W
.Takano
and
Y.N
akam
ura
,In
cre
menta
lO
n-lin
eH
iera
rchic
alC
luste
ring
of
Whole
Body
Motion
Patt
ern
s,
IEE
EIn
tern
ationalS
ym
posiu
mon
Robot
and
Hum
an
Inte
ractive
Com
munic
ation,
pp.
1016
-1021,
2007.
Incre
menta
lLearn
ing
of
Full
Body
Motions
–p.29/3
0
Th
eE
nd
Em
ail:
da
na
@yn
l.t.
u-t
okyo.a
c.jp
We
b:
ww
w.y
nl.t.
u-t
okyo.a
c.jp
/d
an
a
Incre
menta
lLearn
ing
of
Full
Body
Motions
–p.30/3
0