Learning Inverse Dynamics Models with Contacts

Roberto Calandra1, Serena Ivaldi1,2, Marc Peter Deisenroth3, Elmar Rueckert1, Jan Peters1,4

Abstract— In whole-body control, joint torques and externalforces need to be estimated accurately. In principle, this canbe done through pervasive joint-torque sensing and accuratesystem identification. However, these sensors are expensive andmay not be integrated in all links. Moreover, the exact positionof the contact must be known for a precise estimation. Ifcontacts occur on the whole body, tactile sensors can estimatethe contact location, but this requires a kinematic spatialcalibration, which is prone to errors. Accumulating errorsmay have dramatic effects on the system identification. As analternative to classical model-based approaches we propose adata-driven mixture-of-experts learning approach using Gaus-sian processes. This model predicts joint torques directly fromraw data of tactile and force/torque sensors. We compare ourapproach to an analytic model-based approach on real worlddata recorded from the humanoid iCub. We show that thelearned model accurately predicts the joint torques resultingfrom contact forces, is robust to changes in the environmentand outperforms existing dynamic models that use of force/torque sensor data.

I. INTRODUCTION

A key challenge for torque-controlled humanoid robotsis to accurately estimate their dynamics in presence ofcontacts, e.g., during manipulation in clutter [13], whole-body movements [14] or ground contacts in locomotion [1].Analytic dynamics models suffer from inaccurate parameterestimation, unmodeled dynamics (e.g., friction, couplings,elasticities) and noisy sensor measurements. With contactsthe problem is even more challenging due to discontinuitiesand additional non-linearities, which are difficult to modelor estimate. Moreover, if contact locations are not fixed apriori or known with sufficient precision, small errors in thelocalization of the external force can substantially deterioratethe inverse dynamics computation [7].

Nevertheless, many modern control strategies like inversedynamics control [8], computed torque control [22] or modelpredictive control [16] rely on accurate dynamic models.With inaccurate dynamics models they can produce subopti-mal policies by not taking external forces into account, whichare caused by contacts.

As a first step toward a more informed controller thatexplicitly considers the effect of contacts, we propose

*The research leading to these results has received funding from the Euro-pean Community’s Seventh Framework Programme (FP7/2007–2013) undergrant agreements #270327 (CompLACS) and #600716 (CoDyCo). MPD hasbeen supported by an Imperial College Junior Research Fellowship.

We thank V. Padois and A. Droniou for their help with iCubParis02.1Intelligent Autonomous Systems, TU Darmstadt, Darmstadt, Germany2Inria, Villers-les-Nancy, F-54600, France; CNRS, Loria, UMR n.7503

and Universite de Lorraine, , Vandoeuvre-les-Nancy, F-54500, France3Department of Computing, Imperial College London, London, UK4Max Planck Institute for Intelligent Systems, Tubingen, Germany

Fig. 1: The humanoidrobot iCub used in theexperiments.

to learn the inverse dynamicsmodel from tactile sensor read-ings and force-torque sensors. Incontrast to classical techniquesbased on the identification ofdynamics parameters [26], [19],[24], we propose a fully data-driven machine learning ap-proach based on non-parametricmodels, where both the rigidbody dynamics as well as theeffect of external forces on therobot structure are learned di-rectly from data collected on thereal robot. The proposed modelmakes use of the raw sensor dataand does not require a kinemat-ic/dynamics calibration [26], [19], [24]. In particular, it doesnot need a spatially calibrated model of the skin [6]. Wepropose to use a mixtures-of-experts based on GaussianProcesses (GP) to learn the non-linear system dynamics.Each of these GP experts models a single contact “type” andcan be learned straightforwardly. By using a gating networkthat activates and deactivates the individual GP expertswe can switch between contact models and generalize tomore complex environments. We evaluate our model learningapproach on the arm of the iCub humanoid robot [15](see Fig. 1) and compare to a state-of-the-art model-basedapproach. The learned inverse dynamics model outperformsthe analytic approach and we demonstrate that the learnedmodel can generalize to changing contact locations. To thebest of our knowledge this is the first demonstration of howjoint torques can be learned on a humanoid robot equippedwith tactile and force/torque sensors in presence of contacts.

II. PROBLEM FORMULATION

The inverse dynamics of a robot with m degrees offreedom can be generally described by

τ =M (q) q + h (q, q)︸ ︷︷ ︸τRBD

+ε (q, q, q) , (1)

where q, q and q are the joint positions, velocities andaccelerations, respectively, M (q) is the inertia matrix and

h (q, q) = C(q, q)q + g(q) + Fvq + Fs sgn(q)

is the matrix combining the contributions from Coriolisand centripetal, friction (viscous and static) and gravityforces. The term ε(q, q, q) in (1) captures the errors ofthe model, such as unmodeled dynamics (e.g., elasticities

and Stribeck friction), inaccuracies in the dynamic parame-ters (e.g., masses, inertia), vibrations, couplings, and sensornoise. With a set C = {c1 . . . cn} of contacts ci between therobot and the environment, (1) becomes

τ =M (q) q + h (q, q)︸ ︷︷ ︸τRBD

+ε(q, q, q) +∑ci∈C

J>ci(q)γi , (2)

where the last term accounts for the additive effect ofthe external wrenches (forces and moments) γi applied atcontact location ci, and Jci(q) is the contact Jacobian1.

A. Classical model-based approaches for computing therobot dynamics

Classical approaches for computing τ or τRBD rely on thedynamics model with known or identified kinematics anddynamics parameters [12]. The torques τRBD =M (q) q +h (q, q) can be computed analytically through the rigidbody dynamics model of the robot, a standard parametricdescription of the robot [9]. The term ε(q, q, q) is oftenneglected, or implicitly taken into account by consideringa perturbation in the dynamics parameters of τRBD, whichneed to be identified accurately.

Although parameter identification for industrial robots isrelatively easy with exciting trajectories [20], the procedurefor floating-base robots, such as humanoids, is not straight-forward because of two main issues: 1) The generation ofsufficiently large accelerations for the identification whilemaintaining the robot balance and the control of contacts.This issue was well explained by Yamane [26], who proposeda technique to identify the mass and the local center ofmass of the links in a humanoid robot with fixed feet atthe ground and slow joint trajectories. 2) The measurementof the external forces γi exerted on the robot. Note that itmay not be straightforward to measure the external forces γi

as it is not possible to cover the robot body with 6-axisforce/torque sensors to measure the force exerted on everypossible contact location ci. Usually, such sensors are big,heavy and expensive. Thus, they are carefully placed wherethe external forces are critical for the main tasks, for exampleat the end-effectors for manipulation and at the feet forbalancing. In such a case, it is possible to identify thedynamics parameters while balancing and walking with-out additional contacts [19]. When force/torque sensors areplaced proximally, such as in the iCub arms [10], some ofthe dynamics parameters can be identified, but in absence ofcontacts [24].

When multiple contacts are exerted on the robot structureat locations other than the classical end-effectors, it is stillpossible to compute a precise inverse dynamics model, butthis requires both pervasive joint torque sensing, such as inToro [19], and additional force/torque and tactile sensing,

1The contact location ci is not necessarily fixed as the contacts mayoccur on the whole robotic structure and not exclusively at the end-effectors. In such a case, the contact location, if not known a priori, mustbe estimated, typically through distributed tactile sensors. To compute thecontact Jacobian, we need the position of the contact point with respect tothe reference frame of the link [10]. Such a knowledge requires a kinematiccalibration of the skin as explained in [6].

skin unit

force/torquesensor

joint torquesensor

externalforce

Fig. 2: Illustration of the force/torque and tactile sensorsduring a contact of the robot arm with the environment.

such as in iCub [11]. Moreover, it requires the precise knowl-edge of the contact locations detected by the tactile sensors,which necessitates a spatial calibration of the skin [6]. Thisprocedure is prone to errors, and it has been shown thatsmall errors in the kinematics calibration of the taxels (i.e.,the tactile units) can induce non-negligible errors in theestimation of the contact forces [7].

Generally, these model-based approaches have three mainlimitations: 1) It is hard to add details about couplings,elasticity, friction and other nonlinear dynamics, which arerequired for high accuracy; 2) The performance of the data-driven identification strongly depends on the experimen-tal setting (with/without contacts) and the exciting trajec-tories [20]; 3) They make strong assumptions to handlecontacts.

B. Learning the inverse dynamics

An alternative and appealing approach to analytic dynam-ics computation is to use machine learning methods to learnthe dynamics model of a robot [18], [25], [5]. Without theneed for compensating for inaccurate dynamics parametersand accumulated errors, a learned dynamics model canimprove the tracking and control performances of a robot,as shown in [17] for an industrial manipulator. The clearadvantage of learning the inverse dynamics is that we canovercome the limitations of the aforementioned approaches:difficulty in modeling complex nonlinear dynamics, restric-tive assumptions regarding contacts and sensors, prior ac-curate kinematics calibration of the tactile sensors. Despitethe success of learned dynamics models in robotics, to thebest of our knowledge there are no examples in the literaturewhere dynamic contacts are also learned. The inclusion ofdynamic contact models in the dynamics highlights twomain problems: First, switching from a no-contact modelto a contact-model requires to observe the system state andto model a discontinuous function [23]. Second, switchingbetween different contacts ci ∈ C must be properly handled.

Here, we provide a first formulation to this problem, andwe show that it is possible to learn the inverse dynamicsmodel of the arm of the iCub robot by means of prox-imal force/torque measurements F and distributed tactilesensors S (without requiring a spatially calibrated model ofthe skin [6]).

GatingNetwork

Inverse Dynamics

+Contact models

Fig. 3: Our approach extends existing inverse dynamicswithout contacts by learning many contact models, whichserve as correction terms under different contact types. Thedecision of which contact model to activate is made by agating network, which uses skin measurements S, the forcetorque sensors F and the current state q, q, q.

III. LEARNING INVERSE DYNAMICS WITH CONTACTS

In this section, we present our proposed approach tolearning inverse dynamics with contacts. We first formalizethe problem as learning a mixture-of-experts model. Thenwe detail how we implement Gaussian processes as thecorresponding experts.

A. Learning contacts as a mixture-of-experts

When learning inverse dynamics with contacts (2), weassume that the (contact-free) inverse dynamics from (1)can be computed precisely, either from an analytic modelor from a learned model [17]. In our experiments, weemploy a learned GP model as contact-free inverse dynamics.The reason for this choice are the unmodeled dynamicsε (q, q, q), which introduce substantial errors even withoutcontacts. As a result of the pre-existing contact-free inversedynamics, only the model of the residual term of the externalforces

∑ci∈C J

>ci(q)γi has to be separately learned. In this

paper, we consider a robot that is provided with skin mea-surements S from the tactile sensors, force measurements Ffrom the force torque sensors (FTS) and the ground truth ofthe torques τ from the joint torque sensors (JTS), as shownin Fig. 2. Modeling the external forces y =

∑ci∈C J

>ci(q)γi

can be formalized as the regression task

y = f([q,S]) + w , (3)

where w is an i.i.d. Gaussian measurement noise with mean 0and variance σ2

w. Contacts with different parts of the bodylead to different effects in the dynamics. Intuitively, it is nec-essary to consider the skin input S to identify the position ofthe contact. Additionally, it would be desirable to obtain fromthe skin sensors measurements of the force applied by thecontacts. However, the artificial skin used in our experimentsdoes not provide a precise six-dimensional measure of thecontact force. Therefore, in the implementation of our modelwe substitute the force measurement from the skin with theforce/torque measurements F . The corresponding regressionproblem (3) is complicated due to the high-dimensional spaceof the input x ∈ X (the skin measurements S alone accountfor hundreds of dimensions). Therefore, we rephrase this

regression task as a problem of learning a mixture-of-expertsmodel. With this model, we decompose (3) as∑

ci∈CJ>ci(q)γi =

∑j∈J

fj([q,F ]) + w , (4)

where J is the set of active contacts and fj the expertcorresponding to each contact. Note that the skin input Sis no longer explicitly part of the inputs of the experts,since it is use exclusively to determine which contact iscurrently active. Therefore, each single expert fj is nowsufficiently low-dimensional to be modeled independently. Atthe same time the possibility of summing the contribution ofeach contact allows to account for composite behaviors. Assingle expert fj we use Gaussian processes for the mapping[q,F ] 7→ J>j (q)γj . A gating network is used to computethe set of active contacts J and to add their contributions.An illustration of our approach is shown in Fig. 3. In ageneral setting, gating networks are often used to provide”soft” decisions (i.e., continuous) However, in this paperwe implement the gating network as a multi-class classifierJ = g(q,S,F ) that binarily selects if contact is currentlyongoing.

B. Gaussian processes as expert models

Gaussian Processes (GPs) [21] are a state-of-the-art re-gression method. They have been used in robotics tolearn dynamics models [5] and for control [4]. In thispaper, a GP is a distribution over inverse dynamics modelsf ∼ GP (mf , kf ) , fully defined by a prior mean mf anda covariance function kf . We choose as prior mean mf ≡τRBD and as covariance function kf the squared exponentialwith automatic relevance determination and Gaussian noise

k(xp,xq) = σ2f exp

(−1

2 (xp−xq)TΛ−1(xp−xq)

)+σ2

wδpq

where Λ = diag([l21, ..., l2D]) and δpq is the Kronecker delta

(which is one if p = q and zero otherwise). Here, li are thelength-scales, σ2

f is the variance of the latent function f(·)and σ2

w the noise variance.In our experiments, when learning contact models, the

input is defined as x = [q,F ], while the output (obser-vations) y = τ are the torques. Hence, given n traininginputs X = [x1, ...,xn] and corresponding training targetsy = [y1, ..., yn], we define the training data set D = {X,y}.Training the GP corresponds to finding good hyperparame-ters θ = [li, σf , σw], which is done by the standard procedureof maximizing the marginal likelihood [21].

The GP yields the predictive distribution over torques fora new input x∗ = [q∗,F ∗]

p(y|D,x∗,θ) = N(µ(x∗), σ

2(x∗)), (5)

where the mean µ(x∗) and the variance σ2(x∗) are

µ(x∗) = kT∗K

−1y , σ2(x∗) = k∗∗ − kT∗K−1k∗ , (6)

respectively. The entries of the matrix K are Kij =k(xi,xj), and we define k∗∗ = k(x,x) and k∗ = k(X,x).

Time [s]

Pos

ition

[deg

]

0 5 10 15 2015

20

25

30

35

40

45No contact

Contact

(a) Task space

Time [s]

Torq

ue [N

m]

0 5 10 15 200

1

2

3

4No contact

Contact

(b) Torque

Fig. 4: Learning a single contact: Effects of a contact (greencurve) compared to the free movement without obstacle (bluecurve). These effects are visible in the task space position (a)and in the torque measured by the joint torque sensor (b).

IV. EXPERIMENTAL SET-UP AND EVALUATION

In this section, we describe the experimental setting andthe humanoid robot iCub used in the experiments. We presentfour different experiments where we demonstrate that 1)Our approach can learn single contact models; 2) A singlelearned model (i.e., an expert) is robust to small changesin the position of the contact; 3) Our approach extends tomultiple contacts by combining models of single contacts;4) The gating network activating the experts can be learnedto reduce the complexity of manually design it.

A. Experimental set-up

The experiments were conducted on the iCub [15], ahumanoid robot with 53 degrees of freedom, sized as a child(104 cm tall, 24 kg of weight). This robot is equipped withseveral sensors: an inertial sensor in the head, four 6-axisforce/torque sensors placed proximally in the middle of legsand arms, and an artificial skin consisting of many distributedtactile elements (taxels) mounted on the robot covers [2].The information from these three types of sensors is used toestimate the joint torques and the external contact forces bythe iDyn library [11]. In the following, τ IDYN denotes thejoint torques estimated by the iDyn library, which we useas analytical model for comparison. For more detail on itscontact detection and taxels calibration we refer to [6], [7].

The iCub used in the experiments is equipped with threeadditional Joint Torque Sensors (JTSs), two in the shoulderand one in the elbow. The JTSs are calibrated by computingthe offset and gain trough least-square regression with respectto the output of iDyn2. We consider these calibrated JTSs asground truth measurements of the joint torques τ . In ourexperiments, we used the iCub torso and arms (3 and 7degrees of freedom, respectively) and the skin input S fromthe forearm, which consists of 270 sensor measurements.

B. Learning a single contact

In this experiment, we consider the iCub making contactwith a single obstacle. The evaluation is performed on asimple tracking task with the iCub’s end-effector moving

2Calibrating the JTS on iDyn is the most favorable condition for iDyn.Different real-world calibration procedure would introduce modeling inac-curacies and reduce the performance of iDyn.

Time [s]

Torq

ue [N

m]

0 2 4 6 8 100.1

0.2

0.3

0.4

0.5

0.6JTS

IDYNLearned Model

(a) Real data.

Time [s]

Torq

ue [N

m]

0 2 4 6 8 100.1

0.2

0.3

0.4

0.5

0.6JTS

IDYNLearned Model

(b) Filtered visualization.

Fig. 5: Learning a single contact: Comparison of the torquemeasured at the elbow (with contact) by the JTS, estimatedby iDyn and our learned model (shown as mean ± 2 std).Our learned model better predict the torque measured byJTS (a). Additionally, due to the identification of the noisein the model, its prediction is smoother compared to both thenoisy JTS measurements and the prediction from iDyn. Forvisualization purposes we also show the predictions whenfiltering JTS and iDyn (b).

Method Shoulder 1 [Nm] Shoulder 2 [Nm] Elbow [Nm]

Full trajectory iDyn 0.09± 1.1× 10−3 0.16± 1.8× 10−3 0.05± 7.4× 10−4

Our model 0.04± 5.6× 10−4 0.07± 9.8× 10−4 0.02± 3.1× 10−4

Contact only iDyn 0.07± 3.1× 10−3 0.13± 5.7× 10−3 0.08± 3.0× 10−3

Our model 0.03± 1.5× 10−3 0.12± 5.9× 10−3 0.03± 1.3× 10−3

TABLE I: Learning a single contact: Mean and standarddeviation of the mean for the RMSE of the test set for groundtruth, predictions with the iDyn and our learned model. Thelearned model predicts the torque more accurately than iDynfor both the full trajectory and during the only contact phase.

along a circular trajectory. We repeat the task twice: firstwithout any contact and then with a contact at a fixedposition. Fig. 4 shows the effects of the contact in termsof position and torque during the tracking task. When thecontact occurs the position error increases considerably. As aresult, the torque is increased to compensate for the obstacle.We collected 10 repetitions of the trajectory with the contactand used 8 of them to train the model. The remainingtrajectories are used as test set to evaluate the predictiveperformances of our learned model. For this experiment weconsider a single expert (the gating network still decideswhether to activate the expert).

We compare the baseline joint torques (i.e., the JTS) to theestimation by the analytic model iDyn and the torques τ IDMpredicted by our learned model. In Table I, we report theroot mean square error (RMSE) and the standard deviationof the mean of iDyn and our learned model for all the threejoints. Additionally, we report both the error of the learnedmodels (learned RBD plus learned contact model) duringthe full trajectory and exclusively during the contact. In fiveout of six cases, the learned model performs better thanthe analytic model. In the sixth case (contact only, shoulder2), the performance of the learned model is similar to theanalytic model. However, increasing the amount of data usedfor training may further increase the performance of thelearned model. A visual representation of the predictions ofthe test set for the elbow joints is shown in Fig. 5.

This experiment provides evidence that the classical rigid-body dynamics model τRBD(q, q, q) and the iDyn estimation

Position X [cm]

Pos

ition

Y [c

m]

0 5 10 15 200

5

10

15Far ContactClose ContactMedium Contact

(a) Task space

Time [s]

Torq

ue [N

m]

0 2 4 6 8 100.5

1

1.5

2

2.5

3Far Contact

Medium ContactClose Contact

(b) Torque

Fig. 6: Robustness of the single contact model: Effects ofthe contact on the task space and the torque for the threedifferent contact types: contact 1 (far), contact 2 (medium)and contact 3 (close). The task in absence of contact isdisplayed as reference (black dashed curve).

(a) Contact 1 (b) Contact 2 (c) Contact 3

Fig. 7: Robustness of the single contact model. Thedifferent contact locations detected by the forearm skinrespectively for the three contacts: contact 1 (far), contact 2(medium) and contact 3 (close).

(that also exploits proximal force/torque sensing) fail toaccurately estimate the joint torques when the robot is incontact with the environment. Moreover, we show that thelearned contact model, when combined with the RBD model,provides a better approximation of the joint torques.

C. Robustness of the single contact model

In the following, we show that the prediction performanceof each GP expert is robust to small variations in the positionof the contact. This is important since the exact position ofthe obstacle does not need to be known in advance (within asingle expert fj). As in the previous experiment we considera tracking task along a circular trajectory. However, this timethe obstacle is placed at one of three different positionsalong the trajectory: close, medium and far. Each of theseobstacles is shifted 2 cm along the horizontal axis. Obstaclesat different positions along the trajectory lead to differenteffects in terms of both joint position and torque signal,as clearly visible in Fig. 6. Note that the skin input Swill also be affected, as shown in Fig. 7. Hence, we couldpotentially learn a separate expert for each contact. However,we only consider a single expert as we want to demonstrateits generalization capabilities.

The contact model is learned using the data collectedfrom contact 1 and contact 3 (far and close contacts) and asvalidation the data set generated from the unseen contact 2(medium) is used. In Table II, the RMSE for all three contactsare reported for iDyn and our learned model, respectively.The results show that the learned model is robust to unseencontacts and performs equally well or better than the analyticmodel iDyn.

Method Shoulder 1 [Nm] Shoulder 2 [Nm] Elbow [Nm]

Far contact iDyn 0.13± 3.9× 10−3 0.40± 9.7× 10−3 0.06± 1.9× 10−3

Our model 0.06± 1.9× 10−3 0.08± 2.9× 10−3 0.03± 8.0× 10−4

Close contact iDyn 0.09± 2.2× 10−3 0.22± 4.5× 10−3 0.04± 0.9× 10−3

Our model 0.06± 1.4× 10−3 0.06± 1.4× 10−3 0.02± 6.3× 10−4

Medium contact iDyn 0.10± 2.8× 10−3 0.32± 6.7× 10−3 0.05± 1.3× 10−3

Our model 0.06± 1.7× 10−3 0.12± 4.7× 10−3 0.05± 1.7× 10−3

TABLE II: Robustness of the single contact model: Errorsbetween the ground truth (JTS) and the predictions witheither the iDyn and our learned model on the test set. Asingle expert is robust to small variations of the contact.

Method Shoulder 1 [Nm] Shoulder 2 [Nm] Elbow [Nm]

Right contact iDyn 0.10± 1.3× 10−3 0.13± 1.6× 10−3 0.06± 8.1× 10−4

Our model 0.04± 6.3× 10−4 0.07± 1.2× 10−3 0.02± 2.7× 10−4

Left contact iDyn 0.08± 1.2× 10−3 0.16± 2.0× 10−3 0.05± 8.2× 10−4

Our model 0.03± 5.7× 10−4 0.07± 9.6× 10−4 0.02± 2.8× 10−4

Both contacts iDyn 0.10± 1.3× 10−3 0.11± 1.4× 10−3 0.07± 8.4× 10−4

Our model 0.05± 8.3× 10−4 0.10± 1.6× 10−3 0.03± 4.0× 10−4

TABLE III: Learning multiple contacts: Root mean squareerror between the ground truth (JTS) and the predictions withthe iDyn and our learned model on the test set. Our learnedmodel predicts the torque more accurately than iDyn.

D. Learning multiple contacts

After learning single contacts, we now show how to com-bine the learned models to adapt to unseen and more complexenvironments with multiple sequential contacts. We considera scenario having the iCub performing a circular motion withits left arm. We initially performed two experiments with anobstacle either on the left and on the right of the referencetrajectory (see Fig. 8). With the data collected in these twocontact cases, we trained two independent expert models f1,f2, one for each contact. We repeated the experiment, butthis time with both left and right contacts and used this lastunseen case to validate our models. Fig. 9 shows an exampleof the prediction and the corresponding activation of the twocontact models. During both the right and the left contact, thecorresponding experts are activated by the gating network.Therefore, we can successfully combine the contributions ofthe single contact models learned to generalize to unseencases with multiple contacts. Table III reports the RMSEfor the predictions. We notice that even in this experimentthe experts accurately learn the effects of single contacts.Moreover, the gating network allows us to combine theexperts to generalize to unseen environments, such as in thecase of both contacts.

E. Learning the gating network

So far, we used a heuristic gating network to selectthe active experts fj . In this experiment we show that alearned gating network achieves a comparable accuracy as amanually devised heuristic. As ground truth to evaluate theperformances, as well as for training the classifier, we labeledthe data with one of the following labels: no contact, leftcontact, right contact. The heuristic is based on thresholdsof the activation of the skin input S and the force torquesensors F . We train a Support Vector Machine (SVM)classifier (using the library LIBSVM [3]) having as inputq,S,F and as output the contact labels (none, left, right).

We evaluated the performance of the trained classifier onan unseen test set. Fig. 10 shows that the learned SVMachieved a classification accuracy that is similar to the

Fig. 8: Learning multiplecontacts: The robot per-forms a circle with its leftarm. The forearm collidesalternatively with the left,the right or both contacts.

Fig. 9: Learning multiple con-tacts: Prediction of torqueswith multiple contacts and thecorresponding activation of thegating network. Our mixture-of-experts model combines thesingle-contact models into amultiple-contact model.

heuristic gating network. Equivalent results are obtained interms of RMSE of the inverse dynamics when comparingthe experts models learned by the gating networks. However,training the gating network (i.e., training the SVM classifier)requires considerably less expert knowledge compared todesigning a heuristic. Therefore, for more complex systemsan adaptive, data-driven approach may be more suitable.Additionally, since there is no visible performance differencein our experiments3, we conclude that training the gatingnetwork is generally preferable.

V. CONCLUSIONS

Whole-body control strategies that exploit contacts needaccurate models of the system dynamics. This is crucialfor balancing and stabilization, and to increase the numberof potential actions that the robot is able to execute, e.g.,creating a contact to reach for distant objects. We introduceda data-driven mixture-of-experts approach based on Gaus-sian processes for learning inverse dynamics models withcontacts. We evaluated our model on the iCub humanoidrobot using tactile sensors and force/torque sensors as modelinputs. We showed that the model accurately predicts contactforces and outperforms a state-of-the-art analytical approachused to estimate the joint torques in the iCub. The estimationfrom the learned model does not rely on dynamic parameters,but it is completely data-driven and based on tactile sensorsand force/torque sensors. As a result, our approach doesnot require a spatially calibrated model of the skin [6], [7].This is a promising feature for robust control strategies thatexplicitly takes contacts into account.

REFERENCES

[1] R. Calandra, N. Gopalan, A. Seyfarth, J. Peters, and M. P. Deisenroth.Bayesian gait optimization for bipedal locomotion. In LION8, 2014.

[2] G. Cannata, M. Maggiali, G. Metta, and G. Sandini. An embeddedartificial skin for humanoid robots. In MFI, 2008.

3Increasing the number of training data may further increase the accuracyof the gating network in our experiment.

Acc

ura

cy [

%]

No contact Contact left Contact right

Learned gating

Heuristic0

50

100

Fig. 10: Learning the gating network: Classification accu-racy for the heuristic and learned SVM gating networks.

[3] C.-C. Chang and C.-J. Lin. LIBSVM: A Library for Support VectorMachines. ACM Trans. Intell. Syst. Technol., 2(3):27:1–27, 2011.

[4] M. Deisenroth, D. Fox, and C. Rasmussen. Gaussian Processesfor Data-Efficient Learning in Robotics and Control. IEEE TPAMI,37(2):408–423, 2015.

[5] M. P. Deisenroth, R. Calandra, A. Seyfarth, and J. Peters. Toward fastpolicy search for learning legged locomotion. In IROS, 2012.

[6] A. Del Prete, S. Denei, L. Natale, F. Mastrogiovanni, F. Nori,G. Cannata, and G. Metta. Skin spatial calibration using force/torquemeasurements. In IROS, 2011.

[7] A. Del Prete, F. Nori, G. Metta, and L. Natale. Control of contactforces: The role of tactile feedback for contact localization. In IROS,2012.

[8] T. Erez and E. Todorov. Trajectory optimization for domains withcontacts using inverse dynamics. In IROS, 2012.

[9] R. Featherstone and D. Orin. Dynamics. In Springer Handbook ofRobotics, pages 35–65. 2008.

[10] M. Fumagalli, S. Ivaldi, M. Randazzo, L. Natale, G. Metta, G. Sandini,and F. Nori. Force feedback exploiting tactile and proximal force/-torque sensing - Theory and implementation on the humanoid robotiCub. Autonomous Robots, 33(4):381–398, 2012.

[11] S. Ivaldi, M. Fumagalli, M. Randazzo, F. Nori, G. Metta, and G. San-dini. Computing robot internal/external wrenches by means of inertial,tactile and F/T sensors: theory and implementation on the iCub. InHUMANOIDS, 2011.

[12] S. Ivaldi, J. Peters, V. Padois, and F. Nori. Tools for simulatinghumanoid robot dynamics: a survey based on user feedback. InHUMANOIDS, 2014.

[13] A. Jain, M. Killpack, A. Edsinger, and C. Kemp. Reaching in clutterwith whole-arm tactile sensing. IJRR, 32(4):458–482, 2013.

[14] S. Kajita and B. Espiau. Legged robots. In Handbook of Robotics,pages 361–389. Springer, 2008.

[15] L. Natale, F. Nori, G. Metta, M. Fumagalli, S. Ivaldi, U. Pattacini,M. Randazzo, A. Schmitz, and G. G. Sandini. The iCub platform:a tool for studying intrinsically motivated learning. In Intrinsicallymotivated learning in natural and artificial systems. 2013.

[16] M. Naveau, J. Carpentier, S. Barthelemy, O. Stasse, and P. Soueres.METAPOD - Template META-PrOgramming applied to Dynamics:CoP-CoM trajectories filtering. In HUMANOIDS, 2014.

[17] D. Nguyen-Tuong and J. Peters. Model learning for robot control: asurvey. Cognitive processing, 12(4):319–340, 2011.

[18] D. Nguyen-Tuong, J. Peters, and M. Seeger. Local Gaussian processregression for real time online model learning. In NIPS, 2008.

[19] Y. Ogawa, G. Venture, and C. Ott. Dynamic parameters identificationof a humanoid robot using joint torque sensors and/or contact forces.In HUMANOIDS, 2014.

[20] N. Pedrocchi, E. Villagrossi, F. Vicentini, and L. Tosatti. Robot-dyanmic calibration improvement by local identification. In ICRA,2014.

[21] C. E. Rasmussen and C. K. I. Williams. Gaussian Processes forMachine Learning. The MIT Press, 2006.

[22] B. Siciliano, L. Sciavicco, L. Villani, and G. Oriolo. Robotics:Modelling, Planning and Control. Springer, 2009.

[23] M. Toussaint and S. Vijayakumar. Learning discontinuities withproducts-of-sigmoids for switching between local models. In ICML,2005.

[24] S. Traversaro, A. Del Prete, R. Muradore, L. Natale, and F. Nori.Inertial parameter identification including friction and motor dynamics.HUMANOIDS, 2013.

[25] S. Vijayakumar and S. Schaal. Locally weighted projection regression:Incremental real time learning in high dimensional space. In ICML,2000.

[26] K. Yamane. Practical kinematic and dynamic calibration methods forforce-controlled humanoid robots. In HUMANOIDS, 2011.