sensitivity analysis for building adaptive robotic software

SENSITIVITY ANALYSIS FORBUILDING ADAPTIVE ROBOTIC SOFTWAREPooyanJamshidi,MiguelVelezandChristianKästner

INTENT DISCOVERY: SENSITIVITY ANALYSIS FOR CONFIGURATION OPTIMIZATION

REDUCING COSTS WITH TRANSFER LEARNING

USE CASES

Systematic System Evolution To automate or guide intelligent design choices.

Runtime Adaptation To enable runtime adaptation of software configurations

to maintain quality of performance under dynamic

conditions (changing environment, goals, and tasks).

Performance Debugging To guide robot software developers to identify potential

bugs causing low quality of performance.

RESULTS

Motivation:Robotic software expose configurable parameters.

These tunable parameters affect performance of robots.

This can be leveraged to optimize performance.

Source Response Target Response

Transfer learning combines:

Lots of data gathered cheaply from

the simulator

With much less data gathered

expensively from the target robot

To make better predictions overall

PUBLICATIONSP. Jamshidi, M. Velez, C. Kästner, N. Siegmund, and P. Kawthekar. Transfer learning for improving model predictions in highly configurable software. Int’l Symp. Software Engineering for Adaptive and Self-Managing Systems (SEAMS), 2017. P. Kawthekar and C. Kästner. Sensitivity analysis for building evolving & adaptive robotic software, Workshop on Autonomous Mobile Service Robots (WSF), 2016.

Predictive Model

Learn Model

MeasureMeasure

DataSourceTarget

Simulator (Gazebo) Robot (TurtleBot)

Predict Performance

Predictions

Adaptation

Use for analysis

5 10 15 20 25number of particles

5

10

15

20

25

nu

mb

er

of

refin

em

en

ts

0

5

10

15

20

25

5 10 15 20 25number of particles

5

10

15

20

25

nu

mb

er

of

refin

em

en

ts

0

5

10

15

20

25

5 10 15 20 25number of particles

5

10

15

20

25

nu

mb

er

of

refin

em

en

ts

0

5

10

15

20

25CPU usage [%] CPU usage [%]

(a) (b)

(c) (d)

Prediction without transfer learning

5 10 15 20 25

5

10

15

20

25

10

15

20

25

Prediction with transfer learning

Using only a few real data points to predict yields poor results across configuration space

Using transfer learning to combine the few real data points with lots of approximate data yields a good model

Machine Learning

Configuration Parameters

Design of Experiment

Configuration Space

Predictive Model

Sensitivity Analysis

DataMeasurements

Configuration Space

Data

AccuracyEnergy

CPU

0 5 10 15 20 25 30 35mean CPU utilization

0

500

1000

1500

2000

2500

3000

3500

num

ber

of c

onfig

urat

ions

5 10 15 20 25number of particles

5

10

15

20

25

nu

mb

er

of

refin

em

en

ts

6

8

10

12

14

16

18

20

22

24

26

5 10 15 20 25number of particles

5

10

15

20

25

num

ber

of re

finem

ents

5

10

15

20

25

30

35

40

45

CPU

Localisation Error