deep parameters tuning for android mobile apps
TRANSCRIPT
Deep Parameters Tuning for Android Mobile Apps
Federica SarroUniversity College
London
Davide De ChiaraUniversity of
Salerno
Filomena FerrucciUniversity of
Salerno
Mark HarmanUniversity College
London
Yue JiaUniversity College
London
Functional Requirements
Non-Functional Requirements
Why Genetic Improvement for Non-Functional Properties
Humans have to define these
A machine can optimise these
Traditional Genetic Improvement Process
Program
Program
Traditional Genetic Improvement Process
Program GP
Traditional Genetic Improvement Process
Program
Test Cases Fitness
GP
Traditional Genetic Improvement Process
Program
Test Cases Fitness
GPProgram’
Traditional Genetic Improvement Process
Program
Test Cases Fitness
GPProgram’
Traditional Genetic Improvement Process
Previous Work: Deep Parameter Optimisation
Program
Tuneable Parameters
e.g GCC -O3 foo.c Fan Wu, Westley Weimer, Mark Harman, Yue Jia, and Jens Krinke.
Deep Parameter Optimisation, Gecco 2015
Not Many Tuneable Parameters
Limited Power
Previous Work: Deep Parameter Optimisation
Program
Tuneable Parameters
Fan Wu, Westley Weimer, Mark Harman, Yue Jia, and Jens Krinke.
Deep Parameter Optimisation, Gecco 2015 e.g GCC -O3 foo.c
Many Software Systems contain undocumented internal variables or expressions that also affect the
performance of the systems
Previous Work: Deep Parameter Optimisation
Program
Tuneable Parameters
Fan Wu, Westley Weimer, Mark Harman, Yue Jia, and Jens Krinke.
Deep Parameter Optimisation, Gecco 2015
Program
Tuneable Parameters
Tuneable Implicit Parameters
Approach
Previous Work: Deep Parameter Optimisation
Program
Tuneable Parameters
Fan Wu, Westley Weimer, Mark Harman, Yue Jia, and Jens Krinke.
Deep Parameter Optimisation, Gecco 2015
Mutation analysis based approach to expose deep parameters
C Programs and dlmalloc library
12% on time, 21% on memory
Previous Work: Deep Parameter Optimisation
Fan Wu, Westley Weimer, Mark Harman, Yue Jia, and Jens Krinke.
Deep Parameter Optimisation, Gecco 2015
Android Applications
Our Approach
Mutation Testing
Expose Deep Parameters
Optimisation
App App’
Test Cases Performance
Our Approach
Mutation Testing
Expose Deep Parameters
Optimisation
App App’
Test Cases Performance
Mutation Testing
Expose Deep Parameters
Optimisation
App App’
Test Cases Performance
Our Approach
Mutation Operators
Operator Description Original line Mutation example
ICR Inline Constant Replacement const/4 v0, 0x5 const/4 v0, 0x6
UOI Unary Operator Insertion neg-int v2, v2 neg-int v2, v2 neg-int v2, v2
LCR Logical Connector Replacement if-nez v1, :cond_2 if-eqz v1, :cond_3
AOR Arithmetic Operator Replacement add-int/lit8 v0, v0, 0x1 div-int/lit8 v0, v0, 0x1
ROR Relational Operator Replacement if-nez v0, :cond_0 if-eqz v0, :cond_0
RVR Return Value Replacement return-object v0 return-void
Android Mutation Testing tool: MuDroid
Mutation Testing
Expose Deep Parameters
Optimisation
App App’
Test Cases Performance
Our Approach
pseudo equivalent mutants
Our Approach
Mutation Testing
Expose Deep Parameters
Optimisation
App App’
Test Cases Performance
Our Approach
{Our Approach
Arithmetic expression
Logical expression
Given a Location L, and EL is the expression at L
Exposing Deep Parameters
EL(EL + vL)(EL) xor vL
Our Approach - Random
const/4 v0, 0x7
const/16 v0, 0xa
if-nez v1, :cond_2
const/4 v0, 0x7
Our Approach - Grid Search
const/4 v0, 0x7
const/16 v0, 0xa
if-nez v1, :cond_2
const/4 v0, 0x7
01
-1-2
2
Mutation Testing
Expose Deep Parameters
Optimisation
App App’
Test Cases Performance
Our Approach
Mutation Testing
Expose Deep Parameters
Optimisation
App App’
Test Cases Performance
Our Approach
Empirical Study
Empirical Study Design - Goal
Does deep parameters tuning allow us to improve mobile app performance?
Empirical Study Design - Goal
Does deep parameters tuning allow us to improve mobile app performance?
Execution time Cpu time Memory allocated
Empirical Study Design
Loc # Tests Tests Coverage Class Method Block Line Description
A Time Tracker 3453 12 100% 86% 92% 91% Time tracking for any tasks
FillUp 13344 12 100% 85% 90% 92% Fuel consumption tracking
GM Dice 1325 12 100% 85% 91% 89% A dice rolling for role playing game system
Text Edit 3079 8 100% 99% 93% 93% Notepad application
Sandwich Roulette 1560 6 100% 98% 98% 98% Automates the ingredient for Sandwiches
World Clock 1902 6 100% 97% 96% 95% World Clocks
Validation & Evaluation Criteria
10 Deep Parameters
10 Executions
Summary Statistics and Statistical Test
Repeat 10 times each approach for each non-functional properties
Best 10 parameters for each non-functional properties
2 Evaluation Criteria
Results - Random vs Original
Execution Time
6 win 0 tie
0 loss
Wilcoxon Signed-Rank Test
Cpu Time
6 win 0 tie
0 loss
Memory Allocated
2 win 2 tie
2 loss
Execution Time
6 win 0 tie
0 loss
Wilcoxon Signed-Rank Test
Cpu Time
6 win 0 tie
0 loss
Memory Allocated
3 win 2 tie
1 loss
Results - Grid Search vs Original
Results - Grid Search vs Random
Memory AllocatedWorst Case Best Case
min max avg min max avgRandom -2% 0% -1% 0% 3% 1%
Grid Search -3% 2% -1% 0% 6% 3%
Cpu TimeWorst Case Best Case
min max avg min max avgRandom 5% 20% 15% 6% 23% 17%
Grid Search 0% 14% 7% 9% 22% 17%
Execution TimeWorst Case Best Case
min max avg min max avgRandom 0% 3% 2% 1% 12% 4%
Grid Search 0% 2% 1% 2% 13% 6%
Results - Grid Search vs Random
Memory AllocatedWorst Case Best Case
min max avg min max avgRandom -2% 0% -1% 0% 3% 1%
Grid Search -3% 2% -1% 0% 6% 3%
Cpu TimeWorst Case Best Case
min max avg min max avgRandom 5% 20% 15% 6% 23% 17%
Grid Search 0% 14% 7% 9% 22% 17%
Execution TimeWorst Case Best Case
min max avg min max avgRandom 0% 3% 2% 1% 12% 4%
Grid Search 0% 2% 1% 2% 13% 6%
Execution Time Cpu Usage Memory Allocated Hours Days
Random 20.4 17.4 17.2 55 2,3
Grid Search 168.4 161.8 53.6 384 16
Results - Grid Search vs Random
Results - Improvement
23% CPU Time
6% Memory Allocated
13% Execution Time
