scalable product line configuration - ase 2013 palo alto, ca
TRANSCRIPT
Scalable Product Line Configuration:A Straw to Break the Camel’s Back
Abdel Salam SayyadJoseph IngramTim MenziesHany Ammar
Sound bites
Real Software Product Lines…… are big, complex
… representative feature models are now available for research
… scale up or die !
2
What does it take?Multi-objective problem formulationAn optimizer that relies heavily on user preferencesRespect domain constraints, and...
… A Straw to Break the Camel’s Back
Roadmap
① State of the Art② Experiment & Results③ Conclusion
Roadmap
① State of the Art② Experiment & Results③ Conclusion
Feature Models
• Feature models = a lightweight method for defining a space of options
• De facto standard for modeling variability
5
Cross-Tree Constraints
Cross-Tree Constraints
Size ? 10 Features, 8 Rules
Feature Model Repository
6
Size ? 290 Features, 421 Rules
cardinal = 2.26x1049 *Pohl ‘11+
Feature Models of Real Software Projects
7
Size ? 6888 Features; 344,000 Rulescardinal? Forgeddaboutit!
Size isn’t all…
Property SPLOT Linux
Size Significantly smaller Significantly larger
Constraints Significantly less Significantly more
Feature Groups High ratio Low ratio
Leaves Deeper Shallower
8
T. Berger, S. She, R. Lotufo, A. Wasowski, and K. Czarnecki, "A Study of Variability Models and Languages in the Systems Software Domain," IEEE Tran Soft Eng. 2013
A Multi-Objective Formulation
Multi-objective optimization = navigating competing concerns
– Success criteria = choose features that achieve the user preferences!
9
Suppose each feature had the following metrics:1. Boolean USED_BEFORE?2. Integer DEFECTS3. Real COST
Show me the space of “best options” according to the objectives:1. That satisfies most domain constraints (0 ≤ #violations ≤ 100%)2. That offers most features3. That we have used most before4. Using features with least known defects5. Using features with least cost
No single “optimum” solution
10
Higher-level Decision Making
The Pareto Front
The Chosen Solution
State of the Art
11
Features
9
290
544
6888
SPLO
TLi
nu
x (L
VA
T)
Pohl ‘11 Lopez-Herrejon
‘11
Henard‘12
Sayyad ’13a
Olaechea‘13
Sayyad ’13bJohansen
‘11
Benavides ‘05
White ‘07, ‘08, 09a, 09b, Shi ‘10, Guo ‘11
Objectives
Multi-objectiveSingle-objective
Roadmap
① State of the Art② Experiment & Results③ Conclusion
Two Algorithms
13
1) NSGA-II [Deb et al. ‘02+Non-dominated Sorting Genetic Algorithm
2) IBEA *Zitzler and Kunzli ‘04] Indicator-Based Evolutionary Algorithm
Binary Dominance
Continuous Dominance
7 Feature Models
14
Feature Fixing
• Look for mandatory or dead features.
• Fix those feature in the evolution.
• Skip rules with only those features.
15
Experiment
16
Low Crossover & Mutation rates
Stopping Criteria
17
HV = Hypervolume of dominated region%Correct = % of solutions that are fully correct (out of 300 solutions)
Comparing %Correct
18
0%
20%
40%
60%
80%
100%
ToyBox axTLS eCosFreeBSD Fiasco
uClinuxLinux X86
NSGA-II No FF
NSGA-II with FF
IBEA No FF
IBEA with FF
i.e. % of solutions that are fully correct (out of 300 solutions)
How to crack the nut?
• What if we told IBEA what good solutions look like?
19
• Good solution? … A correct solution, with:• Minimal skeleton?• Maximum features?
• Then what?• Seed the initial population with a pre-calculated
“good” solution.
20
• How to generate good (correct) solutions?• Z3 SMT (Satisfiability Module Theory) Solver• Two-objective IBEA, min violations, max features
• Z3: super fast, low on selected features
• 2-obj IBEA: slow, but feature-rich
Correct solutions after 30 minutes for Linux Kernel feature model
21
Correct solutions after 30 minutes for Linux Kernel feature model
22
… A Straw to Break the Camel’s Back
More recent result• added the “PULL” trick (give more objective weight
to constraint violations)
23
Why is this interesting?• 20 correct solutions to choose from within 2 minutes
24
Do I have the user’s attention?
• 2 seconds…
• to stay focused
25
• 10 seconds… • to stay on task
• few minutes? • To provide
timely input at a meeting
Roadmap
① State of the Art② Experiment & Results③ Conclusion
Sound bites
ScalabilityMethod innovation is key
27
Problem FormulationGive the best (comprehensive yet concise) picture to the user.
IBEA with Feature FixingContinuous dominanceRespect domain constraints
Enter Z3?One day we will build that bridge!
Acknowledgment
This research work was funded by the Qatar
National Research Fund (QNRF) under the National Priorities Research Program
(NPRP) Grant No.: 09-1205-2-470.