product comparison matrix (pcm), variability modeling: the wikipedia case study
DESCRIPTION
Product comparison matrices (PCMs) provide a convenient way to document the discriminant features of a family of related products and now abound on the internet. Despite their apparent simplicity, the information present in existing PCMs can be very heterogeneous, partial, ambiguous, hard to exploit by users who desire to choose an appropriate product. Variability Models (VMs) can be employed to formulate in a more precise way the semantics of PCMs and enable automated reasoning such as assisted configuration. Yet, the gap between PCMs and VMs should be precisely understood and automated techniques should support the transition between the two. In this paper, we propose variability patterns that describe PCMs content and conduct an empirical analysis of 300+ PCMs mined from Wikipedia. Our findings are a first step toward better engineering techniques for maintaining and configuring PCMs.TRANSCRIPT
From Comparison Matrix to Variability Model The Wikipedia Case Study Presented at Automated Software Engineering (ASE’13) conference
Product Author
First Name Last Name Age Nat. Ph.D.? Posit. Affil. Spoken Lang
Nicolas Sannier 32 French soon PhD Student, Future
Postdoc?
Inria French, English, Reunion Isl. creole
Mathieu Acher 29 French Yes Associate Prof.
University of Rennes
1, Inria, IRISA
French, English
Benoit Baudry - French Yes Research Scientist, Head of
Triskell team
Inria French, English
Product Comparison Matrix (PCM)
Product Author
First Name Last Name Age Nat. Ph.D.? Posit. Affil. Spoken Lang
Nicolas Sannier 32 French soon PhD Student, Future
Postdoc?
Inria French, English, Reunion Isl. creole
Mathieu Acher 29 French Yes Associate Prof.
University of Rennes
1, Inria, IRISA
French, English
Benoit Baudry - French Yes Research Scientist, Head of
Triskell team
Inria French, English
Compare and Choose your Product Speaker
Delayed with a cancelled plane
Not available
ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 3
ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 4
ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 5
#1 This is a Product Comparison Matrix (PCM)
ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 6
#2 This is a Product Comparison Matrix (PCM)
ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 7
#3 This is a Product Comparison Matrix (PCM)
Promises of Product Comparison Matrices (PCMs)
ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 8
+ Intuitive and easy to understand + Convenient for comparing, input for configuring + Rich source of information and knowledge
ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 9
- Heterogeneous information - As the PCM grows up
“more is less”
- Lack of Formalization - No Automated Support
- Guidance capabilities - Ad-hoc PCMs
Issues and Challenges Product Comparison Matrices (PCMs)
ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 10
- Heterogeneous information - As the PCM grows up
“more is less”
- Lack of Formalization - No Automated Support
- Guidance capabilities - Ad-hoc PCMs
Issues and Challenges Product Comparison Matrices (PCMs)
Understanding PCMs
ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 11
RQ1 What kind information is presented in PCMs?
Syntax? Semantics? Variability patterns?
RQ2 What is the gap between PCMs and Variability Models?
Related work • Extensive work on spreadsheets
• But PCMs are specific spreadsheets • Reverse engineering variability models
• Other artefacts (She et al. ICSE’11, Czarnecki et al. SPLC’07, Abbas et al. CSMR’14) • Boolean PCMs (Haslinger et al. FASE’13, Davril et al. FSE’13)
ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 12
• Open community • One of the most important analyzable repository of PCMs
• 300+ PCMs
• Multiple domains, multiple concerns, large PCMs
The Wikipedia case study
ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 13
• #0 Extraction of all 381 Wikipedia pages entitled “Comparison of …”
• #1 A preliminary analysis of some PCMs (variability patterns definition)
• #2 A Qualitative analysis of randomly selected 50 PCMs
• #3 A Quantitative analysis of all extracted Wikipedia PCMs
Research Methodology
Qualitative Analysis: 8 Variability Patterns
ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 14
1. Boolean yes/no answers 2. Partial/constrained yes/no answers
3. Single-value answers 4. Multiple values answers
5. “Unknown” answers 6. Empty cells
7. Inconsistent cells 8. Additional / Extra information
Automatically Analysis of 300+ Wikipedia PCMs
ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 15
Variability Patterns: Quantitative Results
ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 16
• Results • 75-80% of the PCMs content is manageable as usual by variability constructs • 20-25% remaining represent uncertainty or numerical values
• Calls for more research for modeling and reasoning about variability
Information type 1 2 3 4 5 6 7 8 Qualitative analysis 47.29% 3.71% 22.75% 4.37% 10.86% 4.83% 0.55% 5.64%
Quantitative analysis 49.4 % 0.8% 20.4% 15.1% 7.5% 6.8% - -
1. Boolean yes/no answers 2. Partial/constrained yes/no answers 3. Single-value answers 4. Multiple values answers 5. “Unknown” answers 6. Empty cells 7. Inconsistent cells 8. Additional / Extra information
Research Directions Bridging the Gap between Product Comparison Matrices (PCMs) and Variability Models (VMs)
ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 17
Contributors (writers) No more ad-hoc PCMs PCMs should be easier to create and maintain Hopefully a non intrusive solution, interoperable with Wikipedia
End users (readers) Manageable information
Better readability Better services
Developers (readers and writers) Enabling analysis tools of PCMs
(e.g., synthesis of variability models) Long term, more global vision:
Generating product comparators and configurators from variability models and PCMs Note: VMs act as a formal representation of PCMs and intermediates before devising configurators/comparators/*
VMs are not an end-user solution to visualize the PCM
Compare and Choose your Answerer! PCM-driven of course
ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 18
Look at the Author Comparison Matrix and Choose
Ask your questions ;-)
Product Author
First Name Last Name Age Nat. Ph.D.? Posit. Affil. Spoken Lang
Nicolas Sannier 32 French soon PhD Student, Future
Postdoc?
Inria French, English, Reunion Isl. creole
Mathieu Acher 29 French Yes Associate Prof.
University of Rennes
1, Inria, IRISA
French, English
Benoit Baudry - French Yes Research Scientist, Head of
Triskell team
Inria French, English
Delayed with a cancelled plane
Not available