product comparison matrix (pcm), variability modeling: the wikipedia case study

From Comparison Matrix to Variability Model The Wikipedia Case Study Presented at Automated Software Engineering (ASE’13) conference

Product Author

First Name Last Name Age Nat. Ph.D.? Posit. Affil. Spoken Lang

Nicolas Sannier 32 French soon PhD Student, Future

Postdoc?

Inria French, English, Reunion Isl. creole

Mathieu Acher 29 French Yes Associate Prof.

University of Rennes

1, Inria, IRISA

French, English

Benoit Baudry - French Yes Research Scientist, Head of

Triskell team

Inria French, English

Product Comparison Matrix (PCM)

Product Author

First Name Last Name Age Nat. Ph.D.? Posit. Affil. Spoken Lang

Nicolas Sannier 32 French soon PhD Student, Future

Postdoc?

Inria French, English, Reunion Isl. creole

Mathieu Acher 29 French Yes Associate Prof.

University of Rennes

1, Inria, IRISA

French, English

Benoit Baudry - French Yes Research Scientist, Head of

Triskell team

Inria French, English

Compare and Choose your Product Speaker

Delayed with a cancelled plane

Not available

ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 3

ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 4

ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 5

#1 This is a Product Comparison Matrix (PCM)

ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 6

#2 This is a Product Comparison Matrix (PCM)

ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 7

#3 This is a Product Comparison Matrix (PCM)

Promises of Product Comparison Matrices (PCMs)

ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 8

+ Intuitive and easy to understand + Convenient for comparing, input for configuring + Rich source of information and knowledge

ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 9

- Heterogeneous information - As the PCM grows up

“more is less”

- Lack of Formalization - No Automated Support

- Guidance capabilities - Ad-hoc PCMs

Issues and Challenges Product Comparison Matrices (PCMs)

ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 10

- Heterogeneous information - As the PCM grows up

“more is less”

- Lack of Formalization - No Automated Support

- Guidance capabilities - Ad-hoc PCMs

Issues and Challenges Product Comparison Matrices (PCMs)

Understanding PCMs

ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 11

RQ1 What kind information is presented in PCMs?

Syntax? Semantics? Variability patterns?

RQ2 What is the gap between PCMs and Variability Models?

Related work •  Extensive work on spreadsheets

•  But PCMs are specific spreadsheets •  Reverse engineering variability models

•  Other artefacts (She et al. ICSE’11, Czarnecki et al. SPLC’07, Abbas et al. CSMR’14) •  Boolean PCMs (Haslinger et al. FASE’13, Davril et al. FSE’13)

ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 12

•  Open community •  One of the most important analyzable repository of PCMs

•  300+ PCMs

•  Multiple domains, multiple concerns, large PCMs

The Wikipedia case study

ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 13

•  #0 Extraction of all 381 Wikipedia pages entitled “Comparison of …”

•  #1 A preliminary analysis of some PCMs (variability patterns definition)

•  #2 A Qualitative analysis of randomly selected 50 PCMs

•  #3 A Quantitative analysis of all extracted Wikipedia PCMs

Research Methodology

Qualitative Analysis: 8 Variability Patterns

ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 14

1.  Boolean yes/no answers 2.  Partial/constrained yes/no answers

3.  Single-value answers 4.  Multiple values answers

5.  “Unknown” answers 6.  Empty cells

7.  Inconsistent cells 8.  Additional / Extra information

Automatically Analysis of 300+ Wikipedia PCMs

ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 15

Variability Patterns: Quantitative Results

ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 16

•  Results •  75-80% of the PCMs content is manageable as usual by variability constructs •  20-25% remaining represent uncertainty or numerical values

•  Calls for more research for modeling and reasoning about variability

Information type 1 2 3 4 5 6 7 8 Qualitative analysis 47.29% 3.71% 22.75% 4.37% 10.86% 4.83% 0.55% 5.64%

Quantitative analysis 49.4 % 0.8% 20.4% 15.1% 7.5% 6.8% - -

1.  Boolean yes/no answers 2.  Partial/constrained yes/no answers 3.  Single-value answers 4.  Multiple values answers 5.  “Unknown” answers 6.  Empty cells 7.  Inconsistent cells 8.  Additional / Extra information

Research Directions Bridging the Gap between Product Comparison Matrices (PCMs) and Variability Models (VMs)

ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 17

Contributors (writers) No more ad-hoc PCMs PCMs should be easier to create and maintain Hopefully a non intrusive solution, interoperable with Wikipedia

End users (readers) Manageable information

Better readability Better services

Developers (readers and writers) Enabling analysis tools of PCMs

(e.g., synthesis of variability models) Long term, more global vision:

Generating product comparators and configurators from variability models and PCMs Note: VMs act as a formal representation of PCMs and intermediates before devising configurators/comparators/*

VMs are not an end-user solution to visualize the PCM

Compare and Choose your Answerer! PCM-driven of course

ASE'2013 - Sannier, Acher and Baudry - From PCM to VM. - 18

Look at the Author Comparison Matrix and Choose

Ask your questions ;-)

Product Author

First Name Last Name Age Nat. Ph.D.? Posit. Affil. Spoken Lang

Nicolas Sannier 32 French soon PhD Student, Future

Postdoc?

Inria French, English, Reunion Isl. creole

Mathieu Acher 29 French Yes Associate Prof.

University of Rennes

1, Inria, IRISA

French, English

Benoit Baudry - French Yes Research Scientist, Head of

Triskell team

Inria French, English

Delayed with a cancelled plane

Not available