Online Design Discussion

Rami AkkadFU BerlinOpen Source Seminar - WS 2008/2009Tutor: Christopher Oezbek

Note: All graphics used in this presentation are taken from the focused paper „A socio-cognitive analysis of online design discussions in an Open Source Software community“, F. Barcellini et al. | Interacting with Computers 20 (2008)

2

Contents

1. Introduction

2. Theoretical Framework / Research strategy

3. Methods

4. Results

5. Summary

6. Discussion

Introduction - OSS

• Open Source Software is a constantly growing market

• Widely used

• Projects like Linux, Mozilla or Apache well known

• An OSS community is geographically distributed and asynchronous

• Problems are discussed online in discussion spaces • Mailing lists• Forums• Wikis

3

Introduction – Python project

• The study focuses on the Python project community

• Project has less than 100 developers involved

• Produced code is reasonable

• It has all qualities common OSS projects have

• Past studies made by colleagues focused on python

• They had work the could rely on

4

5

Contents

1. Introduction

2. Theoretical Framework / Research strategy

3. Methods

4. Results

5. Summary

6. Discussion

Theoretical Framework / Research Strategy

• Five different types of participants are recognized:

1. The project leader

2. The core team or administrators

3. Developers

4. Active users

5. Passive users

The status of a participant may improve during online

disucssions.

6

Situation at the time of study (2002)

• Core group of seven people:

– Project leader– Six administrators

• Discussion space seperated into three design spaces:

– Documentation Space– Discussion Space– Implementation Space

7

Situation at the time of study (2002)

8

Python Enhancement Proposal (PEP)

• The PEP process is used to discuss and handle design related proposals

• It protocols everything related to a proposal

• A PEP proposer is called „champion“

9

Python Enhancement Proposal (PEP)

• Example:

1. Champion sends PEP to an editor

2. Editor checks usability of the PEP.

If confirmed, PEP will be published.

3. PEP will be discussed.

4. PEP is approved by the community.

5. Project leader gives his agreement.

6. Code for PEP will be written and implemented into the

Python language.

10

Quotation

• Quoting techniques are used in order to maintain context

• A message quoting another message is seen as a response to the message quoted

• Quotation is widley distributed in the Python project discussion space

11

Three aspects of online discussion dynamics

• The study focuses on the discussoin space, specifically on the python-dev mailing list

• Three aspects of online discussion dynamics are analyzed:

1. Social dynamics

2. Thematic temporal dynamics

3. Design dynamics

• A combination of strutural and content analyses of the online discussions is used to obtain the result data

12

Social dynamics

• Characterizes the influence of participants in design discussions

– How do they evolve?

• Relationship between the status of the posters and their involvment in the discussion based on:

– The number of posted messages– The type of quotation structure– The depth of a quotation– The position in the discussion

13

Thematic temporal dynamics

• Show how design themes emerge and evolve over time

• Design themes adressed by each messages are analyzed

– Provides characterization of organization and temporal emergence of themes in discussions

• Degree of synchronism/asynchronism of messages will be calculated

14

Design dynamics

• Characterizes how collaborative design work leads to the final design

• Which sequences of activities / which discussion led to the final design?

• Messages containing posts with design-alternatives were chosen

• This is a content-based analysis

15

16

Contents

1. Introduction

2. Theoretical Framework / Research strategy

3. Methods

4. Results

5. Summary

6. Discussion

Collection of data

• PEP discussions are all archieved and published online

• At the time of study, 161 PEPs were discussed

• The python-dev mailing list has been searched (by hand) for PEP-related discussion

• 13 PEPs have been found, 8 PEPs are design-related

• Among those, 3 have been just created and were chosen to capture the core of the design process

17

Collection of data

• Two of these three PEPs were compared: PEP 279 and PEP 285

• Both had the same status (accepted and implemented)

• Both were at the same stage in the design process

• Both adressed the introduction of new functionality

18

PEP 279 / PEP 285

• PEP 279 was proposed by a developer

• PEP 285 was proposed by the project leader

they differed in the social position of their respective champion

• PEP 279: 71 messages posted by 21 authors; 3600 lines of text

• PEP 285: 95 messages posted by 22 authors; 3500 lines of text

• PEP processes were compared by analyzing one main discussion for each PEP

19

Data collection: Social view

• Four types of participants were identified in the python-dev mailing list:

1. Project leader (PL)

2. Champion of the PEP (C)

3. Administrators (A)

4. Developers (D)

20

Data collection: Social view

• Involvement of a participant in the online discussion is categorized by a median number of messages posted:

More than two messages posted:

– High participant administrator (HP-A) – High participant developer (HP-D)

Two or fewer messages posted:

– Low participant administrator (LP-A)– Low participant developer (LP-D)

21

Data collection: Social view

Quotation• A line is recognized as a quotation if it starts with a „>“,

for example:

xxx writes:

>So now I’d like to choose between enumerate() and indexer(). Any

>closing arguments?

“indexer” is the name of the built-in full-text indexer, right? ;-)

– Three types of messages:• Text only messages(no quotation)• Single quote messages(one block of quotation followed by a comment)• Multiple quote messages(alternating quotes and comments)

example

22

Data collection: Social view

• Quotation patterns:

– Sequential structure: a message is quoted by only one message

– Branching structure: a message is quoted by two or more messages

– Closing structure: a message is not quoted at all

23

Data collection: Thematic temporal view

• Themes adressed for each message were identified

• There were five themes in the PEP 279 discussion

• And six themes in the PEP 285 discussion

24

Data collection: Thematic temporal view

• PEP 279 discussion themes:

(T1): Choice of a name

(T2): Alternative syntaxes for the proposed function

(T3): Utility of the proposed function

(T4): Error handling

(T5): Orthogonal problem of name binding / status of name spaces (technical issue)

25

Data collection: Thematic temporal view

• PEP 285 discussion themes:

(T1): Consequences of a new build-in data type

(T2): Specific function (return type)

(T3): Name of the constant of the new build-in type

(T4): Elimination of non-boolean operations on booleans

(T5): Specific operator of Python and its return value

(T6): Inheritance relationship between two types

T1 is a “meta-theme”: Possible impacts this change could have are discussed within the community.

26

Data collection: Design view

Design alternatives

• Design alternatives proposed in both PEPs were analyzed

Proposed design alternatives:

– PEP 279/T1: 23– PEP 279/T2: 8– PEP 279/T3: 3

– PEP 285/T1-T6: 3

27

Data collection: Design view

Design activities

• Seven different design activities were identified from the content of messages:

1. Proposal

2. Evaluation

3. Group coordination

4. Synthesis

5. Clarification

6. Explicit Decision

7. Other activities

28

29

Contents

1. Introduction

2. Theoretical Framework / Research strategy

3. Methods

4. Results

5. Summary

6. Discussion

Results – Social viewDistribution of statuses and involvement in the discussions

30

Results – Social viewQuotation activities

31

Results – Social viewPosition of participants and their used quotation structure

32

Results – Social viewPosition of participants and their used quotation structure

33

Results – Social viewPosition of participants and their used quotation structure

34

Results – Social viewPosition of participants and their used quotation structure

35

• PEP 279:

Branching structure PL or C

Sequental structure D

Closing message PL

• PEP 285:

Branching structure PL or LP-D

Sequental structure HP-D

Closing message LP-A or HP-A

Results – Social viewDepth of quotation

36

Results – Thematic temporal view

37

Results – Thematic temporal view

38

Small degree of asynchronism

Results – Design viewDesign alternatives

39

• Branching structures result from messages containing more than one design alternative proposals

• Closing messages include one or zero alternative proposals

• In PEP 279 discussion alternatives were posted mostly by– C– PL– LP-Ds

C and PL are very important in the design process

Results – Design viewDesign activities in comments

40

Results – Design viewDesign activities in quotes

41

Results – Design viewSequences of activities

42

• Which activities are followed by which activity?

• Characterization of the way information and knowledge is gathered in the community

Results – Design viewSequences of activities

43

a) Evaluation Evaluation

b) Proposal Evaluation or proposal

c) Clarification Evaluation or proposal

d) Coordination Coordination

e) Decision Coordination

44

Contents

1. Introduction

2. Theoretical Framework / Research strategy

3. Methods

4. Results

5. Summary

6. Discussion

Summary – Social dynamics

45

• Project Leader has the absolute authority over the project– Messages posted by the PL lead to multiple branches– Messages posted by the PL usually contain multi-quotes

• Only the Champion writes synthesis to gather the information of previous posts

• Messages posted by administrators occured in three different positions– At a beginning branching position– In linear exchanges with developers– In closing positions (acting as the PL when he was not

available)

Summary – Social dynamics

46

• Messagess posted by developers occurred in two different positions– In PEP 285 at a beginning branching position– In PEP 279 in linear sequences of exchanges with other

participants

Developers are actively participating in the design process

Summary – Thematic temporal dynamics

47

• Quotation technique used is based on selective quoting– Lines neccesary to preserve context are chosen

• The timeframe patricipants react to a proposal or a message is different– If a message is considered important, the answer usually

apperas within hours– Certain moments exist in a design-oriented, distributed

online-discussion where communication can be seen as synchronous

– Implicit social rules exist in the community; if a certain issue is not answered by the adressed participant “in time”, he could lose his social status within the community

Summary – Design dynamics

48

• The two most important activities are evaluation and clarification

• Few decisions are taken during the online discussions– But those taken are explicit

• A participants status does not affect the design process he is engaged in

• Patterns of activity interchange do not differ to face-to-face discussions

49

Questions?

50

Contents

1. Introduction

2. Theoretical Framework / Research strategy

3. Methods

4. Results

5. Summary

6. Discussion

Discussion

51

• Are the three analyzed dynamic aspects wise?

• Is it wise, to focus only on one project?

• Is it wise, to focus only on two PEPs in this project and among them only on one discussion?

• Is further research essential to provide a representative study which characterizes OSS discussions in all of its aspects?

52

Thank you!