experimental evaluation of pair programming copyright, 2001 © jerzy r. nawrocki european software...

Experimental Evaluation of Experimental Evaluation of Pair ProgrammingPair Programming

Experimental Evaluation of Experimental Evaluation of Pair ProgrammingPair Programming

Copyright, 2001 © Jerzy R. Nawrocki

European Software Control & MetricsEuropean Software Control & Metrics

ESCOM’01ESCOM’01

European Software Control & MetricsEuropean Software Control & Metrics

ESCOM’01ESCOM’01

Jerzy Nawrocki, Adam WojciechowskiPoznan University of TechnologyPoznan University of Technology

Poznan, PolandPoznan, Poland

J. Nawrocki, Experimental Evaluation ..

Plan of the lecturePlan of the lecturePlan of the lecturePlan of the lecture

IntroductionIntroductionPersonal Software ProcessPersonal Software ProcesseXtremme ProgrammingeXtremme ProgrammingDescription of the experimentDescription of the experimentResultsResultsConclusionsConclusions


IntroductionIntroductionIntroductionIntroduction

Pair programming

CreatorCreator Quality Quality assurerassurer

if (x=y)if (x=y) z=0;z=0;

Must beMust bex==yx==y



After some time..

CreatorCreatorQuality Quality assurerassurer

x-=y;x-=y;elseelse

How toHow totest it?test it?



Nosek’s experiment

Write a script that Write a script that performs a database performs a database consistency check.consistency check.




30 minutes

42 minutes

Completion time (aver.)Completion time (aver.)




71 %

100 %

Completion Completion timetime

EffortEffort

143 %

100 %



Main weakness

‘‘If several tasks each take an If several tasks each take an hour, combine them to form a hour, combine them to form a larger tasklarger task.’.’

Kent BeckKent Beck

Extreme Programming ExplainedExtreme Programming Explained

‘‘If several tasks each take an If several tasks each take an hour, combine them to form a hour, combine them to form a larger tasklarger task.’.’

Kent BeckKent Beck

Extreme Programming ExplainedExtreme Programming Explained

Only one short assignment (45’).Only one short assignment (45’).Only one short assignment (45’).Only one short assignment (45’).



The Utah experiment

1414

11

22

33

. . .. . .

1414

11

22

33

. . .. . .

4 programming assignments4 programming assignments

6 weeks6 weeks

4 programming assignments4 programming assignments

6 weeks6 weeks

50% - 60%

100%

Completion timeCompletion time



Weak points

1414

11

22

33

. . .. . .

1414

11

22

33

. . .. . .

What were the assignments What were the assignments about?about?

What was the process?What was the process?

What were the sizes?What were the sizes?

What was the time in hours?What was the time in hours?

What was the deviation in What was the deviation in time and size?time and size?

What were the assignments What were the assignments about?about?

What was the process?What was the process?

What were the sizes?What were the sizes?

What was the time in hours?What was the time in hours?

What was the deviation in What was the deviation in time and size?time and size?



About our experiment

Aim: experimental evaluation of pair programming.Aim: experimental evaluation of pair programming.

XP-like processXP-like process

PSP-like processPSP-like process

Fall semester 1999/2000;Poznan University of Technology, Poznan, PolandFall semester 1999/2000;Poznan University of Technology, Poznan, Poland



IntroductionIntroduction

Personal Software ProcessPersonal Software ProcesseXtremme ProgrammingeXtremme ProgrammingDescription of the experimentDescription of the experimentResultsResultsConclusionsConclusions


Personal Software ProcessPersonal Software ProcessPersonal Software ProcessPersonal Software Process

Time and defects are recorded.Defect type standard.

00

Coding standard. Size measurement.Process Improvement Proposal.

0.10.1

Software size estimation.Test reports.

11

Task planning.Schedule planning.

1.11.1

22Design templates.Code & design reviews.

2.12.1

Incremental approach 33


Personal Software ProcessPersonal Software ProcessPersonal Software ProcessPersonal Software Process

PlanningPlanning

DesigningDesigning

CodingCoding

CompilingCompiling

TestingTesting

PostmortemPostmortem

RequirementsRequirements

Product + Product + datadata



IntroductionIntroductionPersonal Software ProcessPersonal Software Process

eXtremme ProgrammingeXtremme ProgrammingDescription of the experimentDescription of the experimentResultsResultsConclusionsConclusions


eXtreme Programming (XP)eXtreme Programming (XP)eXtreme Programming (XP)eXtreme Programming (XP)

Pair programming.

Test-centred quality assurance.

Simple solution.

Spike solutions.

Keep moving.

Pair programming.

Test-centred quality assurance.

Simple solution.

Spike solutions.

Keep moving.

We applied:We applied:

User stories.

An on-site customer representative.

Planning game.

CRC cards.

Continuous integration

User stories.

An on-site customer representative.

Planning game.

CRC cards.

Continuous integration

We didn’t apply:We didn’t apply:



IntroductionIntroductionPersonal Software ProcessPersonal Software ProcesseXtremme ProgrammingeXtremme Programming

Description of the experimentDescription of the experimentResultsResultsConclusionsConclusions


Description of the experimentDescription of the experimentDescription of the experimentDescription of the experiment

XP-like pair XP-like pair programmingprogramming

XP2XP2

PSP 0.1 (time, PSP 0.1 (time, defect & size defect & size measurement)measurement)

PSPPSP

Test-centred Test-centred QA, Spike QA, Spike solutionssolutions

XP1XP1


Description of the experimentDescription of the experimentDescription of the experimentDescription of the experiment

Programming assignments

1. Estimate the mean and standard deviation of a sample of n real numbers.1. Estimate the mean and standard deviation of a sample of n real numbers.

2. Calculate the linear regression parameters.2. Calculate the linear regression parameters.

3. Count the logical lines in a program, omitting comments and blank lines.3. Count the logical lines in a program, omitting comments and blank lines.

4. Count the total program LOC, the total LOC in each object the program contains, and the number of methods in each object.

4. Count the total program LOC, the total LOC in each object the program contains, and the number of methods in each object.

C/C++C/C++



IntroductionIntroductionPersonal Software ProcessPersonal Software ProcesseXtremme ProgrammingeXtremme ProgrammingDescription of the experimentDescription of the experiment

ResultsResultsConclusionsConclusions


ResultsResultsResultsResults

Completion Time (Average)

0:00

1:12

2:24

3:36

4:48

Zad 1 Zad 2 Zad 3 Zad 4

PSP

XP1

XP2

There is almost no difference between XP2 and XP1.There is almost no difference between XP2 and XP1.

Prog 1 Prog 2 Prog 3 Prog 4



Completion Time (Std. Deviation)

0:00

0:28

0:57

1:26

1:55

2:24


PSP

XP1

XP2


Pair programming is more predictable than Pair programming is more predictable than individual one.individual one.



Average Program Size (LOC)

0

100

200

300

400

500


PSP

XP1

XP2




Standard Deviation in Code Size

0

50

100

150

200


PSP

XP1

XP2


Pair programming leads to more stable solutions.Pair programming leads to more stable solutions.


ConclusionsConclusionsConclusionsConclusions

• XP-like pair programming appears XP-like pair programming appears less efficient than it is reported by less efficient than it is reported by J.T. Nosek and L. Williams et al.J.T. Nosek and L. Williams et al.

• Pair programming is more Pair programming is more predictable one than individual one predictable one than individual one both in completion time and both in completion time and program size.program size.

• The experiment was restricted to The experiment was restricted to relatively small programs (150 - 400 relatively small programs (150 - 400 LOC).LOC).

experimental evaluation of pair programming copyright, 2001 © jerzy r. nawrocki european software...

Documents

personal software processj

total program loc

total loc

standard deviation

blank lines

logical lines

testcentred quality

fall semester