productivity of agile teams: an empirical evaluation of factors and monitoring processes

51
PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES CSBC 2014 – XXVII CTD Dr. Claudia Melo Advisor: Prof. Dr. Fabio Kon Department of Computer Science, IME-USP

Upload: claudia-melo

Post on 08-Sep-2014

627 views

Category:

Technology


1 download

DESCRIPTION

Presenting my thesis during the National Thesis Contest in Computer Science - top 6 PhD Computer Science Thesis in Brasil/ 2013. XXXIV Congresso da Sociedade Brasileira de Computação (CSBC 2014) - CTD.

TRANSCRIPT

Page 1: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF

FACTORS AND MONITORING PROCESSES CSBC 2014 – XXVII CTD

Dr. Claudia Melo

Advisor: Prof. Dr. Fabio Kon

Department of Computer Science, IME-USP

Page 2: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

OUTLINE

Context & Definitions

Problem statement

Research questions

Research methods & Overview of all studies

Results & Implications for research and industry

Other contributions

3

Page 3: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

GRAND CHALLENGE – AN INSTANCE

UK government case Savage, M.: Labour’s Computer Blunders cost 26bn. The Independent, Tuesday 19 January, London (2010)

4

Page 4: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

UNCOMFORTABLE TRUTH

We are still not very good at software engineering.

§  inability to deal with change requests in the requirements;

§  failure to communicate between the developers and stakeholders;

§  no clear requirements definitions. Anthony J H Simons and W Michael L Holcombe, Vision Paper: Remodelling Software Systems – the 2020 Grand Challenge for

Software Engineering

5

Page 5: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

6

SOFTWARE PRODUCTIVITY IS A GENERAL STRATEGIC CONCERN IN SEVERAL

INDUSTRIAL SECTORS

Page 6: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

WORK HAS CHANGED IN THE 21ST CENTURY

Optimization Mechanistic Process centric Stable, predictable Individual Efficiency

Adaptation Organic

People centric Turbulent, difficult to predict

Team Knowledge work

Productivity = Output/Input Productivity = ?

7

Page 7: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

“THE MOST VALUABLE ASSET OF A 21ST CENTURY INSTITUTION WILL BE ITS KNOWLEDGE WORKERS AND THEIR PRODUCTIVITY” PETER DRUCKER, 1999

8

Page 8: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

KNOWLEDGE WORKER PRODUCTIVITY

9 Y. W. Ramírez and D. A. Nembhard, “Measuring knowledge worker productivity: A taxonomy,” Journal of Intellectual Capital, 2004. 9

Quantity

Quality

Efficiency

Effectiveness

Timeliness

Profitability

Responsibility

Autonomy

Customer Satisfaction

Creativity/Innovation Project Success

Knowledge Worker

Productivity

Continuous life-long learning

Page 9: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

Agile Manifesto: values and principles Scrum, XP, Lean software development, Feature Driven Development, DSDM,

Crystal etc. 10

Page 10: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

“The appearance of Agile methods has been the most noticeable change to software process thinking in the last

fifteen years” Fowler M. (2005). The New Methodology,

www.martinfowler.com.

“Agile methods rapidly joined the mainstream of development

approaches” Forrester Research 2010. Agile development: Mainstream

adoption has changed agility - trends in real-world adoption of agile methods. Technical report, January.

11

Page 11: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

OPEN PROBLEMS

Productivity definition in the 21st century in the context of software development, particularly agile teams?

•  E.g: Riding the paradox between flexibility and efficiency; Re-thinking studies around productivity

Recent studies discuss productivity factors

•  None considers factors impacting agile teams.

•  To manage productivity effectively it is important to identify the most relevant difficulties and develop strategies to cope with them.

High-performance self-organized teams should continuosly monitor their own performance

•  How agile teams can monitor their own productivity?

•  How to consider adaptability in this monitoring?

12

Page 12: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

NATURE OF THE PROBLEMS

Social context and technological content are essential to a proper understanding of the software development.

Mismatch between current used productivity definitions and actual productivity in the 21st century

•  Sometimes paradoxical

Productivity measurement is hard. KW worker productivity might be extremely hard to measure

As a complex system, there are many possible factors influencing productivity, it is hard to interpret

•  Triangulation of data sources, methods, theories and researchers is necessary

Highly embedded in the industry context

•  Manage risks of partnership with industry

13

“the most important figures that one needs for management are

unknown or unknowable, but successful

management must nevertheless take account

of them.” W. E. Deming (1986)

Out of Crisis.

Page 13: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

RESEARCH QUESTIONS

RQ1. How important is productivity for companies adopting agile methods and how do they define productivity?

RQ2: What factors impact agile team productivity and how is this impact from the team point of view? Which agile practices are perceived to impact on a given team’s productivity?

RQ3: How to monitor productivity factors, considering agility and adaptability? How do agile metrics support productivity monitoring?

14

Page 14: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

OVERVIEW OF ALL STUDIES

Multiple-case studies on Agile team productivity definitions and

agile team productivity factors2010-2011

Surveyon Agile productivity

expectationsand benefits

2011-2012

Action research forexploring and assessing

productivity monitoring and measurement in agile teams

2012-2013

Warm-upstudies on Agile

methods impact on productivity

2009-2010

Study of software productivity

definitions, factors and metrics2009-2010

Study of Agile productivity metrics and performance monitoring, measurement

dysfunctions, and monitoring in self-managed teams

2011-2012

Phase I Phase II Phase III

P1,P2

P4, P6, P7, P9

P3

IR1

P5, P8

IR2

Research study

Paper

Industry report

In collaboration with Norwegian University of Science and Technology

August 2009 March 2013

15

Page 15: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

RESEARCH METHODS

•  Quantitative studies by exploring the importance of productivity for agile teams and related context

•  Qualitative studies by exploring and explaining factors and monitoring approaches

•  Sometimes using quantitative data

•  The rationale for mixed methods has been:

•  Triangulation

•  It is useful to take benefit from all available data

•  Answering questions that are not possible to answer otherwise

16 16

Page 16: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

Web-based survey in Brazil (May, 2011 – October, 2011).

Organizations adopting agile methods to develop software.

■  Industry and Universities.

■  471 respondents, 17 states

Exploratory research using non-probabilistic sampling

Snowball sampling.

Convenience sampling.

■  Mailing lists, attendees of past agile conferences, and Agilcoop business contacts.

Statistical analysis: descriptive and inferential

Open data, Replication.

17 17

SURVEY

Page 17: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

MULTIPLE-CASE STUDIES – RESEARCH METHOD

3 large Brazilian companies (> 250 employees) : Financial, Cloud computing/data center, Internet content and services

3 types of data sources (~6 months collecting data): Semi-structured interviews (19), Retrospective sessions documentation, Observation field notes

Data analysis and synthesis method:

•  Cross-case analysis, data source/theory/researcher triangulations

•  Thematic analysis1 and Thematic Networks2

•  Data-driven approach (Inductive)

18 1 Boyatzis, R. E., 1998. Transforming qualitative information: thematic analysis and code development. Sage Publications. 2Attride-Stirling, J., Dec. 2001. Thematic networks: an analytic tool for qualitative research. Qualitative Research 1 (3), 385–405.

18

Page 18: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

19 19 19

NVivo 9

•  6-Month Interviews transcribed in 400 pages + Observation notes + Artifacts

•  Research and data source triangulation, incrementally analyzed in 12 months

•  Conceptual framework developed in the first months. Updated in the last months.

ANALYZING QUALITATIVE DATA

Page 19: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

ACTION RESEARCH METHOD

10-month action research •  Strongly oriented toward collaboration and change (researchers & subjects).

•  Iterative research process

•  Solve practical problems while expanding scientific knowledge

•  Capitalizes on learning by both researchers and subjects within the context of the subjects’ social system

Multinational company

Distributed project on B2C

National research team

Multi-method data collection, Triangulation

Thematic analysis

20

Page 20: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

11 FINDINGS 21

Page 21: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

22

KEY FINDING 1: PRODUCTIVITY AS AN IMPORTANT REASON FOR ADOPTING AGILE METHODS

count

Ch

am

pio

n

Developer

Team leader

Project manager

Development manager

CIO/CTO

President/CEO

Development director

26.3%

23.6%

13.8%

11.5%

10.4%

8.3%

6.2%

0 50 100 150

count

Wo

rrie

s

Lack of documentationLack of predictability

Lack of upfront planningLoss of management control

Lack of team trainingDevelopment team opposed to change

Lack of engineering disciplineRegulatory compliance

Reduced software qualityNo concerns

Inability to scale

51%43.5%

41%37.6%

34.8%32.1%

25.7%24.8%

21.2%12.5%11.9%

0 50 100 150 200 250

Percentage

Re

aso

ns

for

ad

op

ting

Ag

ile

Accelerate time to market

Enhance ability to manage changing priorities

Enhance software maintainability extensibility

Enhance software quality

Improve alignment between IT and business

Improve engineering discipline

Improve project visibility

Improve team morale

Increase productivity

Reduce cost

Reduce risk

Simplify development process

73%

86%

66%

83%

72%

59%

65%

64%

91%

47%

69%

80%

27%

14%

34%

17%

28%

41%

35%

36%

9%

53%

31%

20%

0 20 40 60 80 100

Response

Highest importance

Very important

Somewhat important

No important at all

MELO, C. O.; SANTOS, V. A.; CORBUCCI, H.; KATAYAMA, E.; GOLDMAN, A.; KON, F. Agile methods in Brazil: state of the practice in teams and organizations (in Portuguese). Technical Report MAC-2012-03. Department of Computer Science IME-USP. May, 2012. http://agilcoop.org.br/MetodosAgeisBrasil2011. CORBUCCI, H. ; GOLDMAN, A. ; KATAYAMA, E. ; KON, F. ; MELO, C. O. ; SANTOS, V. S.. Genesis and Evolution of the Agile Movement in Brazil – A perspective from the Academia and the Industry. In: Proceedings of 25th Brazilian Symposium on Software Engineering (SBES), 2011, pp. 98-107.

Page 22: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

23

KEY FINDING 2: PRODUCTIVITY AS PERCEIVED BENEFIT FROM ADOPTING AGILE METHODS

23

MELO, C. O. ; KATAYAMA, E.; SILVA, V. S.; CORBUCCI, H.; PRIKLADNICKI, R. GOLDMAN, A.;KON, F. Genesis and Evolution of the Agile Movement in Brazil – A perspective from the Academia and the Industry. Journal of Brazilian Computer Society 19(4):523-552 (2013).

Page 23: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

KEY FINDING 3: REASONS AND PERCEPTION OF PRODUCTIVITY

WHEN ADOPTING AGILE METHODS ARE NOT ASSOCIATED

WITH THE COMPANY SIZE NOR EXPERIENCE WITH AGILE

(SPEARMAN’S RANK-ORDER - rho - CORRELATION TEST)

MELO, C. O. ; KATAYAMA, E.; SILVA, V. S.; CORBUCCI, H.; PRIKLADNICKI, R. GOLDMAN, A.;KON, F. Genesis and Evolution of the Agile Movement in Brazil – A perspective from the Academia and the Industry. Journal of Brazilian Computer Society 19(4):523-552 (2013).

24

Page 24: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

KEY FINDING 4: AGILE PRACTICES ADOPTED BY

COMPANIES PERCEIVING SIGNIFICANTLY IMPROVED PRODUCTIVITY

ARE

ITERATION PLANNING, RETROSPECTIVES,

UNIT TESTING, DAILY STANDUP, AND

REFACTORING

MELO, C. O. ; KATAYAMA, E.; SILVA, V. S.; CORBUCCI, H.; PRIKLADNICKI, R. GOLDMAN, A.;KON, F. Genesis and Evolution of the Agile Movement in Brazil – A perspective from the Academia and the Industry. Journal of Brazilian Computer Society 19(4):523-552 (2013).

25

Page 25: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

26

E.g.: Timeliness, Quantity (~traditional productivity definition), Quality, Customer satisfaction

KEY FINDING 5: THE DEFINITION OF AGILE TEAM PRODUCTIVITY IS DIFFUSE

MELO, C. O. ; CRUZES, D. S. ; KON, F. ; CONRADI, R. Agile Team Perceptions of Productivity Factors. In: Proceedings of the Agile Development Conference (AGILE), Salt Lake City, USA, 2011, pp. 57-66.

Page 26: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

27

KEY FINDING 6: AGILE TEAM PRODUCTIVITY FACTORS ARE STRONGLY RELATED TO TEAM

MANAGEMENT

MELO, C. O. ; CRUZES, D. S. ; KON, F. ; CONRADI, R. Interpretative Case Studies on Agile Team Productivity and Management. Information & Software Technology 55(2): 412-427 (2013).

Page 27: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

28

Page 28: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

29

KEY FINDING 7: PAIR PROGRAMMING AND COLLOCATION AS KEY PRACTICES

MELO, C. O. ; CRUZES, D. S. ; KON, F. ; CONRADI, R. Agile Team Perceptions of Productivity Factors. In: Proceedings of the Agile Development Conference (AGILE), Salt Lake City, USA, 2011, pp. 57-66.

Page 29: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

30

KEY FINDING 8: NEW MOTIVATORS MIGHT INFLUENCE AGILE TEAM PRODUCTIVITY

e.g.: Challenging work, participation, sense of contribution and progress.

MELO, C. O. ; SANTANA, C.; KON, F. Developers motivation in agile teams. 38th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), Çesme, Izmir, 2012, p. 376-383.

Page 30: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

I1. Group characteristicsTeam design (e.g., team size, collocation, team, diversity)Team member turnoverBeliefs

I2. Stage of team development

I3. Nature of task(e.g., task design, task duration, team autonomy, interdependency)

I4. Organizational context

I5. Supervisory behaviors(e.g., transactional versus transformational, degree of supervision – directive or self-managed teams)

G1. Internal and External processes(e.g., Cohesion, Communication, Conflict management, Coordination, Sharing of expertise, Work procedures)

Inputs

O1. Agile team productivity outcomes (Team's perception on dimensions of productivity, e.g., Customer satisfaction, quantity of work, innovation, creativity, timeliness, product quality, absenteeism, profitability, and team efficiency and effectiveness)

O2. Attitudinal and Behavioral outcomes

Outcomes

Group processes

AGILE TEAM PRODUCTIVITY CONCEPTUAL FRAMEWORK

31

Fig. 1: IPO traditional team framework adapted from Cohen & Bailey [6] and Marks et al. [7]

Page 31: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

UPDATED AGILE TEAM PRODUCTIVITY CONCEPTUAL FRAMEWORK

Inputs OutcomesGroup processes

Conflict management

Agile team productivity

Stage of team development

Member turnover

Sharing of (new) expertise

Intrateam coordination

Agile practices establisment(work procedures)

Agile practices establisment(work procedures)

Group characteristics Team design Personality

Behavioral outcomes Member turnover

Team

mem

bers

turn

over

Agile team productivity

Sharing of expertiseIntra team coordination

Conflict managementIntra team coordinationCommunication

Group characteristics Team design

Attitudinal outcomes Commitment

Team

des

ign

choi

ces

Small teams

Diversity (mixed teams)

Full-time allocation

CollocationCommunicationCohesionPlanning/RE negotiation(work procedures)

Intra team coordination

Agile team productivity

Nature of task Team autonomy/ interdependency

Attitudinal outcomes (lack of) CommitmentIn

ter t

eam

coo

rdin

atio

n

Inter team coordination

Agile practices establisment(work procedures)

Inte

r tea

m m

anag

emen

tIn

tra

team

man

agem

ent

MEL

O, C

. O. ;

CRU

ZES,

D. S

. ; K

ON

, F. ;

CO

NRA

DI,

R. In

terp

reta

tive

Case

Stu

dies

on

Agile

Tea

m

Prod

uctiv

ity a

nd M

anag

emen

t. In

form

atio

n &

Softw

are

Tech

nolo

gy 5

5(2)

: 412

-427

(201

3).

32

Page 32: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

33 33

Diagnosing

Typical day

- daily meeting- update task board- update burndown and selected metrics (if applicable)

Agile project - Release n

Retrospective...

Action Planning Action Taking Evaluating Specify learning

1. Appreciate

problem situation

through:

- Focus groups - Problem solving template - Self-assessment questionnaire - Researcher Immersion and previous knowledge of the company

2. Study

literature:

- Productivity metrics for the context

3. Select solution

approach through:

- Focus groups

4. Develop solu-

tion framework:

- Data collection method and frequency - Tools

5. Apply approach

6. Monitor

through:

- Observation - Informal meetings - Project events

7. Evaluate

experiences

through:

- Focus groups - Informal meetings

Pla

nnin

g

8. Assess metrics

usefulness:

- Focus groups - Interviews - Self-assessment questionnaire

9. Elicit research

results

releases

Page 33: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

CYCLE 0: JUN, 2012 – AUG, 2012

Monitoring productivity:

•  Process: efficiency and speed

•  Product: timeliness

Learning outcomes:

•  Productivity definitions between client and team were misaligned

•  Disfunctional measurement

•  Confirming that agile team productivity was an issue (action research principle)

34

34

Page 34: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

CYCLE 1: SEP, 2012 – DEC, 2012

Monitoring productivity:

•  Personnel: anti-patterns related to trust and motivation

•  Product: quality

Learning outcomes:

•  Productivity monitored through qualitative measurement (patterns identification)

•  Actions on trust and motivation prevented staff turnover (confirming our previous conceptual framework)

35 35

Page 35: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

CYCLE 2: JUNE, 2012 – AUG, 2012

Monitoring productivity:

Process: Leanness/Flow

Product: Quality

Personnel: Teamwork

Learning outcomes:

•  Teamwork assessment generates insights for teamwork improvement

•  Metrics/Charts have both positive and negative aspects for productivituy monitoring

36

Page 36: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

37

KEY FINDING 9: PRODUCTIVITY MONITORING INSTRUMENT

37 37

Dimension Goal How to monitor

Product [Quality, Innovation, etc.]

Personnel [Teamwork, Trust, Motivation etc.]

Project [Speed, Scope etc.]

Process [Leanness, Efficiency, etc.]

Organizational [Inter-team coordination etc

Actions Evaluation

•  5 dimensions, from personnel to organizational aspects •  Incorporating Knowledge worker productivity aspects •  Light approach that can be incorporated by agile teams

Page 37: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

38

KEY FINDING 10: A FRAMEWORK FOR DEVELOPING AGILE TEAM PRODUCTIVITY MONITORING

38

Design

Identifying key

monitoring

goals

Monitor

and

Measure

Review

Act

Implementation Assessing Challenging

Identifying/

Developing

[qualitative or

quantitative]

measures

Implementation of

monitoring and

measurement

Reflect

DiagnosingAction

PlanningAction Taking Evaluating

Specifying

Learning

Developing Agile team monitoring approaches from a Practical perspective

Developing Agile team monitoring approaches from a Theoretical perspective

Dynamically review of

targets, measures,

and goals

MEL

O, C

. O. P

rodu

ctiv

ity a

nd a

dapt

abili

ty o

f agi

le te

ams:

leve

ragi

ng th

e pa

rado

x to

war

ds

inno

vatio

n (in

Por

tugu

ese,

to a

ppea

r), In

: Ant

olog

ia T

houg

htW

orks

Bra

sil. C

asa

do C

ódig

o. 2

014.

Page 38: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

KEY FINDING 11: PRODUCTIVITY METRICS USEFULNESS

§  No overhead

§  Productivity metrics might help just some groups of team members

§  Metrics might drive learning and change §  But sometimes people need guidance to enable learning

§  It was not always clear why or when some metrics were introduced

39 39 39

Page 39: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

CONTRIBUTION FOR THE SPECIFIC PROJECT

§  In this particular instance:

§  Project initially under cancellation threat

§  Project recovery

§  Project became a business case for the client

40 40 40

Page 40: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

CONTRIBUTIONS AND RESEARCH QUESTIONS

Contribution RQ Related papers (P), Technical reports (IR), and Book Chapters (CH)

C1. Empirical verification of the importance of productivity for companies adopting agile, and perceived benefits.

RQ1 P5, P8 IR2 CH1

C2. Rationale on productivity definition in agile methods context.

RQ1 P3, P4, P6, P7 IR1

C3. Empirical verification of agile team productivity factors.

RQ2 P4, P7

C3.1. Motivators in agile teams RQ2 P9

C4. A framework of agile team productivity factors and their impact, to be tested.

RQ2 P7

C5. A case on team productivity monitoring process considering adaptability and evaluation of agile team productivity metrics’ usefulness.

RQ3 P10 CH2

41

Page 41: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

WHY SUCH STUDIES ARE IMPORTANT For research community and industry

42

Page 42: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

43

“Closing the gap between research and practice by encouraging a stronger emphasis on methodological rigor while focusing on relevance for practice”

Barbara A. Kitchenham, Tore Dyba, and Magne Jorgensen. 2004. Evidence-Based Software Engineering. In Proceedings of the 26th International Conference on Software Engineering (ICSE '04). IEEE Computer Society, Washington, DC, USA, 273-281.

Page 43: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

44

Anna Sandberg, Lars Pareto and Thomas Arts. Agile collaborative research: Action principles for industry-academia collaboration. IEEE Software, 28(4):74–83, 2011

Better researcher by working on relevant problems,

better practitioner by identifying and applying scientific methods

Page 44: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

PAPERS, INDUSTRY REPORTS, AND BOOK CHAPTERS

P1. MELO, C. O. ; FERREIRA, G. R. M. Adopting Agile in a Large Government Institution – a case study (in Portuguese). In: Workshop Brasileiro de Métodos Ágeis (WBMA), Conferência Brasileira sobre Métodos Ágeis de Desenvolvimento de Software (Agile Brazil 2010). Porto Alegre. p. 104-117.

P2. MELO, C. O. ; SANTOS Jr., C. D. ; FERREIRA, G. R. M. ; KON, F. An exploratory study of factors associated with learning in agile teams on industry (in Portuguese). Proceedings of 7th Experimental Software Engineering Latin American Workshop, 2010, Goiânia.

P3. MELO, C. O. ; KON, F. Empirical evaluation of agile practices impact on team productivity. In: 12th International Conference on Agile Software Development (XP), Doctoral Symposium, Madrid, 2011, pp. 322-323.

P4. MELO, C. O. ; CRUZES, D. S. ; KON, F. ; CONRADI, R. Agile Team Perceptions of Productivity Factors. In: Proceedings of the Agile Development Conference (AGILE), Salt Lake City, USA, 2011, pp. 57-66.

P5. CORBUCCI, H. ; GOLDMAN, A. ; KATAYAMA, E. ; KON, F. ; MELO, C. O. ; SANTOS, V. S. Genesis and Evolution of the Agile Movement in Brazil – A perspective from the Academia and the Industry. In: Proceedings of 25th Brazilian Symposium on Software Engineering (SBES), 2011, pp. 98-107.

P6. MELO, C. O. ; KON, F. Productivity of agile teams (in Portuguese). Software Engineering Magazine, Brazil, v. 43, p. 1 - 9, 05 dez. 2011.

P7 MELO, C. O. ; CRUZES, D. S. ; KON, F. ; CONRADI, R. Interpretative Case Studies on Agile Team Productivity and Management. Information & Software Technology 55(2): 412-427 (2013).

P8 MELO, C. O. ; KATAYAMA, E.; SILVA, V. S.; CORBUCCI, H.; PRIKLADNICKI, R. GOLDMAN, A.;KON, F. Genesis and Evolution of the Agile Movement in Brazil – A perspective from the Academia and the Industry. Journal of Brazilian Computer Society 19(4):523-552 (2013).

45

Page 45: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

PAPERS, INDUSTRY REPORTS, AND BOOK CHAPTERS (CONT.) P9 MELO, C. O. ; SANTANA, C.; KON, F. Developers motivation in agile teams. 38th Euromicro Conference on Software

Engineering and Advanced Applications (SEAA), Çesme, Izmir, 2012, p. 376-383.

P10 MELO, C. O.; KON, F. Agile team productivity monitoring: it is all about learning. In preparation for the Information and Software Technology.

IR1 MELO, C. O. ; KON, F. Productivity Factors in Agile teams - an exploratory study in Brazilian Companies (in Portuguese). March, 2012.

IR2 MELO, C. O.; SANTOS, V. A.; CORBUCCI, H.; KATAYAMA, E.; GOLDMAN, A.; KON, F. Agile methods in Brazil: state of the practice in teams and organizations (in Portuguese). Technical Report MAC-2012-03. Department of Computer Science. IME-USP. May, 2012. Available at: http://agilcoop.org.br/MetodosAgeisBrasil2011.

CH1 GOLDMAN, A; MELO, C. O. ; KON, F.; CORBUCCI, H.; SANTOS, V. The History of Agile Methods in Brazil (in Portuguese), Chapter 2, In: Métodos Ágeis Para Desenvolvimento De Software. Bookman, 2014.

CH2 MELO, C. O. Productivity and adaptability of agile teams: leveraging the paradox towards innovation (in Portuguese, to appear), In: Antologia ThoughtWorks Brasil. Casa do Código. 2014.

46 46 46

Page 46: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

RELATED RESEARCH WORK

Conference Papers

OLIVEIRA, R. M. ; MELO, C. O. ; Goldman, A . Designing and Managing Agile Informative Workspaces: Discovering and Exploring Patterns. In: 2013 46th Hawaii International Conference on System Sciences (HICSS), p. 4790-4799, Wailea.

§  Nominated for the best paper award (http://www.hicss.hawaii.edu/hicss_46/bp46/bestpapersnoms1219.pdf)

TAKEMURA, C. ; MELO, C. O. Studying agile organizational design to sustain innovation. In: Agile Brazil, 2012, São Paulo. Proceedings of the III Brazilian Workshop on Agile Methods (WBMA 2012), 2012. p. 13-24.

SOUSA, T. C. ; MELO, C. O. . Generating Fit acceptance tests from B Specifications (in portuguese). In: IV Workshop de Desenvolvimento Rápido de Aplicações do Simpósio Brasileiro de Qualidade de Software (WDRA-SBQS), 2010, Belém. Anais do Workshop de Desenvolvimento Rápido de Aplicações do Simpósio Brasileiro de Qualidade de Software, 2010. v. 1. p. 1-8.

Book chapters

Bertholdo, Ana Paula O. ; da Silva, Tiago Silva ; de O. Melo, Claudia ; KON, FABIO ; Silveira, Milene Selbach. Agile Usability Patterns for UCD Early Stages. Lecture Notes in Computer Science, 2014, v. 8517, p. 33-44.

Silva, Tiago Silva ; Silveira, Milene Selbach; O. Melo, Claudia; Parzianello, Luiz Claudio. Understanding the UX Designer s Role within Agile Teams. Lecture Notes in Computer Science, 2013, v. , p. 599-609.

47

Page 47: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

CONTRIBUTIONS TO THE COMMUNITY

Graduate course “MAC5779 Engenharia de Software Experimental”, with Professor Marco Aurélio Gerosa

§  Course proposal

§  Course design and content

§  Course lecturing

Our dataset was used to support a Master Thesis (Eng. Produção, Poli-USP) §  Student José Henrique Dell'Osso Cordeiro, Advisor Prof. Afonso Carlos Correa Fleury, Title: “Ambidestria

em empresas desenvolvedoras de software: barreiras para adoção de metodologias ágeis e seu impacto na escolha do modelo organizacional”. Defended on June/2014.

Invited to be part of the “Supporting Agile Adoption: It's About Change” group, supported by the Agile Alliance. ■  Only participant from the Global South

■  Published content available http://www.agilealliance.org/programs/supporting-agile-adoption-it-is-about-change/

48

Page 48: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

CONTRIBUTIONS TO THE COMMUNITY (2)

Presentations

•  MELO, C. O. . Agilidade no Brasil: Fatos e Mitos. Agile Trends 2013. São Paulo

•  MELO, C. O. . O segredo é a confiança: criando melhores times, com ou sem distância. Agile Brazil 2013. Brasília. MELO, C. O. . Introdução a Métodos Ágeis de Desenvolvimento de Software. Caixa Econômica Federal. 2012.

•  KATAYAMA, E. ; GOLDMAN, A. ; MELO, C. O. Uma introdução ao Desenvolvimento de Software Lean. Invited short course. SBQS 2012.

•  MELO, C. O. Lean Lego Game. Invited workshop. SBQS 2012.

•  SOUSA, F. ; MELO, C. O. ; COLUCCI, T. ; CUKIER, D. Lean startups - Curso de Verão no IME-USP. 2012.

•  MELO, C. O. ; SANTANA, C. ; GOLDMAN, A. ; KON, F. A Primeira Década com Métodos Ágeis: desafios atuais e evidências encontradas. CBSOFT 2011.

49

Page 49: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

Number of possible studies in different areas (Computer Science, Organizational & Management Science, Social Science)

§  Testing the agile team productivity factors framework through confirmatory studies. Replication.

§  Explaining the role of adaptability on team productivity factors.

§  Further exploring metrics driving learning and change.

§  Exploring productivity drivers in agile companies.

Possible partnership with Programa Brasileiro da Qualidade e Produtividade em Software

§  Just 1-2 Brazilian studies cited.

FUTURE RESEARCH

50

Page 50: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

51 51 51

ACKNOWLEDGMENTS

Page 51: PRODUCTIVITY OF AGILE TEAMS: AN EMPIRICAL EVALUATION OF FACTORS AND MONITORING PROCESSES

OBRIGADA Questions?

[email protected] @claudia_melo