test in g - gesellschaft für informatikfg-tav.gi.de/fileadmin/user_upload/tav34/koite_paper.pdf ·...

Tes t in g in A La r g e sc a l e

a g il e Dev el o pm en t En v ir o n m en t

Myth or reality?

Testing in large scale Agile Development | Ericsson Internal | © Ericsson AB 2013 | 2013-01-28 |

1. Introduction 2. Agile on a large scale 3. Testing in a large scale agile environment 4. Conclusion

Co n t e n t /a g e n d a


› Ericsson: – The leading vendor for mobile network infrastructure

› Tough competition on the telecommunication market: – Number of competitors – Merging of traditional Circuit Switched Technology and Packet

Switched Technology towards all IP Network

› Motivation: – Increase the feedback loops towards design – Delivery of high quality products to the customer – Being flexible to adapt faster to the market’s demand – Early and Frequent Delivery

In t r o d u c t io n


AGILE ON A LARGE SCALE


› Manifesto for Agile Software Development › We are uncovering better ways of developing

software by doing it and helping others do it. Through this work we have come to value:

– Individuals and interactions over processes and tools – Working software over comprehensive documentation – Customer collaboration over contract negotiation – Responding to change over following a plan

› That is, while there is value in the items on the right, we value the items on the left more.

Ag il e

Testing in large scale Agile Development | Ericsson Internal | © Ericsson AB 2013 | 2013-01-28 | Page 8

Sc RUM IN A NUTSHELL › One team collaborating to one feature


– Chief Product Owner: 1 – Number of Product owners: 5 – Number of Scrum teams: 8 – Size of the teams: 6-8 – Number of Scrum Masters: 8 – Number of people: around 70 – Number of locations: 4 – Number of Releases: 2 – Number of manhours: ~100000 – Number of User Stories: ~700

l a r g e s c a l e


› Our legacy: The Ericsson Mobile Soft Switch

› 8085042 PLEX statements › 20212606 ASA instructions corresponding to

approximately 95000000 Intel instructions › 2196 SW blocks

LARGE SCALE


TESTING IN A LARGE SCALE AGILE

ENVIRONMENT


› Cross functional team of 7 +/- 1 SW developers: › Virtual teams › Synchronized sprints inside a feature › Common Backlog › Common ceremonies to keep everybody informed

Wa y s o f w o r k in g


Wa y s o f Wo r k in g

Team A Team B Team C Team D

Backlog Setup strictly based on functional areas

Application

Transport

Internet

Network

US1

US3 US2

US4

US5


Wa y s o f Wo r k in g Backlog Setup strictly based on functional areas


Application

Transport

Internet

Network

US1

US3 US2

US4

US5

› Risk of dependencies between User Stories

› Difficulties to test from an “end to end” perspective

› More planning needed › Prone to generate delays




Backlog Setup with end to end focus and functional areas

Application

Transport

Internet

Network

US1

US3

US2

US4

US5


Wa y s o f Wo r k in g Backlog Setup with end to end focus and functional areas


Application

Transport

Internet

Network

US

1

US

3

US

2

US

4

US

5

› Testing possible from “end to end” perspective as the User Story is concluded

› Less planning needed › Reduce dependencies due

to functional areas › Competence build through

training on the job and mix of competences




Virtual Test Team

System Architecture Team

LSV team

Scrum of Scrum

Setup of virtual teams




Virtual Test Team

System Architecture Team

LSV team

Scrum of Scrum

Setup of virtual teams

› Overview over the activities with higher priority – Testing, Architecture, Software Delivery, Organisational topics

Testing in large scale Agile Development | Ericsson Internal | © Ericsson AB 2013 | 2013-01-28 | Page 19 © Ericsson AB 2009 | Ericsson Internal | X (X) | Date

Te s t Ov e r v ie w

Rebase mandatory

Rebase optional Note: Simplified view, details on following slides

LSV

LLV

trac

k

Feature n

LLV

trac

k

Feature 3

LLV

trac

k

Feature 2

LLV

trac

k

Feature 1

1 2 3 4 5 6 7 8 N N+1 M

LSV Delivery


Te s t in g a c t iv it ie s

Functional Tests Examples & Story Tests

Unit Tests System Module Tests

Exploratory Testing Usability Testing

(User Acceptance Tests)

Performance & Load Testing Security Testing

“ility” Testing

Q2 Q1

Q3 Q4

Unit Test

Function Test Integration

Function Test Integration

Verification

Automated

Automated & Manual Manual

Tools Technology facing

Supp

ortin

g th

e Te

am

Critique Product

1: taken from Crispin/Gregory: Agile Testing, Addison Wesley, 2009

Business facing


› Q1 Kind of tests are – Unit tests – testing small pieces of code – System Module Tests – test that units work together correctly

› Q2 kind of tests are – Functional & Integration tests – positive and deterministic negative

tests, if possible to automate – User Story Acceptance Tests



› Q3 Kind of tests – Exploratory Testing – feedback into stories – Feature Test and Regression Test with

Background Traffic – (early) Characteristics

› Q4 Kind of tests – Dimensioned Load on LLV/LSV – High Load/Overload on LSV (most likely) – Parafunctional tests (Performance, stress) – Interoperability, Security



› Detailed view on Testing Timeframe

TEST De t a il e d v ie w

Main

Feature1

Feature2

Team A

Team B

Team C

Team D

LSV

0

0

0

0

0

0

0

0

1

1

1

2

2

3

3

1

1

2

2

3

1

1

4

2 4

2

1

1

1

2

2

3

1

1

4

2 4

2

1

Unit Test

Functional Test

Integration Test


Co n t in u o u s in t e g r a t io n

“…a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily – leading to multiple integrations per day. Each integration is verified by an automated build (incl. test) to detect its errors as quickly as possible…”

Martin Fowler


GEOFF

CI Wo r k f l o w

F2 F1 F4

FCS Sprint 1

Sprint 3

Sprint 2

Sprint n

CI Ser v er

…

FIP

AutoRT preparation: S2 TCs


AutoLT preparation: e2e feature TCs

AutoLT preparation: e2e feature TCs

Dump building

AutoLT execution (appropriate suite)

Dump building


Dump building


Dump building


AutoRT execution S1 TCs

AutoRT execution S1+S2 TCs

AutoRT execution S1+…+S(n-1) TCs

TF


M Ax Week

M Ax Week

M Ax Week

M Ax Week


Te s t TOOLs & En v ir o n m e n t

M-MGW

MSC-S Dual Blade 60

MSC-S Blade Cluster

RNC BSC

MME MME M-MGW

BSC RNC

HLR CSCF


› Unit Test: – Ericsson proprietary tool testing the interfaces

› Function Test: – Manual: using a simulated environment simulating all nodes but with

the possibility to load the real MSC software – Automated: using a proprietary tool with an open source framework

based on TTCN-3

› Regression Test: – Based on the automated test cases from the Function Test – Running on a Continuous Integration Server (Jenkins)

› Feature load (moderate/high/overload): – Proprietary tool based on TTCN-3 generating load on the real

environment

Te s t TOOLs & En v ir o n m e n t


› Coordination › Maintenance of a high reliability › Working high quality Legacy › Documentation › Test tool & environment

CONCLUSION/WRAP-UP

Challenges


› Eliminate waste › Early indication of Software Quality › Early corrective actions possible › Reduce test time enabling earlier release of product › Enabling of Continuous Integration as first step towards a

possible Continuous Delivery › Early customer involvement

Co n c l u s io n

Benefits


› Manifesto for Agile Software Development › We are uncovering better ways of developing

software by doing it and helping others do it. Through this work we have come to value:

– Individuals and interactions over processes and tools – Working software over comprehensive documentation – Customer collaboration over contract negotiation – Responding to change over following a plan

› That is, while there is value in the items on the right, we value the items on the left more.

Co n c l u s io n

test in g - gesellschaft für informatikfg-tav.gi.de/fileadmin/user_upload/tav34/koite_paper.pdf ·...

Documents