Review:Agile Software Testing in Large-Scale ProjectTalha MajeedCOMP 587Spring 2011

2

Agenda Definitions Introduction Adopting XP and diverging from XP Guidelines in four Development areas

Test design and activity execution Working with professional testers Planning Defect management

Summery Conclusion

3

Definitions Agile testing is a software testing practice that

follows the principles of the overall agile manifesto

In agile testing, testing is not a phase. It is integrated into Development & Testing

QA is not the last line of defense but involves at the beginning of the project.

Agile testing focuses on testing iteratively, as often as stable code base is available, until quality is achieved from end customer’s perspective.

4

Definitions Extreme Programming (XP) is a software

development methodology which is intended to improve software quality and responsiveness to changing customer requirements. Frequent "releases" in short development cycles which is

intended to improve productivity . XP Approach is that if a little testing can eliminate a few

flaws, a lot of testing can eliminate many more flaws. Unit tests determine whether a given feature works as

intended. A programmer writes as many automated tests as they can think of that might "break" the code; if all tests run successfully, then the coding is complete.

Acceptance tests verify that the requirements as understood by the programmers satisfy the customer's actual requirements.

5

Introduction Israel Air Force (IAF) Enterprise information

system Critical to daily operations & information security Highly complex and extreme quality

Project was risky size Scope (XP-based method)

Risk management Metrics were developed Collected & analyzed data

6

Project overview Data collection

Four releases (eight months) Quantitative data

Project team used for project tracking & decision making

Qualitative data Recording of formal reflection meeting &

debriefings

7

Confirming to XP Team implemented standard XP

practices Short release

Working software two week iteration Customer feedback Goal to achieve measureable testing

benefits Short average time for defect fixes Short average defect longevity

8

Confirming to XP Planning game practice

To plan project iterations and releases Entire teem met and customer described and

prioritized stories Goal was to achieve simple design by

refactoring activities XP’s whole-team and site-together practices

Two measureable results Defect management overhead was lower Fewer false defects

9

Diverging from XP The project differed from classic XP in three

aspects First aspect

Team was required to produce detailed specification in addition to code and test due to organizational policy.

Second aspect Project’s accepting testing( user testing) was far too

complex for customer to fully specify Third aspect

Modified test-driven development to focus on the continuous creation of automated acceptance tests rather than unit tests. Never consider a feature “complete before all its tests

were written and passed.

10

Test design and execution Each team’s member was responsible for

testing In a traditional project, "everyone is responsible

for quality,” but in agile project, everyone actually writes testes.

Advantages First, It eliminated the team’s reliance's on single

tester Second, Developers test-awareness increased Finally, team members wrote specification and

code to write test cases for their work.

11

Test design and execution Product size = test size Progress was measured by defining the

project’s product size metrics as the number of regression test steps ran at each iteration

The above approach was chosen for two reasons

First, test size is more correlated with complexity than are lines of code or specification

12

Test design and execution Second, only features with full

regression testing at each iteration are counted as delivered product size.

Figure 1 plots the product size metric period of time

13

Test design and execution Untested work = no work (major shifting to agile

development) Untested work equals to no work in all channels

and all under conditions. Written Questionnaire response after two releases 90% team members considered acceptance and

system tests “highly important” 10% ranked as “important” 33% were interested in taking a leading role in

acceptance testing 60% were “interested” in leading system testing.

14

Working with professional testers Why professional testers?

Add value through more testing Developers code more features

Professional tester two key challenges for organization Testing bottlenecks Coordinating the testing work among

testers and programmers

15

Working with professional testers

Easing bottlenecks Developer s simply code less, test more

Figure 2. illustrates by comparing the product size metric points attained by tests that the entire team

ran to the team’s tester alone

16

Working with professional testers Encourage interaction over isolation Initially testers worked alone due to two

reasons Testers must be independent of

programmers to prevent implementation details from affecting their test designs.

Specification were up-to-date so testers write test according to specification

Testers were integrated to project team

17

Activity Planning Planning for quality activities involve

Time allocation Challenges in feature testing Regression testing Repair defects

18

Activity PlanningFigure 3. Testing and defect repair

(a) Net hours devoted to running (b) Net hours devoted to fixing defects and maintaining regression tests.

19

Activity planning Integrate feature testing and coding

Testing and coding time are usually equal Regression testing: divide and conquer Alocate bug-fix time globally

To estimate the time to fix bug is difficult Reasonably predict the time required to

fixing defects in forthcoming iterations

20

Defect management Defect management involves two major

challenges Managing workflow Selecting and scheduling the defects to fix

Use a team-centered defect-management approach Anyone can open defect Anyone can close a defect, after fixing it Anyone who finds a defect also assigns it to

someone to be fixed.

21

Defect management Fix defects as soon as possible

Bug fix time was slightly over an hour Bug fix requires more time ( a day or more) should be

suspended to prevent bottleneck and to ensure deadlines are met.

Advantages Defects require far less time to fix Working on clean highly stable code base makes new

development faster Avoids unpleasant customer negotiations over which

defects to solve.

22

Defect management

Figure 3. Average net time to fix a defect per iteration

23

Summery Presented definitions of Agile testing, XP

, Unit test, Acceptance tests etc. How Project adopts/ diverges from XP Identified How agile software testing

works on large-scale project Showed the graphs of real data from

Israel Air Force enterprise information system.

Described the Guidelines in four Development areas

24

Conclusion Project's data and analysis are useful both

to practitioners wishing to establish new agile teams and to research work

Even on large-scale project, the team achieved full regression testing at each iteration and developer testing

There is a need of quality data on large-scale project so, it is highly encouraged for further investigation on long-term agile projects.