Continuous Integration: Improving Software Quality and Reducing Risk

Preetam PalweAftek Limited

One more title

Do you love bugs ?Or

Are you in love with QC members?

… [Courtesy: Smita N]

Agenda

Motivation Practices Advantages The road ahead Q&A

Motivation “Enter laptops, enter boards and put together with

code, all teams have come together in one abode. Integration in progress.” … taken from an article published in AForce March 2008

Admin Utility User now can be associated with the profile and action like

send sms, call to a specific person or play welcome message on entering in the home.

The UI part for above feature is implemented by XYZ. And backend part is implemented by PQR. Integration of the same is pending and will integrate on next week.

… taken from an weekly status report sent in 1st week of March 2008

So ???

Motivation Integration is an EVENT !!!

Can we have something like following?

Image Courtesy: Book on CI by PMD

What is CI? Continuous Integration is a software development

practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly.

… taken from Martin Fowlers article on CI

Agile manifesto Individuals and interactions <OVER>

processes and tools Working software <OVER> comprehensive

documentation Customer collaboration <OVER> contract

negotiation Responding to change <OVER> following a

plan

That is, while there is value in the items on the right, we value the items on the left more.

Agile principles1. Our highest priority is to satisfy the customer through

early and continuous delivery of valuable software.2. Welcome changing requirements, even late in

development. Agile processes harness change for the customer's competitive advantage.

3. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter time scale.

4. Business people and developers must work together daily throughout the project.

5. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

Agile principles (contd)6. The most efficient and effective method of conveying

information to and within a development team is face-to-face conversation.

7. Working software is the primary measure of progress.8. Agile processes promote sustainable development.9. The sponsors, developers, and users should be able to

maintain a constant pace indefinitely.10. Continuous attention to technical excellence and good

design enhances agility.11. Simplicity—the art of maximizing the amount of work

not done—is essential.12. The best architectures, requirements, and designs

emerge from self-organizing teams.13. At regular intervals, the team reflects on how to

become more effective, then tunes and adjusts its behavior accordingly.

Adaptive (agile) vs Predictive (plan-driven) Agile home ground

Low criticality Senior developers Requirements change very often Small number of developers Culture that thrives on chaos

Plan-driven home ground High criticality Junior developers Requirements don't change too often Large number of developers Culture that demands order

Extreme programming Form of agile software development

Traditional software engineering practices taken to so-called "extreme" levels—leads to a development process that is more responsive to customer needs ("agile") than traditional methods, while creating software of better quality.

Values and/or principles Communication Simplicity Feedback Courage Respect

Activities Coding Testing Listening Designing

XP practices Fine scale feedback

Pair programming Planning Game Test driven development Whole team

Continuous process Continuous Integration Refactoring or Design Improvement Small Releases

Shared understanding Coding Standards Collective Code Ownership Simple Design System Metaphor

Programmer welfare Sustainable pace

Coming back to CI Continuous Integration is a software development

practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly.

… taken from Martin Fowlers article on CI

Practices

Maintain a single source repository Keep everything needed for build in

CVS E.g. code, database schema, IDE

configuration, test scripts etc but not application server and jdk. Even no build artifacts!

Keep branches to minimum (in size and in lifetime)

Practices

Automate the build From compilation to deployment Configure the build:

Master build on server Developers build from IDE Build with test, without test or different

set of tests!

Processes Make your build self-testing

High degrees of automated tests suits to check larger code base

Though TDD not mandatory but its helpful Test case failed: build failed! XUnit tools like JUnit and/or end-to-end

testing tools like selenium Imperfect tests, run frequently, are much

better than perfect tests that are never written at all.

Processes

Everyone commits every day Every commit shall succeed the build Frequent commits: small work

packets for developer Mentoring and practice is the key

Processes Every commit should build the mainline

on an integration machine You shouldn’t go home until the mainline

build has passed with any commits you’ve added late in the day!

Don’t check in untested / broken code Don’t check in when build is broken The whole point of CI is to detect integration

errors as soon as possible Peers pressure within team ensures policies

are followed

Processes Keep the build fast

XP guideline: 10 mins build! Staged build (build pipeline)

Commit build Secondary build Performance build

If secondary build fails move the test case to commit build

Parallel builds

Processes

Test in a clone of production environment Every environmental difference

results in a risk: what happens under test wont happen in production

E.g. Same IP, port, machine, hardware, os etc

Virtualization can help

Processes

Make it easy for anyone to get the latest executable Its very hard to specify what you want

in advance and be correct; people find it much easier to see something that’s not quite right and say how it needs to be changed

Many executables in a day so not in CVS

Processes

Everyone can see what's happening CI is all about communication Build status indicators: lava lamps,

colored labels, hooters, mails Build history and build status reports

indicates the project “health” Matrices are helpful to guide project

management

Processes

Automate deployment Use deployment pipeline Automated rollback

Advantages Reduces risk

Integration is predictable now We know where we are, what works, what doesn't work,

how many outstanding bugs are there Makes dramatically easier to find and remove

bugs Bugs - these are the nasty things that destroy confidence

and mess up schedules and reputations. Bugs in deployed software make users angry with you. Bugs in work in progress get in your way, making it harder to get the rest of the software working correctly.

Broken windows syndrome Break Barriers between customer and

development Add more features rapidly, give rapid feedback

The road ahead Just give it a try Start with commit build Add some automated test Use deployment pipeline Speed up the build Integrate with anything and everything

available like project management tools, bug trackers, code analysis tools etc

Deployment pipeline

Image Courtesy: Paper on Deployment Pipeline by Dave Farley

Cruise control

Open source CI toolkit from ThoughtWorks Support for different plugins like

email, source control, builder, ant etc Support of web ui for build status Integrate with many code analysis

tools, scm tools, build tools, reporting tools etc

Cruise control

Image Courtesy: CC documentation on cc website

References http://martinfowler.com/articles/continuousIntegration.html http://en.wikipedia.org/wiki/Agile_software_development http://en.wikipedia.org/wiki/Extreme_Programming http://cruisecontrol.sourceforge.net/

Q&A

Thanks!