AT4 Agile Development Concurrent Session 11/13/2014 10:00 AM

"Establishing an Agile Testing Culture"

Presented by:

Leigh Ishikawa TripAdvisor

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

With a career spanning more than two decades in various roles, Leigh Ishikawa facilitates change in how organizations view testing by establishing test frameworks and processes, and educating and building teams across multiple continents. In the past decade, Leigh has focused on building test cultures inside high velocity engineering teams. After spending a year and half creating test frameworks and coaching engineers on testing, Leigh is currently a developer on TripAdvisor’s mobile team.

Establishing an testing culture

Leigh Ishikawa

Agile Culture

• It’s not something you mandate for small segment of your population

• It’s something that every individual and process has to respect and uphold

• Must support organization’s vision – Some organizations are not meant for agile

Where do we begin?

Failures Observed

• Promises – Improve quality and customer satisfaction while

reducing development time

• (Trying to) replicate someone else’s culture – Bring a person from another company

• Using Agile as means to solely drive engineering productivity – Business continues it’s merry way

Successes observed

• Agile(like) organizations have accepted the agile culture as a whole

• Agile(like) organizations apply agile to iteratively improve their overall organization as well as their engineering

• Agile(like) organizations value people and fundamentally practice the manifesto

Agile Manifesto

• Individuals and interactions over processes and tools

• Working software over comprehensive documentation

• Customer collaboration over contract negotiation

• Responding to change over following a plan

• That is, while there is value in the items on the right, we value the items on the left more.

Good reasons to adopt Agile

• Desire to make a better product – Better product != $$$

• Desire to have a better work environment – Attract and retain talent

Bad reasons for using Agile

• Improve quality with little/no effort • Improve productivity with little/no effort • Put better process in place

– More monitoring – More metrics – Reduce cost - Move resources overseas

• Agile is just for engineers

Testing

Are we there yet?

Testing is hard

• Most companies struggle • Most never get to where they are really happy

with their testing • Most companies put tools in place that is

ignored and neglected • Many people think there’s a ‘secret recipe’

Poor quality speaks of many things

• Indicator of bigger organizational problems – Either disconnect between staff and management

or staff who are disengaged

• If careless, could be a downward spiral – The more you focus on the wrong things, the

worse it gets

• Qualitative goals are hard to measure – And difficult to enforce correctly

Automation

• Fails most of the time because people underestimate the difficulty – Hire/promote wrong (or bad) talent

• Treat testing as a second class citizen

– Desperately try to use existing resource • Most QA people can’t code and they shouldn’t either

– Buy expensive software/contracts who promises just that – Have a non SE design a test framework

• Your first/key hire often determines the fate

Developers should test

• Good or bad intentions? • Good organizations are willing to attract and

allocate developers for testing projects • Bad organizations will mandate that testing

should happen (more or less) on developer’s personal time and shame them

• Bad organizations think testing just happens • Testing is an art

Test Metrics

• Drive your teams with mandated metrics will lead you to failure – “The Logic of Failure”

• While trying to achieve immediate goals, they destroyed something crucial

• Metrics should be used as a means to observe and understand the state

Metrics – Code Coverage

• Code coverage goals are, generally, a bad idea • That said, In Java (at the package level)

– 40% = Team is beginning to get grasp of testing – 60% = a sweet spot of where your test is getting a

good coverage • Percentage is meaningless without

understanding what is tested. Above % is based on team prioritizing what they are testing. (Ignore setters/getters, etc)

You’ve heard this before

1. Hire the right people 2. Provide infrastructure

– CI, dashboards, etc. 3. Provide goals and visual aids

– Dashboards – Signs - No build break for ### days

4. Re-enforce the good behavior – Build breaks stops everything – Test failures = build breaks

Reality

• You will hire the wrong person/people/consulting firm

• You will buy/build wrong infrastructure • You won’t know what metrics to measure (or

avoid) • ALL THE ABOVE DOESN’T MATTER

Great testing culture is not

• Bugs • Metrics (Test coverage, customer surveys, etc) • Dashboards • Speed of your development

Great Testing culture is

• Desire of individual and organization to build better product

• Open to new ideas • Willing to invest and try different ways to

make your software better through iterative approach

• Transparency within organization and your customers

Your company is unique

Your company is a living entity filled with unique individuals. No two companies are alike and should try to be Each company has their strengths and weakness Understand what makes your company special

Before you attempt to change

• Find your company’s strengths • Understand why your company works the way

they do • Figure out where you company wants to get to

and where it needs to • Iterate over various tools and process to make

improvements over long period of time

Conclusion?

To Establish a testing culture, you must grok organization’s culture and Agile Manifesto. You must practice agile manifesto at the

cultural level while preserving the core values of the organization