does16 london - amy phillips - is your gatekeeper locking out quality?

Is your Gatekeeper Locking out Quality?

Amy Phillips

Songkick

Songkick - Track your favorite artists and never miss them live

Amy Phillips - Songkick - @itjustbroke


More than 500,000 people across the UK and Europe signed up

235,000 tickets sold

53,000 scalpers identified

A story about moving to frequent releases


The problem with releases

• Pressure to deliver

• Tight timeframes

• Often infrequent


A typical release

Code Release candidate

Bug ReportBug

ReportBug

Report

Developer Build Manager Tester Ops Engineer

Release to Production

Sign off

Business Rep


But is this going to deliver quality?


What is Quality?

Do you consider LinkedIn to be a quality website?


Some comments on LinkedIn’s quality

Hardly any problems with it, no glitches. Nice UX and usability.

Bad UI, too many crashes and bugs. Overloaded with features.

It meets my needs but can be a hassle to use at times.

I have found it very buggy lately


What is Quality?

“Quality is value to some person who matters“

- Jerry Weinberg


What is Testing?

Tested = Checked + Explored

“You’re not done testing until you’ve checked that the software meets expectations and you’ve explored whether there are additional risks.”

- Elisabeth Hendrickson


The Gatekeeper process

Can I commit? Can I deploy? Can I release?

Automated integration test and build

Regression test Monitor

Release Commit

Code


The Gatekeeper process

Can I commit?Can I deploy? Can I release?


Regression test Monitor

Release Commit

Code

Who broke the

build?

When will testing be

finished?

How important is this

bug?

When will this be released?

It doesn’t

work


What are we trying to achieve with releases?

Releases when we want them.

Deliver quality products to users.

Without the unexpected surprises of downtime, bugs, unhappy users.


A common problem

Releases require people from different teams to come together to make them happen.

We release something that doesn’t meet the requirements.

Somebody wants reassurance that the release will be successful.

Technical limitations make releases more difficult.


The Songkick Technology Team

Developers

Testers

Tech Ops

Designers

Product Managers

Cross functional teams, Agile approach

Service Oriented Architecture

Ruby on rails, Go, PHP, JavaScript, Java, C#, Swift


The Songkick Gatekeeper process


Manual testing of feature and release Release Commit Code

CI Server Test Environment

~1 hour ~2 hours


Fix the right problem

Step 1 - Identify the single biggest (perceived) bottleneck. Remove it.

Step 2 - Identify the next biggest (perceived) bottleneck. Remove it.

Repeat until you run out of bottlenecks.


A DevOps Approach


Automated release testing

Release Commit

Code

CI Server Test Environment

~20 minutes

Test Test Monitor

Developer Environment Production


The result: Releases month on month

0.00

28.0056.00

84.00

112.00140.00

Category Title

Mar Jul Nov Mar JulAmy Phillips - Songkick - @itjustbroke

In Summary

Locking down a release process and introducing sign offs doesn’t guarantee quality.

Good quality doesn’t just mean bug-free.

Collaborative nature of DevOps can remove the need for gatekeepers AND improve quality.


Songkick is hiring

songkick.com/jobs


http://songkick.com/jobs

Thanks!
