THE EVOLUTION OF QA AT JUST EATJIM BEATTIE – HEAD OF I18N, JUST EAT

2

INTRO

About Me +About JUST EAT

Automation…

4 QA improvementsused by JUST EAT

1 brand new thing

Bleeding Edge!

Why things have changedFor QAs

JIM - BACKGROUND

JUST EAT: LOCAL TAKEAWAY ONLINE

The JE network

• 36,000 restaurants

• 13 countries

• 7M registered members

• $1BN per year generated for

restaurants!

• 1000 employees

QA• 6 QAs in UK (down from 10) • 5 in Kyiv• Everyone is responsible for

Quality

5

HOW THINGS HAVE CHANGED…

6

OLD SCHOOL ENGINEERING PROCESS

7

ENGINEERING IS DIFFERENT NOW

Agile!!!• 1 – 3 week sprints

• Lower risk of failure

• Reduced impact of failure

• Releases are different

• Different philosophy

8

WHAT DOES THIS MEAN FOR QA?

Trust more

Resist change

MISSION

ATTITUDE

Reduce

riskNot about

perfection

9

CONCLUSION – ENGINEERING HAS CHANGED

JUST EAT QA NEEDED TO CHANGE AS WELL

10

SO… HOW CAN WE CHANGE?

11

AUTOMATION: BIGGEST RECENT CHANGE IN QA

Is automation the silver bullet?

Great, there’s no

manual testing to

do!

12

CONCLUSION - AUTOMATION

START WITH AUTOMATION

TEST THE GAPS

13

OK… APART FROM AUTOMATION,HOW DID WE CHANGE?

14

WHAT ABOUT EVERYTHING ELSE

Efficient process

Using users

Exploratory testing(test the gaps)

Using crowds

15

BE EFFICIENT!

16

BE EFFICIENT!

Documentation:• Test Strategy• Test Approach• Test Plan• Entry / Exit Criteria

• Test Cases• Test Execution Matrix

• Test Evidence

What’s the value?

Any drawbacks?

17

LOOK FOR THE VALUE

The value is in the contentBetter content comes from collaboration

Everyone’s got

an iPhone

(or Android!)

18

CONCLUSION – BE EFFICIENT

CHUCK NORRIS DOESN’T WRITE TEST PLANS.HIS TESTS PLAN THEMSELVES.

19

EXPLORE!

20

EXPLORATORY TESTING

The value is in human intelligence

Where’s the gap???

Quality not docs

Break my stuff! Win a prize!Have some fuuuunCelebrate

at show and tells

21

CONCLUSION – EXPLORATORY TESTING

YOU ARE MORE CREATIVE

THAN A ROBOT

PROVE VALUE AND EARN TRUST

22

TAKE EXPLORATION TO ALPHA

23

THE JUST EAT BUG BASH

LunchtimeNo docsEveryone WelcomeGhetto Style!

Great results every time!

24

CONCLUSION – EXPLORATORY TESTING

MORE PEOPLE FIND MORE

BUGS

AND YOU GET FREE PIZZA

25

CROWDSOURCING

1) URL (or APP)

2) Optional Spec

3) GO!

10,000 QAEs

Pay Per Bug

Great value

Great coverage

Drawbacks?BackendFalse positivesLow value bugsGreat coverage?

26

CONCLUSION – CROWDSOURCING

THE CROWD ARE YOUR

FRIENDS

HELP THEM TO IMPOVE

PRODUCT QUALITY

27

AFTER ALPHA COMES…

28

BETAS

Measure KPIsMeasure server statsCollect feedback

Drawbacks?Feedback?What if…

Release to 2 servers

29

BETAS – FEEDBACK

It’s good to get

feedback:-)

30

CONCLUSION – BETA

WE DID IT AND

NOTHING BAD

HAPPENEDWE DON’T EVEN LABEL IT BETA

31

CONCLUSIONS FROM THE HEART

32

WHAT’S NEXT FOR QA?1 BRAND NEW THING ;-)

33

BENEFITS WITH AUTOMATION AND MANUAL

Automation:• Repeatable• Quick• Programmable

• Triggered via CI

Manual / crowds:• Intelligent• Not so brittle

Let’s get married xxx

34

RAINFOREST APP

35

CONCLUSION – QAAAS

AUTOMATION MEETS

OUTSOURCING!

IS THIS THE FUTURE?

36

RECAP

Automation…

3 QA advances used by JUST EAT• CUT WASTE• EXPLORE• USE GROUPS

QAAAS

Why things have changedFor QAs

37