what does it mean to be a test engineer?
TRANSCRIPT
MY BACKGROUND
• 8 years in software engineering
• started as a test engineer in a team of 20 testers
• currently working on improving product quality and development productivity at Spotify
https://www.linkedin.com/in/andriidzynia@adzynia
SATURDAY EVENING
https://twitter.com/mfloryan/status/655433887015948288
LET ME SHOW YOU THE PROCESS
video: https://www.dropbox.com/s/un7m49kdgeuzsbz/test-session-andrii.mov?dl=0
FOUND BUGS
MY COLLEAGUE, FRONT-END DEVELOPER.
WHAT DID I LEARN?
• Software always has bugs. If you have not found any, does not disprove their existence
• Developers do not spend much time looking for edge cases, testers do
WHAT I WANTED YOU TO LEARN?
Testing is not a set of artefacts.
TESTING IS A PROCESS.
@adzynia
TESTING PROCESS
GUESS WHAT BUG MARCIN WAS REFERRING TO?
MORE LESSONS LEARNED
• People are different. It is hard to predict every single combination when user can get stuck
• Think slower, sometimes
WHAT MAKES YOU A TEST ENGINEER?
Critical thinking
Fault models
Patterns recognition
@adzynia
CRITICAL THINKING
https://www.criticalthinking.org/ctmodel/logic-model1.htm
FAULT MODELS“From the model, the designer or user can predict the consequences of a particular fault.” - Wikipedia
video: https://www.youtube.com/watch?v=rEumMkAhwzA
BASED ON EXPERIENCE BRAIN RECOGNISES SEQUENCES WHICH LEADS TO UNDESIRED
OUTCOME.
NOTE: BE CAREFUL WITH FAULTY ASSUMPTIONS. FACE ON THE MOON SURFACE IS PAREIDOLIA.
“Pareidolia is a psychological phenomenon when the mind perceives a familiar pattern of something where none actually exists.” - Wikipedia
LOOK FOR PATTERNS WITH HIGH FAILURE RISK.
video: https://www.youtube.com/watch?v=qoQssWPRNR0
BRINGING VIDEO TO SPOTIFY USERS
http://techcrunch.com/2015/05/20/spotify-introduces-video-clips-podcasts-and-activity-based-playlists/
TEAM SETUP
COMMUNICATION CAN BE A PROBLEM
http://dilbert.com/strip/1989-05-31
SOLUTION WAS
Create a #slack channel
Start asking video integration questions
Rise bugs early and facilitate conversations
DID WE PREDICTED ALL DEFECTS BEFORE TESTING?SURELY, NOT.
Data collected by Stefan Rungardt
BUT HOW DID THAT HELP?
Integration ended up to be much easier
Testing started earlier
# of production issues was low
WAS THAT EASY?
Not really. You have to ask questions
people do not want to hear
http://media2.giphy.com/media/wrTHBbla7suPu/giphy.gif
Tester: Shall video be playing when I lock a screen?
Programmer: I do not know.
Tester: who can know?
Programmer: Product manager?
Product manager: I do not know.
Tester: who can know?
Product manager: let’s take a look how Youtube does it.
After a while we figured why video has to be paused when you lock a screen. Licensing constraints.
DEFECT PREVENTION VS
DEFECT DETECTION
“Bugs are so 90s”
LEAN MANUFACTURING
https://en.wikipedia.org/wiki/Lean_manufacturing
KEEP YOU WORK ENVIRONMENT IN A GOOD SHAPE
• engineering quality
• product quality
• strive for improvements
ENGINEERING QUALITY
• Start testing as early as possible
• Build safety nets
• Fix bugs as fast as they occur
• Conduct root-cause analysis and take actions
http://www.slideshare.net/AndrewDzynia/quality-built-in
PRODUCT QUALITY• Product success metrics
Time to first video frame
Number of users streaming video content
Average viewing time, etc.
• Number of production defects
SPOTIFY TEST CERTIFIED MODEL
Inspired by Googlehttps://mike-bland.com/2011/10/18/test-certified.html
TAKE AWAYS
• Testing is a process, not a set of artefacts
• Cost of defect can be lowered. Think about that
• Find ways to influence quality on organization level. It is hard, but possible, even for a test engineer
