[appclub] shazam mobile apps - data driven project management
DESCRIPTION
Presentation given by me over skype on 30.10.2014 for http://appclub.im/events/details/11030 It is a modified version of talk given in Cracow in Poland for Mobiconf 2014 (http://www.slideshare.net/tomaszkustrzynski/092014-mobiconf-2014-v2-39838125) The first report will be on how the data collected on the basis of the company's managers Shazam to make management decisions, evaluate projects and solve other important issues. Participants should be familiar with basic principles of Kanban methodology to better understand the essence of the report. The presentation will take place via Skype in English.TRANSCRIPT
Shazam mobile apps - Data Driven Project Management
Tomasz Kustrzynski, October 2014
© 2014 Shazam Entertainment
About Shazam
music recognition service and apps
founded in 1999
launched in the UK in 2002
one of the most popular mobile apps
500M+ downloads
© 2014 Shazam Entertainment
Shazam technology - then
(phone you most likely used in 2002)
1. call 25802. play the music3. wait for SMS with results
© 2014 Shazam Entertainment
Shazam technology - now
since 2008 - iPhone and Android appssince 2014 - integrated with iOS8’s Sirimusic streaming, TV ads, live TV,and more
organisation
why and what do we measure
estimation (and causation bias)
handling dependencies
development patterns
Agenda
Organisation
© 2014 Shazam Entertainment
The teams
© 2014 Shazam Entertainment
Technical Project Management
the scientific method
hypothesis experimentanalyse
datadraw
conclusions
improve continuously(Kaizen)
© 2014 Shazam Entertainment
We use Kanban
visualised value stream
enables detailed metrics
JIRA to manage boards
Why and what do we measure?
© 2014 Shazam Entertainment
What do we care about
Users. We now have millions monthly active users.
100+
© 2014 Shazam Entertainment
Data driven decisions
© 2014 Shazam Entertainment
Data gathering
"In God we trust; all others must bring data."Edward Deming
"What's measured improves."Peter F. Drucker
© 2014 Shazam Entertainment
Data gathering
information radiators across the office
© 2014 Shazam Entertainment
Presentation matters
Estimation
© 2014 Shazam Entertainment
Estimates are dangerous
© 2014 Shazam Entertainment
Estimation scale we use
1
TFB - too f* big
NFC - no f* clue
[give me a minute]
cards in the photo by
Lunar Logic (@Lunar_Logic)
FAQ 1
As a Product Manager, when can I expect this Story to be completed?
© 2014 Shazam Entertainment
Disney/Cycle Times
time for a Story to get implemented and tested(or time from entering queue until finishing the ride in Disneyland)
having lots of samples we can use statistics(and talk about probabilities)
© 2014 Shazam Entertainment
Disney/Cycle Times
parallel streams of
work
Cycle Times
t
start of development
start of test
work done
© 2014 Shazam Entertainment
Disney/Cycle Times - statisticshistogram of Disney Times (Cycle Times)
(all items, bugs)density function 50th, 85th percentile
(50/85% items get finished within this time)
© 2014 Shazam Entertainment
When a Story will get done?
1. You can get this story within X days with 85% probability.
2. Probability of you getting it within a week is Y%.
Ok, so when can we get all these 10 stories done?
FAQ 2
© 2014 Shazam Entertainment
it’s easier with Takt TimesTakt Times
Modelling parallel work
parallel streams of
work
parallel work is difficult to model with cycle times
Cycle Times
t
© 2014 Shazam Entertainment
Distribution of average Takt Times
create set of average Takt Time samples based on similar project
resample and ∑many times
(>1000)
average
distribution of average Takt Times
© 2014 Shazam Entertainment
Predicting delivery time for N items
average TT
* Nrepeat many
times (>1000)average TT
average TT
average TT
distribution of expected delivery times
© 2014 Shazam Entertainment
When will all the Stories get done?
1. You can get these Stories within X days with 85% probability.
2. Probability of you getting them within a week is Y%.
Dependencies and blockersAs a Project Manager, what should I focus on next?
© 2014 Shazam Entertainment
© 2014 Shazam Entertainment
Blockers/dependencies
● visualise● prioritise
(what is blocked, for how long)● act● collect statistics and review them
(have any recent changes to the process change amount of blocked items/how long they’re blocked for?)
© 2014 Shazam Entertainment
What should I do first? - answers
Team A: finally doing well
Team B: keeps getting blocked4 blockers including 1 old
Team C: doing well
Team D: 1 new blocker
© 2014 Shazam Entertainment
Blockers statistics
trend of increasing median time to unblock
focus on the bottleneck
Development patterns
FAQs
Why did we end up with large number of stories to test at once?
Do some types of items take longer to develop than others?
Are there times when teams are more productive?
Did we split Stories properly or we missed splitting some big ones?
© 2014 Shazam Entertainment
Patterns and improvement
Being aware of patterns enables us to act on them:
● introduce better policies(e.g. all UI assets for a Story done before dev commences)
● change development cycle(e.g. release more often to prevent batching)
● treat some types of work differently(e.g. allow extra dev time)
© 2014 Shazam Entertainment
Control chart
© 2014 Shazam Entertainment
Control chart
© 2014 Shazam Entertainment
Control chart
© 2014 Shazam Entertainment
Control chartweekends
Thank [email protected]
© 2014 Shazam Entertainment
References
● Real Options: commitment-thebook.com, Olav Maassen, Chris Matts● Real Options: http://www.infoq.com/articles/real-options-enhance-agility● Real Options: http://decision-coach.com/lean-and-real-options/● “High level planning using Monte Carlo simulation”, Dimitar Bakardzhiev● “Thinking, fast and slow”, Daniel Kahneman● http://en.wikipedia.org/wiki/Takt_time