lean analytics

Download Lean analytics

Post on 13-Jan-2015




1 download

Embed Size (px)


Lean Software Development & Lean Analytics overview


  • 1. LEAN ANALYTICS techtalk @ ferretbased on Lean Analytics by Croll&Yoskovitz and wikipedia information

2. LEAN SOFTWARE DEVELOPMENT The term lean software development originated in a book Lean Software Development: An Agile Toolkit, written by Mary Poppendieck and Tom Poppendieck. 3. LEAN PRINCIPLES 4. Focus on the customer and eliminate waste Everything not adding value to the customer is considered to be waste. If some activity could be bypassed or the result could be achieved without it, it is waste.Amplify learning The best approach for improving a software development environment is to amplify learning. Instead of adding more documentation or detailed planning, different ideas could be tried by writing code and building. A data driven cycle of hypothesis-validation-implementation should be used to drive innovation and continuously improve the end-to-end process. 5. Decide as late as possible Delay decisions as much as possible until they can be made based on facts and not on uncertain assumptions and predictions.Deliver as fast as possible The sooner the end product is delivered without major defects, the sooner feedback can be received, and incorporated into the next iteration. The shorter the iterations, the better the learning and communication within the team. 6. Empower the team Find good people and let them do their own job. People do need something more than just the list of tasks and the assurance that they will not be disturbed during the completion of the tasks. People need motivation. The developers should be given access to the customer; the team leader should provide support and help in difcult situations. 7. Build integrity in Understanding the problem domain and solving it at the same time, not sequentially. The information ow should be constant in both directions from customer to developers and back.See the whole Think big, act small, fail fast; learn rapidly. Larger software = more part developed by different teams, but lean thinking has to be understood well by all members of a project, before implementing in a concrete, real-life situation. 8. If you cant measure something, you cant manage it. Peter Drucker, management consultant 9. LINE IN THE SANDMeasurable target that everyone (incl. executives) agrees to. 10. GOOD METRICS Qualitativevs. quantitativequantitative data answers what and how much, qualitative data answers why. Quantitative data has no emotions. Vanityvs. actionableUse metrics you can act on. Vanity metrics might make you feel good, but they dont let you act. For instance, total signup vs. percent of active users. 11. Exploratoryvs. reportableExploratory metrics are speculative, while reporting metrics keep you nearby of normal, day-to-day operations. Leadingvs. laggingLeading metrics give you predictive understanding of the future, while lagging metrics explain the past. Leading metrics are better because you have time to act on them. Correlatedvs. causalIf 2 metrics change together, theyre correlated, but if one causes another to change, theyre causal. Try to nd a causal relationship between smth. you want and smth. you can control. 12. Acquisition, activation, retention, revenue, and referral - AARRR. Pirate Metrics by Dave McClure that every startup needs to watch 13. RIGHT METRIC FOR RIGHT NOW Acquisition How do users aware of you? Metrics: trafc, mentions, cost per click, search results, cost of acquisition, open rate.Activation Do drive-by visitors subscribe, use etc.? Metrics: signups, completed on boarding process, used service at least once, subscriptions 14. Retention Does a one-time user become engaged? Metrics: engagement, time since last visit, daily/monthly active use, churnsRevenue Do you make money from user activity? Metrics: customer lifetime value, conversion rate, shopping cart size, clickthrough revenueReferral Do users promote you product? Metrics: invites sent, viral coefcient, viral cycle time 15. ONE METRIC THAT MATTERS The OMTM is the one number youre completely focused on above everything else for your current stage. 16. It answers the most important question you have you need to identify the riskiest areas of your business, and thats where the most important question is.It forces you to draw a line in the sand and have clear goals after youve identied the key problem, you need to set goals.It focuses the entire company Use OMTM as a way of focusing you entire company. Display it throughout web dashboards, on TV screens, or regular emails.It inspires a culture of experimentation Its critical to move through the build-measure-learn cycle as quickly and as frequently as possible. To succeed on that, you need to actively encourage experimentation. 17. the One Metric That Matters changes over time When you are focused on retention, you may be looking on churn, and experimenting with pricing, features, improving customer support etc. 18. MAIN STAGESYou cant just start measuring at once. You have to measure your assumptions in the right order. 19. Empathy Go inside target market and sure youre solving a problem people care about in a way someone will pay for.Stickiness It comes from a good product. You need to nd out if you can build an acceptable solution to the problem youve discovered. 20. Virality Once youve got a product thats sticky, you need care about acquisition.Revenue Youre giving away free trials, free drinks, or free copies. Now youre focused on maximising and optimising revenue.Scale With revenues coming in, its time to move from growing your business to growing you market. 21. EMPATHY 22. Find a problem to x: - The problem painful enough - Enough people care - They are already trying to solve itTalk with people and rate interviewsMostly qualitative metrics here. Be honest with yourself. 23. STICKINESS 24. Daily, weekly, monthly active usersTime to become inactiveNumber of reactivated inactive after emailTime to spend with feature 25. 7 QUESTIONS TO ASK BEFORE BUILDING A FEATURE Why will it make things better? You cant build a feature without a reason. Ask yourself Why will it make it better? and write out a hypothesis. 26. Can you measure the effect of the feature? Feature experiments require that you measure the impact of the feature. That impact has to be quantiable.How long will the feature take to build? Time is resource you never get back. If something is going to take too much to build, break it into small parts or test the risk with the prototype rst. 27. Will the future over complicate things? Complexity kills products. When discussing a feature with your team, pay attention to how its being described. And is enemy of success.How much risk is there in this feature? Building a new feature always comes with some amount of risks - technical risk, user risk, risk of inuence to further development etc. 28. How innovative is the new feature? Not everything is innovative, but consider innovation when prioritising feature development. Generally, the easiest thing to do rarely have a big impact.What do users say they want? Users are important as well as their feedback. But relying on what they say is risky. Be careful about over prioritising based on user input alone. 29. VIRALITY 30. Invitation rate - the number of invites sent divided by the number of users you haveAcceptance rate - the number of signups or enrolments divided by the number of invitesViral coefcient (OMTM) - the number of new customers that each existing customer is able to successfully convert Viral = invitation rate x acceptance rate 31. REVENUE 32. QRR(x) - the quarterly recurring revenue for quarter xQExpSM(x) - sales and marking expense for the quarter xRatio of inputs to outputs (OMTM) q = [QRR(x)-QRR(x-1)] / QExpSM(x-1) You have problems if q < 0.75 33. SCALE 34. On this stage you already know your product and market. Your metrics now should be focused on the health of your ecosystem and your ability to enter new markets.Customer acquisition payback (OMTM) - the customer acquisition cost divided by the customer lifetime valueCheck metrics across channels, regions, and marketing campaignsTry to understand if youre focused on efciency(try to reduce cost) or differentiation (try to increase margins). 35. CONCLUSION 36. Make Sure Goals are Clearly Understood To prove the value of analytic-focused company, any project needs to have clear goals. Everyone involved in the project needs to be aligned around the goals. Make Things Simple to Digest A good metric is the one thats easy to understand at glance. Metric can be extremely valuable, but used incorrectly they will lead down the wrong path.Ensure Transparency If you are going to use data to make decisions, its important that you share the data and methodologies. 37. Dont Eliminate your Gut Lean Analytics isnt about eliminating your gut, its about proving your gut right or wrong.Ask Good Questions You dont need to guess, you need to know where to focus. You dont know all answers, but you should know the right questions to ask. 38. FUTHER READINGS Lean Software Development: An Agile Toolkit by Mary & Tom PoppendieckLean Analytics by A. Croll, B. YoskovitzThe Lean Startup by Eric RiesRunning Lean by Ash Maurya