Test Managers: How You Can Really Make a Difference
Post on 27-Jun-2015
DESCRIPTIONWhen leading a test team or working in an agile team, becoming a trusted advisor to other stakeholders is paramount. This requires three key skills: earning trust, giving advice, and building relationships. Join Julie Gardiner as she explores each of these skills, describing why and how a trusted advisor develops different mindsets. Julie shares a framework of quick-wins for test managers and team leaders who need to show the value of testing on projects. To help provide timely, relevant information to stakeholders, she shares seven powerful monitoring and predicting techniques. Julie demonstrates three objective measures showing how testing adds value to organizations. To make sure that everyone is on the same page, Julie urges managers to establish a foundation for testing through well-defined policy statements, agreed to and sanctioned by senior management. Receive a set of spreadsheets and utilities to support your activities as a test manager who really makes a difference.
- 1. TP PM Tutorial 10/1/2013 1:00:00 PM"Test Managers: How You Can Really Make a Difference" Presented by: Julie Gardiner The Test PeopleBrought to you by:340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 904-278-0524 firstname.lastname@example.org www.sqe.com
2. Julie Gardiner The Test People In the IT industry for more than twenty years, Julie Gardiner has held positions as an analyst programmer, Oracle DBA, and project manager. She has first-hand experience as a test analyst, test team leader, test consultant, and test manager. Previously with UK-based Grove Consultants for seven years, Julie now is head of agile, talent and transition management with The Test People. A certified ScrumMaster and agile coach, Julie also specializes in risk-based testing, agile testing, test management, and people issues. 3. Passionate about Testing, Excited about Technology Passionate about Testing, Excited by TechnologyTest Managers: How you can really make a difference 13th August 2013 Julie Gardiner Email: email@example.com Twitter: @cheekytesterJulieGardinerPassionate about Testing, Excited about TechnologyContentsWhat makes us valuable on project How to keep testing valuable on projects Showing the value of testing1 4. Passionate about Testing, Excited about TechnologyWhat does a Trusted Advisor look like?Passionate about Testing, Excited about TechnologyWhat senior managers are looking for commitment not just involvement plus a knowledgeable enthusiasm sense of high quality teams humour respect for their decisions J an understanding of office politics an appreciation of other pressures and issues honesty and courage eyes on the deadline but hands firmly on quality a realist with a hint of optimism2 5. Passionate about Testing, Excited about TechnologyWhat if senior managers seem uninterested? lack of understanding solu%on:educa%on what do we do with our time? misconceptions regarding testing/development boredom solu%on: enquire information overload too much detail the wrong information presented (does not understand)The bitterness of poor quality is remembered long after the sweetness of meeting the schedule is forgottenPassionate about Testing, Excited about TechnologyBeing a Trusted Advisor requires three core skillsSource: The Trusted Advisor David Maister, Charles Green Robert Galfordbuilding relationships3 6. Passionate about Testing, Excited about TechnologyTrust must be earned & deserved trust grows over time rather than just appears beware that trust can be lost in an instant trust is personal it involves both rational and emotional factors trust is a two-way relationship trust is different for the test manager & senior manager trust will entail risk if risks are not created, trust will flounderearning trustto be trusted is never a right JulieGardiner it is always a privilege and honourPassionate about Testing, Excited about TechnologyBuilding relationships deal with senior management as human beings formula to building a strong relationship BSR = AQ + LTAbuilding relationships be open and honest not a sycophant when you need help ask this can be a sign of strength rather than weakness show appreciationmake it your goal to be someone managers turn to first when issues arise and advice is required.4 7. Passionate about Testing, Excited about TechnologyGiving advice effectively be prepared look beyond the now & understand the whole situationgiving advice advice is almost never a purely logical process dont exaggerate or lie ever! (this destroys trust) give good news before bad news (dont hide bad news) be fair with information know your audience the advice we give must be helpful different people need different informationPassionate about Testing, Excited about TechnologyDeveloping the right mindsethaving a right attitude and mindset is a massive step closer to becoming a Trusted Advisor to Senior Managementsix key attitudes to consider focus on the other person being sincere having confidence (in oneself) adopting a blameless culture taking responsibility recognise strengths and weaknesses in you and your team5 8. Passionate about Testing, Excited about TechnologyContentsWhat makes us valuable on project How to keep testing valuable on projects Showing the value of testingPassionate about Testing, Excited about TechnologyCompany Objectives? QualityPriority QUALITYTIMECostTimeCOSTJulieGardiner6 9. Passionate about Testing, Excited about TechnologyManaging testing Policy overall approach to testingPlanMeasure StrategyReviewGoalPassionate about Testing, Excited about TechnologyTest policies represents testing philosophy for organisation outlines the corporate goals for testing guides to stated goals and objectives communicates good practice for the organisation approach, culture and standards developed by the IT, R&D or Product Development apply to both new development and maintenance short, static, high-level document some areas covered include test process, levels of testing, success factors, measurements, test process improvement7 10. Passionate about Testing, Excited about TechnologyTest policy examples testing is to be riskdriven reviews will include testers auditable test documentation records will be kept entry and exit criteria will be specified and enforced the value of testing will be monitored and reportedWhat policy statements do you have in your organisation?Passionate about Testing, Excited about TechnologyPolicy, Strategy and PlansCompany Policy for Testing Company/Project Strategy for Testing Project (High Level) Test Plan8 11. Passionate about Testing, Excited about TechnologyMonitoring regular and timely supply of information what is actually happening to compare with the plan right use of words, graphs and tables interpretation & explanation is often required enables effective and timely control used to predict what is likely to happennow to introduce some powerful monitoring techniques to display on your dashboardPassionate about Testing, Excited about TechnologyOne: S-curves HowtouseS-curves S-curvesfor: testsrun/a@empted testspassed/complete bugsfound bugsxed PmePlotthepointsyouknow,andplannedendpoint e.g.no.testsrunsofar&plannednalno.atenddate Plottrendline:3rdorderpolynomial UsefulapproximaPon:straightline Source:MarnieHutcheson9 12. Passionate about Testing, Excited about TechnologyExample using S-CurvesSource: Tim Trew, PhilipsPassionate about Testing, Excited about TechnologyTwo: Defect Measure (DM) RateDM = 10*H+5*M+LDM Rate = DM/(hours of test effort)10 13. Passionate about Testing, Excited about TechnologyThree: System Availability Host System Availability Wed Tue Mon SunDaysSatAvailableFriDown timeThu Wed Tue Mon0246810Hours available in a dayBased on 10 hour day we lose 2 days of down time!Passionate about Testing, Excited about TechnologyFour: Regression Test Coverage TestProcedure SoloPersonnel SoloPayroll PersonnelBreadth PayrollBreadth PersonnelScenario1 PayrollScenario1Key:.0 Release .1 Release1 Release1.0 Release11.1 Release1.2.2notrun defectsfound nodefects11 14. Passionate about Testing, Excited about TechnologyFive: Radar Diagram Rating Code Turmoil(loc) Defects found this week Total Open Defects Test Success Rate Test Completion Rate TOTAL RATING Raw Data Code Turmoil Defects found this week Total Open Defects Test Success Rate Test Completion Rate10/23/2013 1127 23 24 95.50% 44.80%Ranges 1 = 0-50, 2 = 51-100, 3 = 101-150, 4 = 151-200, 5 = 201-250, 6 > 250 1 = 0-2, 2 = 3-4, 3 = 5-6, 4 = 7-8, 5 = 9-10, 6 > 10 1 = 0-10, 2 = 11-20, 3 = 21-30, 4 = 31-40, 5 = 41-50, 6 > 50 1 = 100%-98%, 2 = 97%-95%, 3 = 94%-92%, 4 = 91%-89%, 5 = 88%-86%, 6 < 86% 1 = 100%-95%, 2 = 94%-90%, 3 = 89%-85%, 4 = 84%-80%, 5 = 79%-75%, 6 < 75% 5 to 10 11 to 15 >156 6 2 1 5 20Test Completion Rate17-Feb-13 1409 16 32 98.70% 70.00%24-Feb-13 1020 6 7 99.50% 71.80%Code Turmoil(loc) 6 5 4 3 2 1 0Test Success Rate03-Mar-13 799 13 12 99.80% 78.90%Defects found this weekTotal Open DefectsSource: Mike Ennis, Managing the End Game of a Software Project, STAREastPassionate about Testing, Excited about TechnologySix: Outstanding Risksstart Residual Risks today end date residual risks of releasing TODAYall risks open at the start Progress through the test plan Source: Risk Based E-Business Testing Paul Gerrard & Neil Thompson12 15. Passionate about Testing, Excited about TechnologyObjective Objective Objective Objective Objective Benet Benet Benet Benet Benet Benet Benefit (or objective) based test reportingOpen RisksClosedOpenClosedClosedOpenOpenClosed Source:PaulGerrardBenefits available for releasePassionate about Testing, Excited about TechnologySeven: Reliability/Confidence Curve alternative 1alternative 2Time (minutes)Reliability/Confidence Curve 100 80 60 40 20 0 00.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9Release11.1Time to failureR 1.0R 1.1R 1.2R 1.313 16. Passionate about Testing, Excited about TechnologyContentsWhat makes us valuable on project How to keep testing valuable on projects Showing the value of testingPassionate about Testing, Excited about TechnologyThree key measures to considercost saving per bugDefect Detection Percentage (DDP)Pre-release Defect Fixed Percentage (DFP)14 17. 1000 Passionate about Testing, Excited about Technology10010Testing is expensive1RedDesTestUse compared to what? what is the cost of NOT testing, or of bugs missed that should have been found in test? cost to fix bugs escalates the later the bug is found poor quality software costs more to use o users take more time to understand what to do o users make more mistakes in using it o morale suffers o => lower productivity what does a bug cost in your organisation?Passionate about Testing, Excited about TechnologyHow much does it cost (based on Boehm)How much do defects cost? 18000 16000 14000 12000 10000 8000 6000 4000 2000 0Folksamru nn ing li v esy st em /a cc ep ta nc ein iti a l te stat ion sp ec i ficre qu i re men tsBoehmSource: Kristina Bihlar, Folksam BR15 18. Passionate about Testing, Excited about TechnologyHow much do our defects cost Defect in: requirements specification initial test system/acceptance test live runningManhour to fixCost 1 2.5 4 8 15Cost per hour in SEK:665 1663 2660 5320 9975665What if we found 100 defects during testing: During system/acceptance test what could have been found in initial test what could have been found in analysis But it could have cost: Source: Kristina Bihlar, Folksam BRWe saved: We saved:532,000 266,000 465,500997,500 Passionate about Testing, Excited about TechnologyHow expensive for you? do your own calculation calculate cost to fix bugs missed by testing calculate cost to fix bugs found in testing calculate cost of testing opeoples time, machines, tools estimate if no data available your figures will be the best your company has!16 19. Passionate about Testing, Excited about TechnologyQuestions you may be asked How good is the testing anyway? Can you prove you are doing a good job? Your testing can still be just as good in less time, cant it? (That deadline pressure really didnt matter, did it?) Is the testing any better for this release? (Have we learned anything?) (Have we really improved our testing?) How many bugs have we missed?Passionate about Testing, Excited about TechnologySome questions for you do you keep track of defects? defects found in testing o different test stages (e.g. system test, user acceptance test), o different releases (e.g. testing for an incremental release) defects found in live running o reported by users / customers can you find these numbers from a previous project and your current project? do you have a reasonable number of defects found?if so, you can use DDP to measure your test effectiveness17 20. Passionate about Testing, Excited about TechnologyDefect Detection Percentage (DDP) DefectsfoundbythistesPng TotaldefectsincludingthosefoundaYerwards "this" testing could be a test stage, e.g. component, integration, acceptance, regression, etc. all testing for a function or subsystem all testing for a systemPassionate about Testing, Excited about TechnologyEffectiveness at finding defectsDefects Found50 45 40 35 30 25 20 15 10 5 0 Release100% 90% 80% 70% 60% DDP 50% 40% 30% 20% 10% 0%TimeDefects found Defects found Total defects 10 50 38 37 35 27 24 19 0 88 87 85 77 74 69 62 50 42 after testing: 12 found: in testing:DDP =50 69 62 87 85 77 74 50 8857 59 65 68 72 81 = 100 %18 21. Passionate about Testing, Excited about TechnologyOptions for measuring DDP what defects? simplest: all test defects / all defects so far by severity level how "deep" to go? (how much analysis) deeper analysis gives more detailed information obut is it really worth the extra effort that would be needed? start simple simple information is much better than none learn from what information you havePassionate about Testing, Excited about TechnologyDDP versus DFP DefectsfoundbytesPngDefectDetecPonPercentage=alldefectsfoundTesting Defects found andxedPre-release DefectFixPercentage=Defects found andnotxedDefectsfound aYerreleasedefectsxedbeforerelease alldefectsfoundbeforerelease19 22. Passionate about Testing, Excited about TechnologyTop 10 common responses from the survey whatdoseniormanagersactually lookforinatestmanager?Passionate about Testing, Excited about TechnologyTop 10 surprise responses from the surveywhatdoseniormanagersactually lookforinatestmanager?20 23. Passionate about Testing, Excited about TechnologyWays forward for you set yourself an agenda choose most important improvement areas be practical plan the assessment of the changes plan actions and dates assign responsibility how to support the initiative timetable for improvement milestones the value of today is what you do nextPassionate about Testing, Excited about Technology Passionate about Testing, Excited by TechnologyQuestions? Phone: +44 (0)113 8151779 Email: firstname.lastname@example.org Web:JulieGhttp://www.thetestpeople.com ardiner21