no estimates - 10 new principles for testing
TRANSCRIPT
![Page 1: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/1.jpg)
#NoEstimates
10 New Principles for
Software Development
(and testing!)
Vasco Duarte@duarte_vasco
Tweet at:#TAHelsinki
![Page 2: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/2.jpg)
Vasco Duarte
@duarte_vasco
http://SoftwareDevelopmentToday.com
http://bit.ly/vasco_slideshare
http://NoEstimatesBook.com
![Page 3: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/3.jpg)
#NoEstimates is for you if…
![Page 4: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/4.jpg)
![Page 5: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/5.jpg)
![Page 6: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/6.jpg)
![Page 7: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/7.jpg)
![Page 8: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/8.jpg)
Principle #1Trust your process, or change your
process
![Page 9: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/9.jpg)
Nu
mb
er
of
Bu
gs
Timeline
Bug evolution in a non-agile project
Open
Closed
Submit
Development phase Desperately testing and fixing phase
Waterfall
![Page 10: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/10.jpg)
Principle #2Shorten the feedback cycle
![Page 11: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/11.jpg)
Can estimates be accurate?
![Page 12: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/12.jpg)
Accidental complication
![Page 13: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/13.jpg)
Cost/Duration =
Essential Complication
x
Accidental Complication
![Page 14: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/14.jpg)
![Page 15: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/15.jpg)
Yet, some people still argue we can be good at estimating… Let’s
look at the evidence
![Page 16: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/16.jpg)
80% Late or Failed
Source: Software Estimation by Steve McConnell
Reality: 80% is late or failed
![Page 17: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/17.jpg)
Lets break that down
Chaos Report 1995For the same combined challenged and impaired projects, over one-third also experienced time overruns of 200 to 300%. The average overrun is 222% of the original time estimate. For large companies, the average is 230%; for medium companies, the average is 202%; and for small companies, the average is 239%.
Time Overruns % of responses
Under 20% 13.9%
21 – 50% 18.3%
51 – 100% 20.0%
101 – 200% 35.5%
201 – 400% 11.2%
Over 400% 1.1%
~68% of projects 51% or more late!
![Page 18: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/18.jpg)
Let’s continue to break that down
Chaos Report 2009
Average cost overrun: 45%
Average time overrun: 63%
Chaos Report 2011
Average time overrun: 63%
![Page 19: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/19.jpg)
Just in case you don’t like the CHAOS report
![Page 20: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/20.jpg)
Source Gartner survey of project failure, 2012
Failure, means total failure, not just late
![Page 21: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/21.jpg)
Of the large systems that are completed, about 66% experience schedule
delays & cost overrun
Source: “Project Management Tools and Software Failures and Successes” by Capers JonesCrosstalk, the Journal of Defense Software Engineering
![Page 22: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/22.jpg)
Traditional projects: 53% failed or challenged
Source: Project success survey by Scott Ambler
![Page 23: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/23.jpg)
Agile project: 40% failed of challenged
Source: Project success survey by Scott Ambler
![Page 24: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/24.jpg)
17 percent of large IT projects go so badly that they can threaten the very existence of the company
Source : McKinsey & Company in conjunction with the University of OxfordType of survey : Study on large scale IT Projects
Date : 2012
![Page 25: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/25.jpg)
More data coming, sign-up at NoEstimatesBook.com to receive
this report when available
![Page 26: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/26.jpg)
Principle #3Believe the data, not the estimates
![Page 27: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/27.jpg)
In 1986, Profs. S.D. Conte, H.E. Dunsmoir, andV.Y. Shen proposed that a good estimationapproach should provide estimates that arewithin 25% of the actual results 75% of the time
--Steve McConnel, Software Estimation: Demystifying the Black Art
![Page 28: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/28.jpg)
Principle #4Use alternatives to Estimate-driven
decision making
![Page 29: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/29.jpg)
Comparison of 17 projects ending between 2001 and 2003. (Average: 62%)
![Page 30: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/30.jpg)
Testing for value, a story…
![Page 31: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/31.jpg)
Principle #5Test for value first, then test for
functionality
![Page 32: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/32.jpg)
Can #NoEstimates work in RealCompanies?
![Page 33: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/33.jpg)
http://j.mp/NoEstimates-Book
![Page 34: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/34.jpg)
Principle #6Estimation is waste, reduce it’s
impact on your business
![Page 35: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/35.jpg)
Principle #7Measure progress only with validated, running software
![Page 36: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/36.jpg)
In 1986, Profs. S.D. Conte, H.E. Dunsmoir, andV.Y. Shen proposed that a good estimationapproach should provide estimates that arewithin 25% of the actual results 75% of the time
--Steve McConnel, Software Estimation: Demystifying the Black Art
![Page 37: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/37.jpg)
Source: Software Estimation by Steve McConnell
“Good” estimates: 25% of estimated
![Page 38: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/38.jpg)
I AM GOING TO GO AHEAD AND
ASK YOU TO DELIVER 10
STORIES NEXT SPRINT...
![Page 39: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/39.jpg)
WTF!!!!!!#%&!
![Page 40: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/40.jpg)
0
1
2
3
4
5
6
7
8
9
10
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
velocity
Average=
LCL
UCL
Target
Actual, measured throughput over 21 sprints
![Page 41: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/41.jpg)
The 4 Estimates Dysfunctions
1. Estimate Bargaining
2. Internal Politics
3. Blame Shifting
4. Late Changes
5. Sunk Cost fallacy
![Page 42: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/42.jpg)
Principle #8The system where you work has
predictable outputs, learn to understand the system
![Page 43: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/43.jpg)
Wow! But I have a business to run!
Is there a way do better than that?
![Page 44: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/44.jpg)
#NoEstimates delivers!
![Page 45: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/45.jpg)
Counting Stories vs. Estimated Story Points
Q: Which ”metric” is more accurate when compared to
what actually happened in the project?
![Page 46: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/46.jpg)
A long project
24Sprints
![Page 47: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/47.jpg)
Which metric predicted most accurately the output of the
whole project?
a) After only the first 3 Sprints
b) After only the first 5 Sprints
![Page 48: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/48.jpg)
Disclaimer...This is only one project!
Find 21 more at: http://bit.ly/NoEstimatesProjectsDB
![Page 49: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/49.jpg)
After just 3 sprints
# of Stories predictive powerStory Points predictive power
The true output: 349,5 SPs
completed
The predictedoutput: 418 SPs
completed
+20%
The true output: 228 Stories
The predictedoutput: 220
Stories
-4%!
![Page 50: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/50.jpg)
After just 5 sprints
# of Stories predictive powerStory Points predictive power
The true output: 349,5 SPs
completed
The predictedoutput: 396 SPs
completed
+13%
The true output: 228 Stories
The predictedoutput: 220
Stories
-4%!
![Page 51: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/51.jpg)
Q: Which ”metric” is more accurate when compared to
what actually happened in the project?
![Page 52: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/52.jpg)
But there is more...
![Page 53: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/53.jpg)
What difference does a Story Point make in a project that used both Story Points and
#NoEstimates?
![Page 54: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/54.jpg)
Next you will see the forecasted release date when
using Story Points (values 1:3:5)
![Page 55: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/55.jpg)
6871 71 71 71 71 71 72 72 72 73 73
0 3 7 7 9 11 12 13
20 20 22 23
0
10
20
30
40
50
60
70
80
90
100
1/3 1/17 1/31 2/14 2/28 3/14 3/28 4/11 4/25 5/9 5/23 6/6 6/20 7/4 7/18 8/1 8/15 8/29 9/12 9/26 10/10 10/24 11/7
Product Backlog Cumulative Flow Diagram
Remaining
Done
Linear (Remaining)
Linear (Done)
Release on 20th October
2014
![Page 56: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/56.jpg)
Next you will see the forecasted release date when
using Story Points (values 1:2:3)
![Page 57: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/57.jpg)
48
51 51 51 51 51 51 52 52 52 53 53
0 2 5 5 7 8 9 1015 15 17 18
0
10
20
30
40
50
60
70
80
1/3 1/17 1/31 2/14 2/28 3/14 3/28 4/11 4/25 5/9 5/23 6/6 6/20 7/4 7/18 8/1 8/15 8/29 9/12 9/26 10/10 10/24 11/7
Product Backlog Cumulative Flow Diagram
Remaining
Done
Linear (Remaining)
Linear (Done)Release on
14th October 2014
![Page 58: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/58.jpg)
Next you will see the forecasted release date when
#NoEstimates (or, all stories = 1 story point)
![Page 59: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/59.jpg)
28
31 31 31 31 31 3132 32 32
33 33
0 1 3 35 5 6 7
10 1012 13
0
10
20
30
40
50
60
1/3 1/17 1/31 2/14 2/28 3/14 3/28 4/11 4/25 5/9 5/23 6/6 6/20 7/4 7/18 8/1 8/15 8/29 9/12 9/26 10/10 10/24 11/7
Product Backlog Cumulative Flow Diagram
Remaining
Done
Linear (Remaining)
Linear (Done)
Release on 29thSeptember 2014
![Page 60: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/60.jpg)
Conclusion...
![Page 61: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/61.jpg)
All dates within 3 weeks of each other in a 38 to 42 week
project (a margin of ~8%)
![Page 62: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/62.jpg)
Data used with permission from Bill Hanlon at Microsoft
”At that point, I stopped thinking that estimating
was important.”
Bill Hanlon: http://bit.ly/BHanlon
![Page 63: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/63.jpg)
In 1986, Profs. S.D. Conte, H.E. Dunsmoir, andV.Y. Shen proposed that a good estimationapproach should provide estimates that arewithin 25% of the actual results 75% of the time
--Steve McConnel, Software Estimation: Demystifying the Black Art
In this presentation you have seen examples that far outperform what estimation specialists consider a ”good estimation”. In common they have one way to look at work: #NoEstimates
![Page 64: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/64.jpg)
#NoEstimates testimonial
The deviation between estimated and actual velocity would have been approximately 15% lower if we would have used #NoEstimates.
We have analyzed data from 50 Sprints…
…at no time the story point based estimation was better than #NoEstimates.
![Page 65: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/65.jpg)
Principle #9Don’t bet your company on poor
track record methods, use methods with a proven track record
aka: Hope is a bad management strategy
![Page 66: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/66.jpg)
Carmen faces a very difficult situation…
![Page 67: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/67.jpg)
The 7 Step Journey To NoEstimates
1. Start using Story Points 2. Stop estimating tasks3. Limit the calendar duration for
Stories and Features4. Reduce the number of
allowed estimates (say, 1,2,3 and 5 only)
5. Track your data6. Use your data7. Simply count the number of
stories
![Page 68: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/68.jpg)
http://j.mp/NoEstimates-Book
![Page 69: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/69.jpg)
http://j.mp/NoEstimates-Book
![Page 70: No estimates - 10 new principles for testing](https://reader033.vdocuments.mx/reader033/viewer/2022050613/58778d8c1a28ab0f778b49c1/html5/thumbnails/70.jpg)
http://j.mp/NoEstimates-Book