stefaan luckermans - number for passion, passion for numbers - eurostar 2010
DESCRIPTION
EuroSTAR Software Testing Conference 2010 presentation on Number for Passion, Passion for Numbers by Stefaan Luckermans. See more at: http://conference.eurostarsoftwaretesting.com/past-presentations/TRANSCRIPT
Numbers for Passion
Passion for Numbers
1
Stefaan Luckermans
2
Owner of
Father of Jordy and Ndu Husband of Thabile
Content Table
1. Rationale : Why metrics?
2. Resistance : Why metrics raise resistance?
3. Passion : Why metrics are linked to passion?
4. Support : Why metrics are accepted?
5. Benefit : Why are metrics in demand?
3
Rationale: Why metrics?
Management has set 3 priorities:
– More quality– Better planning of software development activities– Better/Proof of traceability that testing covers the requirements
Management likes balanced scorecards, KPI’s, etc.
Management wants to see one ‘score’ even for more complex matters. E.g. traceability is 94%.
4
Rationale: Why passion?
Management’s view on their testers:
– They are professionals/experts– Their intention is to do their best `– They know what to do– They are individualists but function well in a team– They want quality
Management says:
– ‘Our testers are passionate about their job’
5
Rationale: Why passion?
Management’s expectations:
– Testers must be professionals/experts– It is common sense that they do their best `– They must know what to do– They work for one company– It is normal they want quality
– Our testers must be passionate
6
Rationale: Why passion and numbers?
7
Management wants a number to confirm what they think, expect and say
Rationale: Why passion and numbers?
8
Management wants a number to confirm what they think, expect and say
Management wants a number to point out to them possible issues, even before these
issues become problems
Rationale: Why passion and numbers?
9
Management wants a number to confirm what they think, expect and say
Management wants a number to point out to them possible issues, even before these
issues become problems
Content Table
1. Rationale : Why metrics?
2. Resistance : Why metrics raise resistance?
3. Passion : Why metrics are linked to passion?
4. Support : Why metrics are accepted?
5. Benefit : Why are metrics in demand?
10
Resistance?
11
What is it with numbers ?
I like numbers why don’t you like numbers ?
Numbers mean different things to different people?
The psycho(logy) of numbers
Psychology of numbers?
12
Sentiment ?
Feeling ?
Reaction ?
Psychology of numbers?
13
Sentiment ?
Feeling ?
Reaction ?
Psychology of numbers?
14
TEST RESULTS
94%
YES!
First of the class,
My parents will be pleased
68%
Good enough for
me, but my
parents…
35%
I did study,
Just don’t
understand it.
The teacher must
think I am stupid !!
49%
I studied and did
better than last
time (40%).
I know the teacher
is proud of me !!
Psychology of numbers?
15
35%I did study,Just don’t
understand it.The teacher must
think I am stupid !!
35%Does not reflect the
effort I put in studying
35%Why should I do the effort to study, it has
no result !!
35%I am not good in
doing tests !!
35%She will not let me
pass !!
35%Is reason enough to
stop studying
Psychology of numbers?
16
35%I did study,Just don’t
understand it.The teacher must
think I am stupid !!
35%Does not reflect the
effort I put in studying
35%Why should I do the effort to study, it has
no result !!
35%I am not good in
doing tests !!
35%She will not let me
pass !!
35%Is reason enough to
stop studying
Content Table
1. Rationale : Why metrics?
2. Resistance : Why metrics raise resistance?
3. Passion : Why metrics are linked to passion?
4. Support : Why metrics are accepted?
5. Benefit : Why are metrics in demand?
17
Psychology of numbers?
NUMBERS ONLY RESULT IN RESISTANCE IF THE PEOPLE ARE PASSIONATE OF WHAT
THEY ARE DOING
18
Psychology of numbers?
19
35%I did study,Just don’t
understand it.The teacher must
think I am stupid !!
35%Does not reflect the
effort I put in studying
35%Why should I do the effort to study, it has
no result !!
35%I am not good in
doing tests !!
35%She will not let me
pass !!
35%Is reason enough to
stop studying
Franklin D. Roosevelt
Men and women are
not prisoners of fate,
but only prisoners of
their own minds.
20
Why resistance?
20
Franklin D. Roosevelt
Men and women are not
prisoners of fate, but only
prisoners of their own minds.
21
Why resistance?
But How can we support
them to change their
minds?
21
Franklin D. Roosevelt
Men and women are not
prisoners of fate, but only
prisoners of their own minds.
22
Why resistance?
But How can we support
them to change their
minds?
22
Content Table
1. Rationale : Why metrics?
2. Resistance : Why metrics raise resistance?
3. Passion : Why metrics are linked to passion?
4. Support : Why metrics are accepted?
5. Benefit : Why are metrics in demand?
23
Support
Management wants evaluation
– Could be perceived as an accusation
Passionate testers want/need support
– If there was a way to find out themselves before management knew
24
Elias Canetti
People love as self-
recognition what
they hate as an
accusation
25
Elias Canetti (Bulgarian: Елиас Канети; 25 July 1905–14 August 1994) was a Bulgarian-born novelist and non-fiction writer of Sephardi Jewish
ancestry who wrote in German. He won the Nobel Prize in Literature in 1981
25
Support
Management wants evaluation
– Could be perceived as an accusation
Passionate testers /need support
– If there was a way to find out themselves before management knew
26
TOOL
Why Tool?
Impersonal
Uniform
Best practices
Track history
Evidence
Plenty of triggers to make it a success
27
Triggers
POSITIVE
+
100%
NEGATIVE
-
0%
28
Triggers
POSITIVE
+
100%
NEGATIVE
-
0%
29
Just like Pavlov’s dogs, also
humans are conditioned to
react in a specific way on
certain signals, colors,
sounds, symbols, etc.
Tool
Excel tool
Traceability from test design to test cases to test execution
Traceability from defects
Test coverage of test design and test cases
Defect recognition/ Defect resolution
30
Based on one of the three priorities set by management
Traceability
Traceability
Test setsTest execution
TCS
TDS
Void
31
Test setsTest execution
TCS
TDS
TDS can be traceable to multiple Test sets
Test sets can be traceable to multiple TDS
Void
TDS can be
traceable to
multiple
TCS
TCS can be
traceable to
multiple
TDS
TCS can be traceable to nothing (TCS not
covering TDS and/or not used in a test set
TDS can be traceable to nothing (TDS not yet covered)
Test set can be
traceable to
multiple TCS
TCS can be
traceable to
multiple
Test sets
Traceability
Test sets
can be
traceable to
nothing
(empty test
set
32
Test setsTest execution
TCS
TDS
Void
TDS can be
traceable to
multiple
TCS
TCS can be
traceable to
multiple
TDS
MA_1_1
MA_1_3
TDS can be traceable to multiple Test sets
Test sets can be traceable to multiple TDS
MA_1_2 MA_1_14
TCS can be traceable to nothing (TCS not
covering TDS and/or not used in a test set
MA_1_4
MA_1_10
TDS can be traceable to nothing (TDS not yet covered)
Test sets
can be
traceable to
nothing
(empty test
set
MA_1_11
Test set can
be traceable
to multiple
TCS
TCS can be
traceable to
multiple
Test sets
MA_1_5
MA_1_6 MA_1_7
MA_1_8MA_1_9
MA_1_12
MA_1_13
MA_1_14
Traceability
33
MA_1_2
MA_1_1
MA_1_3
MA_1_4
MA_1_5 MA_1_7 MA_1_8
MA_1_11
MA_1_12 MA_1_13MA_1_14
MA_1_14
34
Tool
35
Testers
Management
Excel assessment tool, only 5 numbers
Excel (self) assessment tool, 5
groups, 43 numbers
Excel self assessment tool, 5 groups, 43 numbers, Training
material, organization wide rollout
36
Content Table
1. Rationale : Why metrics?
2. Resistance : Why metrics raise resistance?
3. Passion : Why metrics are linked to passion?
4. Support : Why metrics are accepted?
5. Benefit : Why are metrics in demand?
37
Management
Have a SPOC to get a report + analysis
Think this tool is good enough
– It pushes people to work in a uniform way
– It allows to compare projects
– It allows to see progress over time
– It allows to see improvement over time
38
Test managers
Must include final evidence in test summary report
Can use the tool when they want
Have a SPOC when they encounter problems
Think this tool is not (good) enough
– Simplification – less numbers
– Not enough – more numbers
– Buttons for more support -> improvement lists
39
Test managers
Must include final evidence in test summary report
Can use the tool when they want
Have a SPOC when they encounter problems
Think this tool is not (good) enough
– Simplification – less numbers
– Not enough – more numbers
– Buttons for more support -> improvement lists
40
TOOL IS ALIVE
NO RESISTANCE
LOTS OF FEEDBACK
Feedback vs. Resistance
Feedback: use it, but
Traceability to bus. Requirements
Info on consistency of a test stage (test level)
Improvement lists
Changing baselines
Scope creep
Resistance : can’t use because
My project has no test stages(test levels)
I have no test design, nor test plan
Test cases in excel
41
Feedback vs. Resistance
Feedback: use it, but
Traceability to bus. Requirements
Info on consistency of a test stage (test level)
Improvement lists
Changing baselines
Scope creep
Resistance : can’t use because
My project has no test stages(test levels)
I have no test design, nor test plan
Test cases in excel
42
Training
What came out of it?
Excel – ms access – (.net, sql server)
Improvement lists are a fact
Data consistency check per test level
Traceability check from and to business requirements
(Baseline consistency check)
43
What came out of it?
Excel – ms access – (.net, sql server)
Improvement lists are a fact
Data consistency check per test level
Traceability check from and to business requirements
(Baseline consistency check)
44
Conclusion
General usable
Worth a try
Is free
45
Future 4 prototypes are ready
Documentation is being developed
Development of final tool has to start
Partners are needed/wanted
Open source : www.sweetthaste.org
Versions:
– In the cloud
– Desktop application single user
– Web application multi user
– Mobile : iPhone and iPad
46
4747