Download - Measuring team performance with Polls
MEASURING TEAM PERFORMANCE
Peter Rainey Delivery Lead | Project Manager
@PeterRaineyAU
Not everything that can be measured should be!
Delivery Leads Melbourne Meetup - 10th March 2016
WELCOME
• Who I am?
• Mobile devices are welcome, please set to silent but keep it handy.
• Poll Anywhere - Interactive
• Why this topic? - Hackathon
• Some questions to get to know our group
• Group exercise
• Close
Measuring Team performance - Not everything that can be measured should be!
2
Measuring Team performance - Not everything that can be measured should be!
3
If you would like to participate in tonight’s session, I’ll be using “Poll Everywhere”.
This allows you to respond to polls or vote using the app on a smartphone or tablet.
Respond via … iOS and Android
1) Install the Poll Everywhere App
2) Join presentation…
PollEv.com/dlmelbmeetup676
Respond via … Web browser
PollEv.com/dlmelbmeetup676
OR
Measuring Team performance - Not everything that can be measured should be!
4
Q. If your team gave you a box and walked away.
What would be inside?
WHY THIS TOPIC?
Measuring Team performance - Not everything that can be measured should be!
5
Well it because of Hackathons…
I recently participated in a couple of hackathons. Which was very satisfying and great fun.
It amazed me how over such a short amount of time, individuals who didn’t know each other
could work together and produce a solution that exceeded the problem owners expectations.
All in 48 hours!
It may be wonder how this was possible. I compared this experience to the teams I’ve worked
with in the past. They were not like this at all.
Performance was not really measured but if I think about it, there was a real time measuring
(monitoring) taking place with all the team members.
Clear common goal – Trust – collaboration – communication – regular feedback – purpose –
accountable – alignment – skill mix
Measuring Team performance - Not everything that can be measured should be!
6
Q. Have you participated in a hackathon or innovation day?
Measuring Team performance - Not everything that can be measured should be!
7
Stages of
team growth Commonly used framework, developed in the mid-1960s by Bruce W. Tuckman.
Measuring Team performance - Not everything that can be measured should be!
8
Q. At what stage is your team on their journey to high performance?
Measuring Team performance - Not everything that can be measured should be!
9
What is a team?
A group of people with a full set of complementary skills required to complete a task, job, or
project. A team becomes more than just a collection of people when a strong sense
of mutual commitment creates synergy, thus generating performance greater than
the sum of the performance of its individual members.
What is measurement?
Values made meaningful by quantifying into specific units. Measurements act as labels
which make those values more useful in terms of details. For example, instead of saying that
someone is tall, we can specify a measurement and specify that the individual is 6 feet tall.
What are metrics?
Metrics are the means to monitor how one is performing against one's goals. Metrics are best
when defined based on a clear set of objectives. Metrics are not the goal.
SO IS THAT ALL WE HAVE TO DO
Measuring Team performance - Not everything that can be measured should be!
10
Measure something and it will magically happen?
How does measuring weight ensure weight loss?
How does tracking monthly revenue actually bring in the money?
What should you measure and how should you report on it?
Don’t waste your valuable time measuring that provoke no questions and generate no
action.
Perhaps the team is a better position to decide the measures.
Measuring Team performance - Not everything that can be measured should be!
11
What makes a good metric?
Meaningful. They should tie back to something your company wants to
achieve. For example, when your goal is highly responsive support, time to
first response is an ideal metric. Resolution time may not matter.
Moveable. You should measure things on which your team can have
impact. In Taylor Morgan’s words, “If there are metrics that aren’t moving,
or we feel like they aren’t important, we just drop them.”
Authentic. Your reports must tell a true story. It’s possible to use real numbers
to send a misleading message. Be honest even when it hurts.
Contextualized. Numbers in isolation can be stripped of meaning, so
provide them in context.
Consistent. The trends over time are usually more important than specific
data, and looking back over a quarter or a year can give you some
fantastic insights and encouragement.
Measuring Team performance - Not everything that can be measured should be!
12
Q. What comes to mind when thinking about measuring team performance?
Word Cloud
SPORT IS A WORLD OF MEASUREMENT
• Weight
• Time
• Distance
• Height
• Speed
• Power
Measuring Team performance - Not everything that can be measured should be!
13
Measuring Team performance - Not everything that can be measured should be!
14
I love my sport. I am a swimmer and triathlete.
When I first started I would study races and memorise the
times of all my competitors. This obsession spread into
studying world ranked swimmers. Their personal bests,
world records and their winning times at various events.
I can spend hours looking at training data and race results
Its worse now as more data is available. Power, Heart Rate,
DPS, stride length, cadence etc.
I feel this can lead to distraction and not being in the
moment especially when training and racing.
Measuring Team performance - Not everything that can be measured should be!
15
Question: Who knows what Sabermetrics is?
Answer: SABeRmetrics is the statistical analysis of baseball.
SABR (Society for American Baseball Research)
It began in the middle of the 20th century and turned mainstream
after the bestselling book Moneyball was released in 2004. A movie
was released in 2011 starring Brad Pitt.
It also has it’s own Manifesto.
So what is Moneyball’s message?
Measuring the right
things in the right way
Measuring Team performance - Not everything that can be measured should be!
16
Moneyball’s message…
Moneyball describes how the Oakland Athletics baseball team
used carefully chosen statistics to build a winning team with a
limited budget.
Their rigorous statistical analysis demonstrated that on-base
percentage and slugging percentage were better indicators of
success, and these player qualities were cheaper to obtain on
the open market than more historically valued qualities.
So statistics such as stolen bases, runs batted in, and batting
average, typically used to gauge players value, where seen as
subjective, often flawed and relics of the 19th-century.
Measuring the right
things in the right way
Measuring Team performance - Not everything that can be measured should be!
17
Moneyball’s message con’t…
By re-evaluating the strategies that produce wins on the field
the team in 2002 won 20 consecutive games and also finished
first in the League.
With only a budget of US$44 million they were competitive with
larger teams such as the New York Yankees, who spent over
US$125 million.
The key business lesson…. businesses continue to use the wrong
statistics.
Measuring the right
things in the right way
SPORT IS A WORLD OF MEASUREMENT
Lets take a sport like Aussie Rules Football.
Players attempt to kick goals.
Lets pretend we are a coach selecting a new player for the team.
We are considering a player who kicks an average of 7 goals a game.
Question: Are they … a Good player, a Bad player or a Great player?
Answer: A good coach would know based on this limited information it would be impossible to answer this question with certainty.
Football is more than just scoring…but some fans would disagree.
Measuring Team performance - Not everything that can be measured should be!
18
SPORT IS A WORLD OF MEASUREMENT
Players can tackle, mark, handball and assist their team mates with scoring….but they can also make mistakes that may benefit other team.
Some additional information that might help….
If I tell you this player averages 7 goals, 20 tackles and 15 marks a game.
Question: Are they… a Good player, a Bad player or a Great player?
Answer: A poor coach might be tricked into answering this question.
A good coach would realise those statistics only mention certain aspects of effectiveness, to truly assess this player you would still need more information.
Measuring Team performance - Not everything that can be measured should be!
19
SPORT IS A WORLD OF MEASUREMENT
A good coach would want to know how the player made use of their resources.
More information that might help to know…
• How many shots at goal did they make
• How many mistakes did they make
• How many possessions did their team have during game
• How many minutes did they play
Measuring Team performance - Not everything that can be measured should be!
20
Example for one player
Ok suppose I gave you all the statistics.
Would you feel more comfortable making a decision on
that player?
SPORT IS A WORLD OF MEASUREMENT
What about situational data?
Suppose those 7 goals were accumulated by a tall 25 year old man, against the worst defences, in one-sided games, in a league of 7 year olds!
Question: Do you still feel good about this players statistics?
Not every set of data is sufficient in helping us make good decisions.
Measuring Team performance - Not everything that can be measured should be!
21
Measuring Team performance - Not everything that can be measured should be!
22
• Performance metrics can actually motivate the behaviours of athletes, employees, students
and business partners.
• The key is to use an appropriate metric. What is appropriate will depend on the team as
every team is different.
• The current stage (Forming, Storming etc) of the team should be taken into account when
deciding on.
Be mindful that the longer particular metric is used, the more likely it can are to be
unconsciously - or for that matter consciously- “gamed”.
Appropriate Use of Metrics
Measuring Team performance - Not everything that can be measured should be!
23
Here is a common statement that I came across a number of times.
“We measured the wrong thing. The statistic we relied on to assess our performance was disconnected from our overall objective of profitability. As a
result, our strategic and resource allocation decisions didn’t support that goal.”
Appropriate Use of Metrics
Measuring Team performance - Not everything that can be measured should be!
24
Velocity Burndown
charts
Items (stories)
in-progress
Peer code
reviews
Acceleration
Acceleration
Interruptions
per day (limited
time metric)
Task hour
burndown
Feature
validation with
customers/users
Anonymous
surveys
Obstacles
escaping
iterations
Resource
Utilization
Adherence to
doneness/
acceptance
criteria
Usability tests
(i.e., time on
task)
Adherence to
checklists
(checklist item
burnup)
Earned Value
Backlog
prioritization
Estimates vs
actuals
Story point
burnup
Trust level within
the team
Hours Worked
Increasing T-
shaped skills
Value delivered On time
delivery
Learning fast
and frequently
Lines of Code
Common Measures
MEASURE WISELY
Measuring Team performance - Not everything that can be measured should be!
25
Suggestions of Good Metric characteristics
(Hartmann & Dymond - 2007)
• Affirms and reinforces Lean and Agile principles.
• Follows trends, not numbers.
• Belongs to a small set of metrics and diagnostics.
• Measures outcome, not output.
• Is easy to collect. • Provides fuel for meaningful conversation.
• Provides feedback on a frequent and regular basis.
• May measure Value (Product) or Process.
• Encourages "good-enough" quality.
Evaluation Checklist for a Metric
PERFORMANCE IN REAL TIME
Measuring Team performance - Not everything that can be measured should be!
26
Many measures are historical. It may be better not to assess teams by looking at
lagging indicators.
But instead get a better understanding of the current level of teamwork.
How can you really measure this without relying on subjective statements like “We
work well as a team”?
Some ways to diagnose a team’s potential and performance in real time…
1. Measure Interaction
2. Address Conflict
3. Align Decisions
4. Create Accountability
5. Trust
PERFORMANCE IN REAL TIME
Measuring Team performance - Not everything that can be measured should be!
27
1. Measure Interaction
Teams thrive when their members are courteous to each other, take time to fully
understand where team members are coming from, and expand on ideas from other
team members to create new strategies previously thought impossible. You can
measure those interactions by recording how many positive or encouraging
comments versus negative or degrading comments are made during meetings.
Because negative emotions are much more powerful than positive ones, high-
performing teams consistently show ratios between 3-to-1 and 7-to-1 (positive to
negative). Anything above 8-to-1 signals artificial harmony and avoidance of tackling
tough issues, while ratios below 1-to-1 are signs of failing and dysfunctional teams.
PERFORMANCE IN REAL TIME
Measuring Team performance - Not everything that can be measured should be!
28
2. Address Conflict
For many, conflict is scary and needs to be avoided at all cost. This is true when we
are talking about negative or combative conflict in which team members are
personally attacked, or if some members have a win-at-all-costs, my-way-or-the-
highway attitude. But avoiding conflict entirely means you may not be tackling the
challenges that really need to be addressed. High-performing teams thoroughly
discuss these critical challenges and exchange their experiences and ideas openly to
find the best possible solution. In most cases, this is a solution no one thought of at the
beginning of the meeting.
Record how many team members actively participate in the idea-generation and
selection process. High-performing teams strive for an equal participation of all team
members, while dysfunctional teams have just one or two people who dominate the
discussion.
PERFORMANCE IN REAL TIME
Measuring Team performance - Not everything that can be measured should be!
29
3. Align Decisions
Critical decisions should be made only after all team members have contributed to,
aligned with, and bought into the developed idea. Everyone should be deeply
knowledgeable about the important details and able to defend the decision based
on facts.
Dysfunctional or “OK” teams agree to decisions made by the most vocal person and
show no commitment to follow through with the steps outlined for implementation.
PERFORMANCE IN REAL TIME
Measuring Team performance - Not everything that can be measured should be!
30
4. Create Accountability
Members of high-performing teams hold each other to the same behavioral and
performance standards and enforce these standards without the team leader’s help.
Do team members show up on time, and are they well-prepared for the team
meetings? Are disruptions, such as side conversations, checking e-mails, and playing
with electronic gadgets, kept to a minimum?
When team members don’t enforce standards and rely on the leader to enforce
rules, they’re really saying that they do not care enough for each other, don’t
understand that it is their responsibility to speak up, or fail to get engaged and put in
the necessary work.
PERFORMANCE IN REAL TIME
Measuring Team performance - Not everything that can be measured should be!
31
5. Trust
Trust, similar to employee engagement, is a psychological construct. It needs to be
made demonstrable in order for it to be measured. In other words, what will be the
tangible evidence that trust exists and how will this be captured?
Identify the types of things that are associated with trust. Put these on a continuum
where you see trust identifiers at one end and the opposite at the other. Assign a
rating scale between the continuums.
Curiosity Defensiveness
Active open listening Poor listening & monologues
Direct communications Indirect communication
Transparency & Clarity Hidden & Distortion
Receptive to Change Doubting of Change
Measuring Team performance - Not everything that can be measured should be!
32
Q. What should you measure when measuring team performance?
Brainstorm
&Vote
GROUP EXERCISE
10-15 mins
1) Please form 4 – 5 groups
2) Each group should select at least two of the measurements listed in previous poll.
3) Pick the most important one according to your group
4) At the end of 10-15 mins present why it was considered the most important by the group
5) Lets discuss
Note: To extend this further the group could also discuss the measure that should not be used.
Measuring Team performance - Not everything that can be measured should be!
33
KEY POINTS
• Something that is measurable is not necessarily meaningful
• Choose the right metrics
• Measure the right things in the right way
• Measurement can drive behaviour
• Don’t be distracted by looking at lagging indicators
• Measure performance in real time
• Avoid ongoing measurement for measurement’s sake.
• Regularly re-evaluate the measures you are using
• Not everything that can be measured should be!
Measuring Team performance - Not everything that can be measured should be!
34
Measuring Team performance - Not everything that can be measured should be!
35
“Not everything that matters can be measured
and not everything that can be measured matters”
- Albert Einstein
THANK YOU!…
FOR SHARING YOUR EXPERIENCES
Also thanks to our hosts
and Meetup organiser Olga Burakevych &Louise Lonergan .
Measuring Team performance - Not everything that can be measured should be! 36
ONE LAST THING…
Measuring Team performance - Not everything that can be measured should be!
37
Tonight's Meetup was “__________________”.
REFERENCES & INSPIRATION
Cannot Measure Productivity by Martin Fowler
Hack: Up or back by Group of MBA Candidates (IMD Lausanne Switzerland)
The Juice in Hackathons is in the Teams, Not the Ideas by Mark Batson Baril
How commitment affects team performance by J W Bishop, Dow Scott
No-Nonsense Guide to Measuring Productivity by W. Bruce Chew
The True Measures of Success by Michael J. Mauboussin
What Gets Measured Gets Done. Or Does It? By Ruth Henderson
An Appropriate Use of Metrics by Patrick Kua
Appropriate Agile Measurement by D. Hartmann and R. Dymond
Guidelines for Measuring Trust by Katie Delahaye Paine
What is a Good Agile Metric? by Mark Levison
Measuring Trust by Third Horizon
Measuring Team performance - Not everything that can be measured should be!
38