how to avoid being fooled by percent changes
DESCRIPTION
A fictional tractor building company looks to understand why sales have decreased in the past year. Comparing the performance of two sales guys based on control charts yields very different results than a traditional "year-over-year" percent change analysis.TRANSCRIPT
![Page 1: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/1.jpg)
What the board room can learn from the shop floor
or, how to avoid being fooled by percent changes
By Ben Joneshttp://dataremixed.com
![Page 2: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/2.jpg)
Meet Jim
![Page 3: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/3.jpg)
(Jim says Hi)
![Page 4: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/4.jpg)
Jim runs a company selling…tractors.
![Page 5: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/5.jpg)
Tractor Sales are down 3.6% this year
-3.6%
![Page 6: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/6.jpg)
Jim wants answers…
![Page 7: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/7.jpg)
Meet Jim’s two sales guys, Joe and Larry
Joe Larry
![Page 8: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/8.jpg)
Larry’s numbers are up 2.9%…
Larry
+2.9%
![Page 9: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/9.jpg)
Joe’s numbers are…not.
Joe-8.3%
![Page 10: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/10.jpg)
What should Jim do?
![Page 11: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/11.jpg)
(Jim is polishing up his resume…)
![Page 12: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/12.jpg)
but what if it was all just a big misunderstanding?
![Page 13: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/13.jpg)
Instead of just looking at % change “Year
Over Year”…
-3.6%
![Page 14: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/14.jpg)
…what if we plotted units sold by month?
![Page 15: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/15.jpg)
and what if we applied some brand new
techniques?
http://amzn.to/dOWKBwWalter Shewhart written in 1939
![Page 16: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/16.jpg)
to understand the variation in the data:
x-bar (or average)
UCL – Upper Control Limit
LCL – Lower Control Limit
“Common Cause” variation
“Special Cause” variation
“Special Cause” variation
![Page 17: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/17.jpg)
So wait, only LAST month is a “down” month…
![Page 18: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/18.jpg)
…and WHO do you think caused THAT?
Joe Larry
![Page 19: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/19.jpg)
Probably Joe, right?
Joe-8.3%
![Page 20: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/20.jpg)
But if we look at Joe’s numbers over time…
-8.3%
2009: 96 Units
![Page 21: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/21.jpg)
But if we look at Joe’s numbers over time…
-8.3%
2010: 88 Units
![Page 22: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/22.jpg)
Joe
We see no statistically significant change!
“Common Cause” variation
![Page 23: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/23.jpg)
But Larry can’t be the reason, his numbers are
up, right?
Larry
+2.9%
![Page 24: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/24.jpg)
Let’s look at Larry’s numbers over time…
2009: 69 Units
![Page 25: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/25.jpg)
Let’s look at Larry’s numbers over time…
2010: 71 Units
![Page 26: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/26.jpg)
Looks like Larry has some explaining to do about
December…
Larry“Special Cause” variation
![Page 27: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/27.jpg)
These two approaches tell two very different stories…
Percent Change “YOY”
Control Charts
![Page 28: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/28.jpg)
Blaming Joe would be mistaking NOISE for a
SIGNAL
![Page 29: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/29.jpg)
Rewarding Larry would be missing the opportunity
to identify a SIGNAL
![Page 30: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/30.jpg)
Either way, Jim’s business loses an opportunity to
LEARN and GROW…
![Page 31: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/31.jpg)
…and the fate of our sales guys’ careers hangs
in the balance…
Joe Larry
![Page 32: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/32.jpg)
The Morals of the Story:• DON’T just compare % change
• DON’T assume a % change is statistically significant just because you don’t like it.
• DO use a Control Chart to filter out noise in a data set and identify signals
• NEVER adjust the y-axis to cross the x-axis at anything other than 0 (did you catch that trick? The column charts grossly over-exagerate the changes)
![Page 33: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/33.jpg)
For bar charts, always set the y-axis to 0
y-axis starts at 157 y-axis starts at 0
Appears to be 4X less
![Page 34: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/34.jpg)
Got all that, Jim?
![Page 35: How to avoid being fooled by percent changes](https://reader033.vdocuments.mx/reader033/viewer/2022061206/54832a96b4af9f730d8b498f/html5/thumbnails/35.jpg)
More about Control Charts (in business)
• Understanding Variation– Donald J. Wheeler
Presentation by Ben Joneshttp://dataremixed.com