understanding standard deviation – bacon bits

8/17/2019 Understanding Standard Deviation – Bacon Bits

1/17


http://datapigtechnologies.com/blog/index.php/understanding-standard-deviation-2/

Bacon BitsDelicious bits of Excel and Access Training from DataPig Technologies

Understanding Standard

Deviation

datapig March 26, 2014 Business Statistics, Excel

Formulas 28 Comments

Standard Deviation is one of those statistical terms thrown

around the corporate world with vague abandon. Many business

analysts don’t truly understand the concept of Standard

Deviation. If you’re one of those folks, you can stop living the lie.

In today’s Pulitzer worthy post, you’ll learn how this

underestimated statistical measure can help you better

understand the data you’re working with.

.

What Standard Deviation Measures

Imagine you supervise two deli managers who sell bacon. Since

you don’t want any bacon to be wasted, it’s important that these

two managers hold a steady inventory. In an effort to measure

how well they manage inventory, you decide to analyze the

boxes of bacon each manager ordered in the last six weeks.

Taking the average of the last 6 weeks shows that each

manager orders an average of around 32 boxes of bacon per

week. On the surface, the averages make it look like they are

performing equally.

Search

Try these Add-ins!

Recent Posts

Power Pivot and Power

Query for Dummies

Editing Shape Points to

Create Custom Graphics

Entering More than 31

Characters in a Worksheet

Tab Name

Understanding the MOD

function

Use Power Query to Get a

List of Hidden and System

Home Add-ins and Training Excel Power BI BootCamp

http://datapigtechnologies.com/blog/index.php/entering-more-than-31-characters-in-a-worksheet-tab-name/http://www.datapigtechnologies.com/ExcelExplosion.htmhttp://datapigtechnologies.com/blog/index.php/category/http://datapigtechnologies.com/blog/http://datapigtechnologies.com/blog/http://www.datapigtechnologies.com/DashboardTools.htmhttp://feeds.feedburner.com/datapigtechnologies/QWkshttps://classic.regonline.com/builder/site/?eventid=1733146http://www.datapigtechnologies.com/Products.htmhttp://datapigtechnologies.com/blog/http://datapigtechnologies.com/blog/index.php/use-power-query-to-get-a-list-of-hidden-and-system-directories/http://datapigtechnologies.com/blog/index.php/understanding-the-mod-function/http://datapigtechnologies.com/blog/index.php/entering-more-than-31-characters-in-a-worksheet-tab-name/http://datapigtechnologies.com/blog/index.php/editing-shape-points-to-create-custom-graphics/http://datapigtechnologies.com/blog/index.php/power-pivot-and-power-query-for-dummies/http://www.datapigtechnologies.com/DashboardTools.htmhttp://www.datapigtechnologies.com/ExcelExplosion.htmhttp://datapigtechnologies.com/blog/index.php/category/http://datapigtechnologies.com/blog/index.php/category/http://datapigtechnologies.com/blog/index.php/understanding-standard-deviation-2/http://datapigtechnologies.com/blog/index.php/author/admin/http://datapigtechnologies.com/blog/http://feeds.feedburner.com/datapigtechnologies/QWks


2/17


http://datapigtechnologies.com/blog/index.php/understanding-standard-deviation-2/ 2

But if you look closer, you’ll see that one of the managers has

weekly orders of 22, 34, 58, 52, 10 and 21 boxes. For this

manager, the average may be mathematically correct, but it

hides the volatility of his weekly orders. In other words,

sometimes the average of a dataset doesn’t do a good job

representing the data. This is where Standard Deviation comes

in.

.

Standard Deviation gives you a sense of how dispersed (spread

out) the data in your sample is from the Mean (Average). Said

another way – it lets you know if you can rely on the Mean to

give you a meaningful representation of the data.

.

In our example, we use the STDEV function in Excel to give us

Standard Deviation along with our Mean.

In the case of the first manager, the Standard Deviation is 2.

This tells us that each data point in the sample sits an average

distance of 2 statistical data points from the Mean (Average). Is

that good? Well, think of it this way – a Standard Deviation of 0

would say every data point is exactly equal to Mean of the

sample (32.3 in this case). So a Standard Deviation of 2 is not

far off from that, indicating that a majority of data points are

positioned extremely close to the Mean. The closer the Standard

Deviation is to 0, the more reliable the Mean is. More than that

Directories

Sites I Visit

BeExcellence

Chandoo Excel Tips

Chris Web BI Blog

Clearly and Simply

Contextures

Daily Dose of Excel

Excel Campus

Excel TV

ExcelGuru

PowerPivot Pro

PTS Blog

SpreadsheetGuru

Your Sum Buddy

Recent Comments

Carl Farrington on Be

Careful Drilling into

PivotTables with Slicers

Rod on Avoiding Data Cut-

Off in Excel

Irion on Be Careful Drilling

into PivotTables with

Slicers

flavioHenrique on Editing

Shape Points to Create

Custom Graphics

Rob on Fill the Empty Cells

Left by Pivot Table Row

Fields

safder on Color Pie Chart

http://datapigtechnologies.com/blog/index.php/color-pie-chart-slices-to-match-their-source-cells/comment-page-1/#comment-1442348http://datapigtechnologies.com/blog/index.php/fill-the-empty-cells-left-by-pivot-table-row-fields/comment-page-2/#comment-1442384http://datapigtechnologies.com/blog/index.php/editing-shape-points-to-create-custom-graphics/comment-page-1/#comment-1442424http://datapigtechnologies.com/blog/index.php/be-careful-drilling-into-pivottables-with-slicers/comment-page-1/#comment-1442462http://datapigtechnologies.com/blog/index.php/avoiding-data-cut-off-in-excel/comment-page-1/#comment-1442547http://datapigtechnologies.com/blog/index.php/be-careful-drilling-into-pivottables-with-slicers/comment-page-1/#comment-1442778http://www.css-networks.com/http://yoursumbuddy.com/blog/http://www.thespreadsheetguru.com/blog/http://peltiertech.com/WordPress/http://www.powerpivotpro.com/http://www.excelguru.ca/blog/http://excel.tv/http://www.excelcampus.com/blog/http://www.dailydoseofexcel.com/http://blog.contextures.com/http://www.clearlyandsimply.com/http://cwebbbi.wordpress.com/http://chandoo.org/wp/http://wmfexcel.com/http://datapigtechnologies.com/blog/index.php/use-power-query-to-get-a-list-of-hidden-and-system-directories/


3/17



though, Standard Deviation close to 0 tells us that there is very

little volatility in the sample. With a Stanadard Deviation of 2, the

first manager’s weekly orders are remarkably consistent.

.

In the case of the second manager, the Standard Deviation is

18.9. The average distance each data point is from the Mean is

18.9 statistical data points. That’s a huge spread! The further

away a Standard Deviation is from 0, the less accurate the Mean

for that sample. In this case, a Standard Deviation of 18.9 alerts

us that the Average shown for this manager (32.8 boxes per

week) is just not reliable. It also indicates that this manager’s

weekly orders are extremely volatile. Of course, with only six

data points, you can confirm the volatility with your eyeball.

.

That is basic Standard Deviation in a nutshell. Although it doesn’t

get the attention afforded to other statistical measures (Mean,

Median, Mode, etc.), Standard Deviation is actually critical to

many statistical calculations. An understanding of how Standard

Deviation works will pave the way for you to do things like:

determine the volatility of a stock, normalize comparisons

between datasets, identify outliers, create standardized z-

scores, and much more.

.

How Standard Deviation is Calculated

OK, we know what Standard Deviation shows. Let’s now take a

look at how it’s actually calculated.

We’ll start with this set of numbers (in black). As you can see,

I’ve already used the STDEV function to calculate the Standard

Deviation of 21.6 (in orange).

Slices to Match their

Source Cells

beth on Excel 2010 Slicers

Keri on Be Careful Drilling

into PivotTables with

Slicers

Alessio on Hack into

Password Protected VBA

Projects

Aayaan s hah on Hack into

a protected Excel 2007 or

2010 Workbook

Categories

Access Functions (6)

Access Queries (15)

Access Tips and Tricks

(23)

Access VBA (25)

Ask DataPig a Question (1)

Business Statistics (11)

Excel & Access Integration

(16)

Excel 2010 (18)

Excel 2013 (18)

Excel Add-Ins (15)

Excel Charts (26)

Excel Formatting Tips (39)

Excel Formulas (47)

Excel Number Formatting

(14)

Excel Tips and Tricks (103)

Excel VBA (81)

Interviews (11)

MS Query (4)

Oldies but Goodies (5)

http://datapigtechnologies.com/blog/index.php/category/http://datapigtechnologies.com/blog/index.php/category/http://datapigtechnologies.com/blog/index.php/category/http://datapigtechnologies.com/blog/index.php/category/http://datapigtechnologies.com/blog/index.php/category/http://datapigtechnologies.com/blog/index.php/category/http://datapigtechnologies.com/blog/index.php/category/http://datapigtechnologies.com/blog/index.php/category/http://datapigtechnologies.com/blog/index.php/category/http://datapigtechnologies.com/blog/index.php/category/http://datapigtechnologies.com/blog/index.php/category/http://datapigtechnologies.com/blog/index.php/category/http://datapigtechnologies.com/blog/index.php/category/http://datapigtechnologies.com/blog/index.php/category/http://datapigtechnologies.com/blog/index.php/category/http://datapigtechnologies.com/blog/index.php/category/http://datapigtechnologies.com/blog/index.php/category/http://datapigtechnologies.com/blog/index.php/category/http://datapigtechnologies.com/blog/index.php/category/http://datapigtechnologies.com/blog/index.php/hack-into-a-protected-excel-2007-or-2010-workbook/comment-page-3/#comment-1442176http://datapigtechnologies.com/blog/index.php/hack-into-password-protected-vba-projects/comment-page-2/#comment-1442221http://datapigtechnologies.com/blog/index.php/be-careful-drilling-into-pivottables-with-slicers/comment-page-1/#comment-1442222http://datapigtechnologies.com/blog/index.php/excel-2010-slicers/comment-page-1/#comment-1442233http://datapigtechnologies.com/blog/index.php/color-pie-chart-slices-to-match-their-source-cells/comment-page-1/#comment-1442348


4/17



.

Here are the steps Excel took to calculate 21.6 as the Standard

Deviation.

Note these steps are purposefully visual to better relay what’s

going on. In reality, all this stuff happens instantly behind the

scenes.

.

First, Excel calculates the Mean (the Average) for the Sample. In

this case, the Mean is 40. It then calculates the difference

between each data point and 40. For instance, the difference

between 50 and 40 is +10 statistical data points. The difference

between 10 and 40 is -30.

.

.

The next thing Excel does is Squares those differences so that

all the differences will be a positive number (+10 would become

100; -30 would become +900).

Pivot Tables (35)

Saturday Bacon Recipe

(41)

Spreadsheet Auditing (11)

Training (22)

Visualizations (69)

Worksheet Protection (8)

Xcelsius (1)

~ Bacon Bits (151)

http://datapigtechnologies.com/blog/index.php/category/http://datapigtechnologies.com/blog/index.php/category/http://datapigtechnologies.com/blog/index.php/category/http://datapigtechnologies.com/blog/index.php/category/http://datapigtechnologies.com/blog/index.php/category/http://datapigtechnologies.com/blog/index.php/category/http://datapigtechnologies.com/blog/index.php/category/http://datapigtechnologies.com/blog/index.php/category/


5/17



.

.

Excel then adds up all the Squared Differences to get the Total

Squared Differences.

Next, Excel uses the Total Squared Difference to calculate the

Sample Variance. This is done by dividing the Total Squared

Differences by the count of data points in the sample minus 1. In

this example the count of data points is 7, so we divide the Total

Squared Differences by 7-1.

.

.

Finally, Excel calculates the square root of the Sample Variance.


6/17



This square root becomes our Standard Deviation.

.

.

Population Standard Deviation vs Sample

Standard Deviation

In statistics, you’ll often hear the terms Population and Sample.

These terms refer to the completeness of the data in your

possession. The differences between the two are sometimes not

all that clear.

.

If you’re using a complete dataset, you’re using a population.

An example of a population would be if you were analyzing the

time in service for all the sales reps in your company. You would

have the data for all the sales reps that exist in your company; a

complete population.

.

If you are using a partial set of data, or a subset of data, you’re

dealing with what is called a sample. An example would be if

you were analyzing sales data for one quarter of a year. A

quarter is merely a subset of an entire year’s data, so in this

case, you’re working with a sample.

.

The reason this population and sample designation matters is

because the calculation for Standard Deviation changes slightly

depending on the nature of the data you’re dealing with.

Specifically, the way you calculate the Sample Variance changes.


7/17



Remember in Step 4 of the calculation steps above? We said

that Excel divides the Total Squared Differences by the count of

data points in the sample minus 1. Well, this only applies if your

data is a sample (a subset of a bigger data pool).

.

If your data is made up of the entire population, the calculation in

Step 4 changes to divide the Total Squared Differences by the

complete count of data points. In other words, there is no need

to subtract 1.

.

This difference in calculation will obviously yield different

Standard Deviations.

.

So why the difference in calculation? Well, the difference is not

an Excel quirk. It’s an actual statistics tenent called Bessel’s

Correction. Bessel’s Correction states that when you use a

sample dataset instead of a population, you need to subtract 1

http://en.wikipedia.org/wiki/Bessel's_correction


8/17



from the count of data points used (written in the statistics world

as N-1). This correction accounts for the bias introduced by

estimating a Mean using a subset of data instead of using the

true population Mean. The reasoning behind the need for this

correction is admittedly a bit difficult for us non-math geeks to

wrap our brains around. I’ll try to explain it in my simplified

understanding.

.

When you use a sample, you’re using a subset of data chosen

from the true population. In this case, you won’t have the benefit

of using the true population Mean. You will need to estimate a

new Mean based on data you grabbed in the sample. This

estimated Mean is already biased towards fitting the data

chosen in the sample, so you need to exclude that one point (the

estimated Mean) from calculating the Sample Variance. This is

called losing a degree of freedom. Another way to look at it is

this. If I said you could have five variables in an Excel formula,

but one of them would have to be used to calculate the Mean,

you would have only four variables available to you (5-1). The

fact that you have to calculate Mean loses you a degree of

freedom.

.

If you use a population set instead of a sample set, you have the

benefit of the true population Mean. So you’re not losing a

degree of freedom by being forced to introduce an estimated

Mean. Thus there is no need to subtract 1.

.

.

Standard Deviation Formulas in Excel

Excel has the ability to handle Standard Deviation calculations

for both population and sample datasets. Simply click in any cell

and start to enter a =STDEV. You’ll see a tool-tip dropdown that

gives you, what seems to be, a ridulous number of Standard


9/17



Deviation functions.

Here’s a quick rundown of what each function does.

STDEV: Calculates Standard Deviation for a sample using

Bessel’s Correction (N-1).

STDEVP: Calculates Standard Deviation for a population.

STDEV.S: Calculates Standard Deviation for a sample using

Bessel’s Correction (N-1). This function technically replaces

the STDEV function.

STDEV.P: Calculates Standard Deviation for a population.

This function technically replaces the STDEVP function.

STDEVA: Calculates Standard Deviation for a sample using

Bessel’s Correction (N-1). Allows for text and TRUE/FALSE

values.

STDEVPA: Calculates Standard Deviation for a population.

Allows for text and TRUE/FALSE values.

.

I can’t imagine any scenario where I would use the STDEVPA

and STDEVA functions. I think you can safely ignore those.

.

As far as I can tell, there is no discernible difference between

STDEV.S and STDEV. Microsoft says you should move toward

the newer STDEV.S function, but you can technically use those

two functions interchangably.

.

Likewise, I don’t see any discernible difference between

STDEV.P and STDEVP. Although STDEV.P is the newer function,

you can use those two functions interchangably.


10/17



← Three Tips for Making your Pivot Table Formatting Stick

Chandoo Interviews DataPig →

.

.

Until next week…Happy Mathing!

Business Statistics, Formulas

28 thoughts on

“Understanding Standard

Deviation”

1. Ola

March 28, 2014 at 1:14 am

Much better than Wikipedia:

http://en.wikipedia.org/wiki/Standard_deviation

Can also compare the std deviation to mean.

2. Roger Govier

March 31, 2014 at 2:41 pm

Very nice article Mike.

I wish this had been around when I did Statistics as part of

my Economics at University. I did manage to grasp to grasp

the techniques back then – but this would have made life so

much easier.

Of course, life would have been much easier if I had Excel

http://en.wikipedia.org/wiki/Standard_deviationhttp://datapigtechnologies.com/blog/index.php/tag/http://datapigtechnologies.com/blog/index.php/tag/http://datapigtechnologies.com/blog/index.php/chandoo-interviews-datapig/http://datapigtechnologies.com/blog/index.php/three-tips-for-making-your-pivot-table-formatting-stick/


11/17



available to me back in 1963!!!

3. Doris Choo

April 8, 2014 at 7:02 pm

The best explanation and simplified version, I came across

so far. Put my

4. Doris Choo

April 8, 2014 at 7:11 pm

Sorry using 2 thumbs typing with my IPAD. Click wrong

button.

I meant – put my old stat teacher to shame. All the other

Stat books used in college are full of mumbo jumbo, makethe subject more complicating than it seems, most all hard

to absorb.

Thanks Mike you are the Best!

5. Peter Bedson

May 12, 2014 at 12:34 pm

Generally, if N> about 20 you can ignore Bessel’sCorrection completely because the sample mean and the

population mean are effectively the same.

6. john k

May 27, 2014 at 7:28 pm

Very nice explanation – simple yet effective.

One thing I would add to the standard deviation section is

Relative St Dev, where

RSD(%) = (stdev/mean)*100

This makes the stdev more meaningful with respect to your

mean. Sure, stdev = 2 is close to zero, but if your mean is

around 5 (RSD = 40%), your data is much more disperse

than if your mean is around 40 (RSD = 5%).


12/17



7. datapig

July 16, 2014 at 8:57 pm

John: Great tip! Thanks!

8. Brad J

July 26, 2014 at 10:53 pm

Excellent summary, wish I had you at my school.

One thing to considering your example is you have to

understand the context of the data. The Deli managers are

different in how they order with the similar average is what

the data tells us. If you know more context about the

operation the data may tell you more. Maybe the sales

target is 32 boxes and one manager just orders close to

that without knowing if it will impact inventory and the other

manager is compensating for the first by balancing sales

and inventory and doing a better gob of forecasting sales.

Alternatively maybe the second manager orders wildly for

other reason and isn’t the better performer. Without

knowing the context of the data all we can say is they order

significantly different but not sure of the reasons or which is

better.

Excellent Summary once again.

9. sankar

July 28, 2014 at 9:40 am

Useful insight

10. Nick Leifeld

July 30, 2014 at 5:28 pm

Very nice explanation, I wish Excel had a STDEV function

for calculating Cpm in which the average variation around

he Target is calculated and not the average variation

around the Mean

http://www.datapigtechnologies.com/


13/17



11. Gabby

August 5, 2014 at 5:43 pm

Thanks for the explanation.

It helps me a lot to differentiate STDEV from MEAN and

how to relate it to my job.

12. Alanex

August 21, 2014 at 5:12 am

Useful and well done article.

13. Sivakumar Viswanathan

September 20, 2014 at 11:02 am

This is the simplest & clearest way to explain SD. Wish I hadthis 2 decades ago. But I come across SD on a daily basis

as a quality evaluation tool for our incoming raw materials. I

hope many of the technical experts in manufacturing refer

to SD without actually grasping the underlying inference

Now I understand exactly what I am talking about. Also

wonder how I topped Statistical Quality Control paper during

my engineering days 22 years back. Thanks for this article

14. julian

October 3, 2014 at 9:47 pm

Congratulation! It is really perfect expalanation!

Unfortunately I read it too late

15. roskin

November 1, 2014 at 5:04 am

wow ,now i understand it ,will definitely share the

knowledge.thanks

16. Ahsan

November 7, 2014 at 10:29 am


14/17



Very Very easy to understand S.D through this example.

17. Tim McGuire

December 20, 2014 at 3:17 pm

Great summary, easy to understand with the example

given.

18. Anna

January 5, 2015 at 11:05 am

Million thanks for this great explanation, you made my day! I

am just writing my Master’s thesis and was in trouble

understanding basic statistics, now I am a step further down

on the road! I will share this article with my classmates!Wish you good work and looking forward reading more

great articles!

19. Yukiko Sawada

January 6, 2015 at 6:50 pm

This article was very interesting for me.

I learned a lot from that.Thank you.

20. Yukiko Sawada

January 6, 2015 at 6:52 pm

This article is interesting for me.

I learned a lot from that, thank you.

21. Lilian Salan

February 9, 2015 at 11:36 am

Now I understand the STANDARD DEVIATION. The

discussion, explanation, and examples are very clear and

simple. Its give real situation for the example.

Thank you so much!


15/17



22. Nandy

March 27, 2015 at 11:52 am

Bravo, this is the brief and clear explanation i have seen.

straight to the point. i wish you could be my stat teacher. i

give you five stars.

23. Moha

May 18, 2015 at 12:53 pm

Thanks for your clear explanation

24. Samuel David

July 14, 2015 at 1:24 pm

Very simple and easy understand example to comprehendStandard Deviation.

Thanks a lot.

Regards,

Samuel David

25. Samuel David

July 14, 2015 at 1:25 pmVery simple and easy to understand example to

comprehend Standard Deviation.

Thanks a lot.

Regards,

Samuel David

26. Ahmad August 26, 2015 at 10:21 am

The explanation is very clearly and sample ,we understood

very easily.

Thank you so much

27. arash


16/17



February 3, 2016 at 9:26 pm

a million thanks.very clear and simple to

understand.it helped me a lot …..

28. Pradeep

February 4, 2016 at 11:03 amVery clearly explained! thanks

Leave a Reply

Your email address will not be published. Required fields are

marked *

Comment

Name *

Email *

Website

Confirm you are NOT a robot

post comment

understanding standard deviation – bacon bits

Documents