Confidence IntervalsConfidence Intervalsfeeling comfortable with feeling comfortable with

errorerror

Richard Lambert, Ph.D. Richard Lambert, Ph.D.

General OverviewGeneral Overview

Our goal in conducting a study Our goal in conducting a study is often to estimate a particular is often to estimate a particular parameter of interest with as parameter of interest with as little error as possible. little error as possible.

The % that report they will vote The % that report they will vote for a given candidate is a good for a given candidate is a good example.example.

General OverviewGeneral Overview

The % in your sample reporting they will The % in your sample reporting they will vote for a given candidate if the election were vote for a given candidate if the election were held today is your point estimate of the held today is your point estimate of the corresponding parameter.corresponding parameter.

It is your estimate of the population value It is your estimate of the population value given the sample results. given the sample results.

However, you know it not likely to be exactly However, you know it not likely to be exactly correct because of the influence of sampling correct because of the influence of sampling error.error.

General OverviewGeneral Overview Therefore, the most honest way to Therefore, the most honest way to

express your results is to report a express your results is to report a confidence interval around your best confidence interval around your best guess of the population parameter.guess of the population parameter.

The confidence interval expresses a The confidence interval expresses a range of plausible values that we are range of plausible values that we are confident contains the true population confident contains the true population value and gives a picture of the value and gives a picture of the possible influence of sampling error on possible influence of sampling error on the results.the results.

Political Poll ExamplesPolitical Poll Examples

Follow this link:Follow this link:Hillary takes big lead

This poll showed that 50% of This poll showed that 50% of democrats favored Hillary. If this democrats favored Hillary. If this poll was conducted with +/- 4% poll was conducted with +/- 4% error, that means that the 95% error, that means that the 95% confidence interval for these confidence interval for these results would be 46%-54%.results would be 46%-54%.

ExampleExampleSuppose you want to conduct a survey of Suppose you want to conduct a survey of

all the teachers in your school (N=150). all the teachers in your school (N=150).

Your survey includes just one question: Your survey includes just one question: “Are you in favor of switching to block “Are you in favor of switching to block scheduling?”scheduling?”

You hope every teacher in the school will You hope every teacher in the school will complete the simple online survey and complete the simple online survey and return it by the deadline.return it by the deadline.

ExampleExample Suppose you have responses from 30 teachers and Suppose you have responses from 30 teachers and

you are interested in finding out what these results you are interested in finding out what these results tell you. Assume that there is no systematic non-tell you. Assume that there is no systematic non-response bias, meaning the 30 really are response bias, meaning the 30 really are representative of the 150.representative of the 150.

46.7%, or 14 teachers, said “Yes” (in favor of block 46.7%, or 14 teachers, said “Yes” (in favor of block scheduling) and 53.3%, or16 teachers, said “No” scheduling) and 53.3%, or16 teachers, said “No” (not in favor of block scheduling).(not in favor of block scheduling).

You know that your best guess of the percentage of You know that your best guess of the percentage of teachers in favor of block scheduling, 46.7%, teachers in favor of block scheduling, 46.7%, probably suffers from some influence of sampling probably suffers from some influence of sampling error given the small sample size (n=30).error given the small sample size (n=30).

ExampleExample Using the procedure your text, take a few Using the procedure your text, take a few

minutes and calculate the 95% Confidence minutes and calculate the 95% Confidence Interval for this situation. Interval for this situation.

You should arrive at the following results: You should arrive at the following results: Lower Limit=28.8%, Upper Limit=64.5%.Lower Limit=28.8%, Upper Limit=64.5%.

Therefore, the most honest way to report this Therefore, the most honest way to report this findings would be to say that 46.7% of our findings would be to say that 46.7% of our sample was in favor of block scheduling. We are sample was in favor of block scheduling. We are 95% confident that between 28.8% and 64.5% of 95% confident that between 28.8% and 64.5% of the teachers in our school are in favor of block the teachers in our school are in favor of block scheduling. scheduling.

ExampleExample Now let’s look at some simulation results that Now let’s look at some simulation results that

will help us understand what this confidence will help us understand what this confidence interval means.interval means.

Suppose the true percentage of teachers in your Suppose the true percentage of teachers in your school that favor block scheduling is 50%.school that favor block scheduling is 50%.

Let’s look at what would happen if we took Let’s look at what would happen if we took repeated samples of size 30 from this repeated samples of size 30 from this population.population.

Simulation ResultsSimulation Results

This picture represents 50 separate samples of size 30. For each sample a 95% This picture represents 50 separate samples of size 30. For each sample a 95% confidence interval has been constructed around the estimate of the percentage of confidence interval has been constructed around the estimate of the percentage of the population that would say yes to the survey question. This picture assumes that the population that would say yes to the survey question. This picture assumes that the real population percentage in favor is 50%. the real population percentage in favor is 50%.

95% Confidence Intervals for 50 samples of n=30

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50

Sample Number

Est

imat

e o

f P

op

. %

95% UL

95% LL

Estimate

Simulation ResultsSimulation Results

Notice how the pink point, the sample result, changes from sample to sample due to Notice how the pink point, the sample result, changes from sample to sample due to sampling error. Notice how the upper limit and lower limit to the confidence sampling error. Notice how the upper limit and lower limit to the confidence intervals also change from sample to sample. What does not change is the true intervals also change from sample to sample. What does not change is the true population value. If we were to conduct this simulation over an infinite number of population value. If we were to conduct this simulation over an infinite number of trials, 95% of the intervals we would construct would capture the true value of 50%.trials, 95% of the intervals we would construct would capture the true value of 50%.

95% Confidence Intervals for 50 samples of n=30

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50

Sample Number

Est

imat

e o

f P

op

. %

95% UL

95% LL

Estimate

Simulation ResultsSimulation Results

Notice in this example that most of the intervals, even though they are not the Notice in this example that most of the intervals, even though they are not the same, do include the real population value of 50%. How many do not? How many same, do include the real population value of 50%. How many do not? How many would you expect to capture or not capture 50%?would you expect to capture or not capture 50%?

95% Confidence Intervals for 50 samples of n=30

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50

Sample Number

Est

imat

e o

f P

op

. %

95% UL

95% LL

Estimate

Simulation ResultsSimulation Results There were two intervals that did not There were two intervals that did not

capture 50% and 48 that did.capture 50% and 48 that did.

A 95% confidence interval means that we A 95% confidence interval means that we expect 95 out of every 100 intervals that expect 95 out of every 100 intervals that were constructed using this method to were constructed using this method to capture (or include) the true population capture (or include) the true population parameter, in this case 50%. parameter, in this case 50%.

We know that sampling error will move the We know that sampling error will move the estimate around from sample to sample. estimate around from sample to sample. Sometimes we will over-estimate 50% and Sometimes we will over-estimate 50% and sometimes we will under-estimate 50%.sometimes we will under-estimate 50%.

Simulation ResultsSimulation Results

However, we can be 95% However, we can be 95% confident that any one interval confident that any one interval actually contains the correct value.actually contains the correct value.

By expressing our results with the By expressing our results with the confidence interval, we are confidence interval, we are reporting the possible influence of reporting the possible influence of sampling error on our results.sampling error on our results.