power1

1

Power, Power Curves and Sample Size

2

Planning Reliable and Efficient Tests

For any job, you need a tool that offers the right amount of power for the task at hand. You wouldn't use a telescope to examine a stamp collection, or a handheld magnifying glass to search for new galaxies, because neither would provide you with meaningful observations. To complicate matters, if detecting a galaxy really was your goal, the cost of gaining the necessary power might be more than you can afford.

3


Anyone using statistical tests faces the same issues. You must consider the precision you need to meet your goals (should your test detect subtle effects or massive shifts?), and balance it against the cost of sampling your population (are you testing toothpicks or jet engines?).

4


You also want the confidence in your results that's appropriate for your situation (testing seat belts demands a greater degree of certainty than testing shampoo). We measure this certainty with statistical power – the probability your test will detect an effect that truly exists.

5


Minitab's Power and Sample Size tools, with Power Curves, help you balance these issues that may compete for your limited resources. Here are three examples of how a quick Power and Sample Size test can help you save time and money getting results you can trust.

6

Don't Leave Success to Chance

A paper clip manufacturer wants to detect significant changes in clip length. They sample thousands of clips because it is cheap and quick to do. But this huge sample makes the test too sensitive: the broken line (next slide) shows it will sound the alarm if the average length differs by a trivial amount (0.05).

7


0.300.250.200.150.100.050.00

1.0

0.8

0.6

0.4

0.2

0.0

Difference

Pow

er

Alpha 0.05StDev 1Alternative >

Assumptions

1002000

SizeSample

Power Curve for 1-Sample t Test

8


This Power Curve shows they are wasting resources on excessive precision. A sample size of just 100 will detect meaningful differences (0.25) without “crying wolf” at every negligible blip.

9


10


An aerospace company is designing an experiment to test a new rocket. Each rocket is very expensive, so it is critical to test no more than necessary.

11


1.00.50.0-0.5-1.0

1.0

0.8

0.6

0.4

0.2

0.0

Effect

Pow

er

Alpha 0.05StDev 1# Factors 3# Corner Pts 8# Blocks none# Terms Omitted 0

Assumptions

6, 0Ctr Pts Per BlkReps,

Power Curve for 2-Level Factorial Design

12


This Power Curve confirms an experiment with 6 replicates will give researchers the power they need without spending more than they must.

13


14


“We've always done it this way.” That's why a lumber company would sample 10 beams to test whether their strength meets the target.

15


1.00.50.0-0.5-1.0

1.0

0.8

0.6

0.4

0.2

0.0

Difference

Pow

er

Alpha 0.05StDev 1Alternative Not =

Assumptions

1034

SizeSample


16


According to the Power Curve, this small sample size made their test incapable of detecting important effects. They must sample 34 beams to detect meaningful differences (0.50).

17


18

See the Big Picture

A power analysis helps you weigh your resources against your demands, and quantifies a test's ability to answer your question. It can expose design problems, like the lumber company's insufficient sample size. It can also reveal design solutions you hadn't considered.

19

See the Big Picture

Take for instance the packaging plant of a snack company. Customers complain that the company's pretzel bags are sealed with glue that's too strong, so researchers use One-Way ANOVA to compare their current glue with three potential replacements.

20

See the Big Picture

Differences in seal strength less than 10 are undetectable to most people, so their test only needs to detect a difference of 10. A power value of 80% is acceptable, but 90% is ideal.

What sample size meets their needs?

21

See the Big Picture

30 samples of each glue ensure the test detects a difference of 10 with 90% power.

22

See the Big Picture

Or, they could detect the same difference with 23 samples and 80% power. If this represents considerable savings, the researchers may consider using the smaller sample.

23

See the Big Picture

The Power Curve illustrates this information, but it also charts every other combination of power and difference for a given sample size.

24

See the Big Picture

20151050

1.0

0.8

0.6

0.4

0.2

0.0

Maximum Difference

Pow

er

Alpha 0.05StDev 10# Levels 4

Assumptions

2330

SizeSample

Power Curve for One-way ANOVA

25

See the Big Picture

The solid line indicates researchers can attain 90% power with just 23 samples if they are willing to seek a difference of 12 instead of 10. This might just be the ideal choice.

26

See the Big Picture

27

How to Create Power Curves in Minitab

Performing power analyses with Power Curves couldn't be simpler. You supply the factors you know, and Minitab calculates the one you omit.

28


Suppose a trainer wants to compare two training courses for forklift operators.

She will use a two-sample t-test to compare the average scores that operators from each course earned on the final exam.

29


She knows she must detect a difference of 5 in either direction between the two courses with 80% power, and historical data suggest a standard deviation of 5.

But how many participants must she sample from each course?

30


Choose Stat > Power and Sample Size

> 2-Sample t In Differences, type -5, 5 In Power values, type 0.80 In Standard deviation, type 5 Click OK

31


32


6420-2-4-6

1.0

0.8

0.6

0.4

0.2

0.0

Difference

Pow

er

Alpha 0.05StDev 5Alternative Not =

Assumptions

17Size

Sample


33


According to this Power Curve, sampling 17 participants from each class enables her test to detect the difference she seeks with 80% power.

34


35

Putting Power Curves to Use

Without knowing the power of your test, it's hard to know if you can trust your results: your test could be too weak to answer your question, or too strong for your needs. Minitab's Power Curves (available for many common statistical procedures) help you balance your resources against your goals and design a test you can trust that costs no more than necessary.

36

Putting Power Curves to Use

Power Curves graph the dynamic relationships that define power, revealing the big picture and ensuring no option escapes your consideration. And perhaps most importantly, they make power analysis an easier and more accessible part of every project. Empower your test. Trust your results.

power1

Documents

sample size test

power curves

statistical power

quick power

necessary power

power analysis

chancethis power curve

efficient testsminitabs