Sample Size/Power Calculation:

Principles and ApplicationsIntroduction to Statistical Software

6/1/18

Presented by the HDFCCC Biostatistics Core

Outline

§ Introduction to HDFCCC Biostatistics Shared Resource

§Basic Principles of Hypothesis Testing & Sample size & power calculation

§Demo of Sample Size/Power Calculation

§Statistical Software

§Speakers

Mi-Ok Kim, PhD Li Zhang, PhD Alan Paciorek, BS

Biostatistics Core

§Core Website: http://cancer.ucsf.edu/research/cores/biostatistics

§Key Services

• Study design, conduct, data analysis and reporting, teaching

• Committed to new research development (grant application or protocol development), mentoring & teaching of junior faculty, fellows/post-doc

§Collaboration Request Submission: We strongly encourage contacting us as early as possible. We recommend minimum two weeks in advance for brief consultations, two month for any consultation requiring a significant time investment including grant development.

Basic Principles of Hypothesis Testing & Sample size & power calculation

What you’ll learn in this part

§Basic Principles of Hypothesis Testing

§Basic Principles of Sample Size & Power Computation

Basic Principles of Hypothesis Testing §Hypotheses

• Predictions or explanations that are suggested by knowledge or observation but have not, yet, been proved or disproved

• Testable (verifiable or falsifiable)

§Types of Hypothesis

• Null Hypothesis (H0): represents a theory that has been put forward, either because it is believed to be true or because it is to be used as a basis for argument, but has not been proved.

• Alternative Hypothesis (HA or H1): typically a statement of what a hypothesis test is set up to establish.

‒ Opposite of H0, frequently a actual desired conclusion of researchers

Basic Principles of Hypothesis Testing

§Types of Decisions: reject H0 or not reject H0 (NOT accept Ha/H0)

§Type I error (α): selected error rate of rejecting H0 when it is true. 5% is a generally well accepted error rate.

§Type II error (β): selected error rate of failing to reject H0 when it is not true. 10~20% is generally well accepted.

§H0: “believed to be true or used as a basis for argument, but has not been proved”

• The type I error is more serious and is targeted to be controlled.

Basic Principles of Hypothesis Testing

§Based on an appropriate test, conduct of a hypothesis test underH0 (assuming H0 is true)• Compute a test statistic. (its sampling dist. under H0 is known)

• Compute a probability of observing a test statistic value as or more unusual (extreme) than observed (p-value) under H0.

• Small p-value = small chance of obs. data by chance under H0

• Reject H0 if the p-value < Type I error rate (α) = reject H0 since the chance of observing as or more unusual data is smaller than the (selected, so accepted) risk of the decision being a mistake.

Basic Principles of Hypothesis Testing §H0: the survival rate is the same for trt and control group (no diff.)

§HA: the survival rate is different between trt and control group

under the fixed margins:

Basic Principles of Hypothesis Testing §H0: no difference vs HA: difference in the survival rate

§Do not reject H0, since p-value of 0.07 > 0.05 (selected α level)

§Prob. of as or more extreme data in the direction of interest

• P-value/2=0.035. Why couldn’t we reject H0 based on this p-value is <0.05?

• No, since this p-value needs to be smaller than α/2.

Basic Principles of Hypothesis Testing §By computing the prob. of as or more extreme data in the direction

of interest , we changed

• HA: survival rate in control < survival rate in trt

• H0 is a complement of HA, that is, what is not HA . Hence H0 has changed to

‒ H0 :survival rate in cntrl ≥ survival rate in trt.

‒ Considering a generally positive effect of trt, this requires that the survival rate in the trt group is as high as that of the control group to be “believed to be true or accepted as a basis for argument”.

‒ As long as the possibility of survival rate in cntrl > survival rate cannot be ruled out, do two-sided tests (hence allowing 50% of type I error in the direction of interest)

Basic Principles of Hypothesis Testing §H0: no mean dif. vs. HA: mean in the control ≠ mean in the trt

• Test statistic = mean difference (control – trt) / natural variation

• Under H0, we know the sampling distribution of the test statistic.

• Critical values: cutoff values where the tail probability under the sampling distribution counted for the type I error rate

• Reject H0 if |the test stat. value based on data| > |critical values|

0

0.05/2= 0.025

0 0

0.05/2= 0.025

Basic Principles of Hypothesis Testing §Hypothesis testing: conducted assuming H0 is true by considering

• Type I error rate (α)

• Generally test statistic = observed difference / natural variation

• Reject H0 if the p-value < α or, equivalently, |the observed test stat value| > |critical values|

§Sample size & Power computation: conducted assuming HA is true

• Hypothesize difference under HA (Effect size)

• Compute test statistic under HA = Effect size / natural variation

• We would reject H0 if |the test stat. value under HA | > |critical values|

• Power = 1 - type II error (β) = Prob. correctly rejecting H0

Basic Principles of Sample size & Power computation§Mean comparison (μ0=control group mean, μ1=trt group mean)

§Effect size = μ1-μ0 (Δ)

§Natural variation (SD / square root of sample size)

§Critical value = cutoff values for rejecting H0 (depending on α)

(α/2 ) if two-sided

1-β (power)

Δ

Basic Principles of Sample size & Power computation

(α/2 ) if two-sided

1-β (power)

Δ

Break!