THE ROLE OF SUBGROUPS IN CLINICAL TRIALS

Ralph B. D’Agostino, Sr., PhDBoston University

September 13, 2005

OUTLINE

• Clinical trial scenarios for subgroup (subset) analysis

• Illustrations of possible outcomes– GOOD and PROBLEMATIC PRACTICES

• Role of formal Interaction Test• Statistical properties of analyses• Closing Comments

POSSIBLE SCENARIOS FOR SUBSET ANALYSIS

• Primary Hypothesis is for an overall statistically significant effect (All data combined) If Yes– Subgroups examined for consistency– Special subgroups examined for additional effect– Latter two activities are secondary analyses

• Primary Hypothesis anticipates subgroup effect– Subgroups examined for equal effect to justify pooling

and an overall analysis

Primary Hypothesis is for an overall statistically significant effect

• Primary hypothesis is that two treatments are significantly different

If yes to above, concern then is that this should be consistently seen in all relevant subgroups (THIS IS A SECONDARY ANALYSIS)

OVERALL TEST IS SIGNIFICANT

GOOD: T 1 better than T 2 Events Plots over Study Time

0.0

.25

. 50

30.0

.75

1.00

0 0.5 1.0 1.5 2.0 2.5 3.0

TREATMENT 1

TREATMENT 2

-

Time from randomization in years

0.1 1 10

Overall

GenderMalesfemales

Age < 65>65

PreviousCondition

Hazard Ratio

yesno

1 better 2 better

Note: Many subgroups may not show statistical significance, but do show consistentcy

0.1 1 10

Overall

GenderMalesfemales

Age < 65>65

Years with Condition

Hazard Ratio

yesno

Location XNOYES

1 better 2 better

Special subgroup

21

Relative Risk Reduction by Qualifying Condition

IS n = 6431MI n = 6302PAD n = 6452

Total n =19185 30 20 10 0 10 20Clopidogrel Better Aspirin Better

OVERALL TEST IS NOT SIGNIFICANT

• TECHNICALLY YOU CANNOT GO BEYOND THIS WITH ANY STATISTICAL STATEMENTS

• USEFUL TO LOOK AT SUBSETS AS EXPLORATORY ANALYSIS (NOT EVEN APPROPRIATE TO CALL IT A SECONDARY ANALYSIS)

PROBLEM: T1 not better Events Plots over Study Time

0.0

.25

. 50

30.0

.75

1.00

0 0.5 1.0 1.5 2.0 2.5 3.0

TREATMENT 1

TREATMENT 2

-

Time from randomization in years

0.1 1 10

Overall

GenderMalesfemales

Age < 65>65

PreviousCondition

Hazard Ratio

yesno

1 better 2 better

Note: Only age > 65 is “significant.” Do we believe it?

0.1 1 10

Overall

GenderMalesfemales

Age < 65>65

Years with Condition

Hazard Ratio

yesno

Location XNOYES

1 better 2 better

Location is “significant”. Was it pre-specified as primary? No

Primary Hypothesis anticipates possible subgroup effect

• Subgroups identified by pre-randomization or post-randomization stratification

• Subgroups tests for equal vs. unequal effect. This is done formally by INTERACTION TEST

Procedure• If significant interaction, do not pool and test

groups• If no significant interaction pool• May need to add variable in analysis for groups

Primary Hypothesis anticipates subgroup effect (continue)

• INTERACTION TEST may be avoided and subgroups can be tests separately with control of error rates

• For example, if there are two groups then each can be tested at 0.025 level of significance

• For example, groups can be tested sequentially with error rate control

Location X: YES

0.0

.25

. 50

30.0

.75

1.00

0 0.5 1.0 1.5 2.0 2.5 3.0

TREATMENT 1

TREATMENT 2

-

Time from randomization in years

Location X: NO

0.0

.25

. 50

30.0

.75

1.00

0 0.5 1.0 1.5 2.0 2.5 3.0

TREATMENT 1

TREATMENT 2

-

Time from randomization in years

0.1 1 10

Overall

Hazard Ratio

Location X

NO

YES

1 better 2 better

Location: Interaction of location is significant

Do not pool

Statistical properties of analysesOverall test

• If Primary hypothesis of overall significance is satisfied

• Then as secondary analyses we can examine subgroups and control error rates (that is, can control chance of identifying falsely significant subgroups) for specified subgroups

• If number of subgroups is unspecified, then analysis is exploratory even here

Statistical properties of analysesOverall test (continue)

• If Primary hypothesis of overall significance is not met (that is, we do not achieve statistical significance), then we cannot control error rate (falsely identifying significant subsets). We have used “alpha” on overall test

Primary hypothesis anticipates subgroup differences

• Level of significance is chance of rejecting at least one false null hypothesis and it can be controlled.

• If there are potentially k subgroups this error rate may be as high as k(0.05) if each subgroup is tested at 0.05 level of significance.

• With k=2, 0.10; k=3, 0.15, k=5, 0.25

Closing Comments

• Error rates (level of significance) can be controlled even for looking at subgroups if structure of statistical approach is clearly stated.

• We must not confuse test for subgroups stated in a pre-specified manner from post hoc identification and tests