What is Risk Prediction?

Andrew N. Freedman, PhDClinical and Translational Epidemiology Branch

National Cancer Institute

EGRP, DCCPS, NCIJune 11, 2014

“Prediction is very difficult, especially if it’s about the future”

Niels Bohr

Outline

Prediction Models for Cancer Risk, Susceptibility and Treatment Management

Background Model Development and Evaluation Applications Challenges

Prediction Models for Cancer Risk, Susceptibility and Treatment Management

Increased attention to statistical models WebsitesHandbooks Professional societiesCommercial companies offering genetic

risk profiling

Proliferation of Personal Genomic Tests

Prediction Models for Cancer Absolute Risk Assessment Models

Estimates the probability of developing cancer over a defined period of time

Genetic Susceptibility Risk ModelsEstimates the likelihood of detecting a mutation in a

cancer susceptibility gene in a given family or individual

Cancer Outcome Risk ModelsPrognostic- estimates the likelihood of a patient

outcome, regardless of treatmentPredictive- estimates response to treatment

Absolute Breast Cancer Risk Models

NCI BCRAT “Gail” Model: (Gail et al. JNCI 1989) CASH “Claus: Model: (Claus et al. AJHG 1991) Group Health (Taplin et al. Cancer 1991) DevCan (Feuer et al. JNCI 1993) NHS (Rosner et al. JNCI 1996) BRCAPRO (Parmigiani/Berry, AJHG 1998) Jonker et al (CEBP 2003) IBIS (Tyrer/Cuzick et al. Stat Med 2004) BOADICEA (Antoniou et al, BJC 2004)

Risk models for predicting BRCA1/2 cancer susceptibility genes

Couch et al. NEJM, 1997. Shattuck-Eidens et al. JAMA, 1997. Myriad (Frank et al. JCO, 1998/2002.) BRCAPRO: Berry et al. JNCI 1997 Parmigiani, AJHG, 1998. Hartge et al. AJHG, 1999. Ontario FHAT (Gilpin et al. Clin Genet 2000) Vahteristo et al. Br J Can 2001  de la Hoya et al. Int J Cancer 2002 Jonker et al (CEBP 2003) LAMBDA (Apicella et al. Breast Can Res 2003 BOADICEA (Antoniou et al, BJC 2004) Manchester (Evans DG et al. J Med Gen 2004/2005)

Sponsored byDivision of Cancer Control and Population Sciences

Division of Cancer Epidemiology and GeneticsOffice of Women’s Health

National Cancer Institute, National Institutes of Health, Department of Health and Human Services

Cancer Risk Prediction Models: A Workshop on Development, Evaluation,

and Application

Washington, D.C. May 20-21, 2004

Num

ber o

f Gra

nts

Fiscal Year

Number of Cancer Risk Prediction Grant Applications Submitted and Awarded

DCCPS (FY05 – FY13)

Cumulative number of cancer risk and susceptibility prediction models

• Large cohort and case-control datasets, consortiums, and registries

• RCTs with biospecimens• Development of EHRs and availability of

large administrative claims databases• Evidence for effective screening,

intervention, prevention and treatment strategies

Current Opportunities in Cancer Risk Prediction

Promising new genomic and other “omic” markers

New risk prediction methodologies and evaluation techniques

Progress in research for communicating risk, decision-making and decision aids

Modeling cost-effectiveness and burden of disease

Current Opportunities in Cancer Risk Prediction

Prediction Model Development Risk Factors

• Environmental• Demographic, reproductive, smoking, medications

• Inherited• Family history• High penetrance alleles• Low penetrance polymorphisms

• Clinical • Age, sex, blood pressure, cholesterol, enzyme levels,

inflammatory markers, previous screening, treatment, etc.• Molecular/Genomic markers

• Somatic alterations, tumor subtype, protein expression, methylation, metabolomics, etc.

• Interactions

Prediction Model Development

Data• Cohort, case-control, nested case-control,

family and clinical studies, population surveys, SEER, and clinical trials

• Expert opinion

Risk Calculation• Empirical, logistic regression, proportional

hazards, Bayesian analyses, log Incidence, Markov models/decision theory, artificial neural network, classification tree, etc.

Prediction Model Evaluation Calibration

• Ability of a model to predict incidence of a disease in a group of individuals

Discriminatory Accuracy• Measures a model’s ability to discriminate at the

individual level among those who develop disease from those who do not

Clinical Usefulness• Decision-Analytic Approach, Decision Curves

Internal Validity• Data-splitting, cross validation, bootstrapping

External Validity• New independent sample

Gail Breast Cancer Risk Assessment Model (http://cancer.gov/bcrisktool)

Risk Factor Category Relative Risk of IBC in next 5 years

Age at menarche, y > 14 12-13 12

1.00 1.101.21

No. of breast biopsies Age at counseling, 50 y old

0 1 2

1.001.702.88

Age at counseling, 50 y old 0 1 2

1.001.271.62

Age at first live birthNumber of first-degree relatives with breast cancer

< 20 years 0 1 2

1.002.616.80

20-24 years 0 1 2

1.242.685.78

25-29 years or nulliparous 0 1 2

1.552.764.91

30 0 1 2

1.932.834.17

Baseline 5-year risk of invasive breast cancer in percentages, by age and race

Baseline 5-year risk, %

Age in years Black White Hispanic

20-24 0.003 0.003 0.006

25-29 0.025 0.022 0.021

30-34 0.076 0.077 0.057

35-39 0.165 0.191 0.126

40-44 0.285 0.366 0.235

45-49 0.343 0.540 0.378

50-54 0.376 0.640 0.456

55-59 0.474 0.788 0.537

60-64 0.581 0.969 0.623

65-69 0.592 1.135 0.727

70-74 0.656 1.209 0.824

75-79 0.761 1.285 0.798

80-84 0.876 1.280 0.730

Example: Breast Cancer Risk Projection Using the Gail Model A 42-year-old white women

Began menstruating at age 12 years, RR=1.10

No children and no affected first-degree relatives, RR=1.55

One previous benign breast biopsy, RR=1.70

Overall RR = 1.10 X 1.55 X 1.70= 2.90 Projected 5-year risk of invasive breast

cancer = 2.90 X 0.366 = 1.06%

Prediction Model Applications

Planning intervention trials Estimating the population burden of

disease Identifying individuals at high risk

Designing prevention strategies

Clinical decision-making

Prediction Model Applications

Planning intervention trials Estimating the population burden of

disease Identifying individuals at high risk

Designing prevention strategies

Clinical decision-making

Determining Trial Eligibility For Breast Cancer Chemoprevention Trials

age 35 years or older, and

a 5-year risk of invasive breast cancer of at least 1.67%

based on the Gail Breast Cancer Risk Assessment Model.

1Five-year projected risk of invasive breast cancer IBC is greater than or equal to 1.67%.

Estimates of the total number of U.S. women eligible for tamoxifen chemoprevention Trial, by race and age

0

10

20

30

40

50

60

White

Black

Hispanic

35-39 40-49 50-59 60-69 70-79

Age

Pe

rce

nt

Prediction Model Applications

Planning intervention trials Estimating the population burden of

disease Identifying individuals at high risk

Designing prevention strategies

Clinical decision-making

Graubard et al. CEBP 2010;19:2430-6

Tamoxifen Chemoprevention Eligibility and Positive Benefit/risk Index

0

10

20

30

40

50

60

35-39 40-49 50-59 60-69 70-78

% white womeneligible for tamoxifen

% white women with apositive benefit/riskindex for tamoxifen

AgeAge

Pe

rce

nt

Freedman et al. JNCI 2003;95:526-32

2.4 million women who could benefit from tamoxifen

Prediction Model Applications

Planning intervention trials Estimating the population burden of

disease Identifying individuals at high risk

Designing prevention strategies

Clinical decision-making

National Comprehensive Cancer Network (NCCN)Guidelines for Breast Screening

Women > 35 y with 5-year risk for IBC > 1.7%Annual mammogram+ clinical breast exam

every 6-12 mConsider risk reduction strategies

Women who have a lifetime risk >20% as defined by models that are largely dependent on family historyAnnual mammogram+ clinical breast exam

every 6-12 mConsider risk reduction strategiesConsider annual breast MRI

Bevers et al. JNCCN 2009;7:1060-96.

Distribution of risk by allele number.

Dunlop M G et al. Gut 2013;62:871-881

Copyright © BMJ Publishing Group Ltd & British Society of Gastroenterology. All rights reserved.

Prediction Model Applications

Planning intervention trials Estimating the population burden of

disease Identifying individuals at high risk

Designing prevention strategies

Clinical decision-making

Tamoxifen Effects on Events

LIFE-THREATENING RR (95%CI) INVASIVE BREAST CA 0.51 (.39-.66) HIP FRACTURE 0.55 (.25-1.1) ENDOMETRIAL CA

<50 2.5 (1.4-5.0)50+ 4.0 (1.7-11)

STROKE 1.6 (0.9-2.8) PUL. EMBOLUS 3.0 (1.2-9.3)

OTHER SEVERE EVENTS IN SITU BREAST CA 0.50 (0.33-0.77) DEEP VEIN THROMBOSIS 1.60 (0.91-2.86)

Calculating Tamoxifen Benefit/Risk Index

Net number of life-threatening events prevented

(the total number of invasive breast cancers + hip fractures -

the total number of endometrial cancers, strokes, and pulmonary embolisms)

+ half the net number of serious events prevented (the number of in situ breast cancers - the number of deep vein

thromboses)

over a 5-year period.

Gail et al. JNCI 1999;91:1829-46

10,000 40 year old white women with a uterus, with a 5-year risk of invasive breast cancer of 2%.

LIFE-THREATENING EXPECTED PREVENTED INVASIVE BREAST CA 200 97 HIP FRACTURE 2 1 ENDOMETRIAL CA 10 -16 STROKE 22 - 13 PUL. EMBOLUS 7 -15

net 54OTHER SEVERE EVENTS IN SITU BREAST CA 106 53 DEEP VEIN THROMBOSIS 24 -15 net 38

net 38 Net Risk Benefit Index = 1x54 + 0.5x38 = 73

1Five-year projected risk of invasive breast cancer IBC is greater than or equal to 1.67%.

Benefit/risk indices for tamoxifen chemoprevention by level of 5-year projected risk of invasive breast cancer among white women with and without a uterus, by age group

70-79 60-69 50-59 40-49 30-39

White Women With A Uterus White Women Without A Uterus

7.0

6.5

6.0

5.5

5.0

4.5

4.0

3.5

3.0

2.5

2.0

1.5

70-79 60-69 50-59 40-49 30-39 5-y projected risk of IBC, %

Benefits outweigh risks

Risks outweigh benefits

Benefit/risk indices for tamoxifen and raloxifene chemoprevention by level of 5-year projected risk for invasive breast cancer (IBC) for white non-Hispanic women with a uterus, by age group.

Update with raloxifene b/r slide

Freedman et al. JCO 2011;29:2327-33.

Paik, S. et al. J Clin Oncol; 24:3726-3734 2006

Kaplan-Meier plots for distant recurrence comparing treatment with tamoxifen (Tam) alone versus treatment with tamoxifen plus

chemotherapy (Tam + chemo)

Oncotype Dx: Computation of the Recurrence ScoreTM

• Determine the expression of 21 individual genes• Multiply the individual expression profile by a coefficient contained in an algorithm• Calculate the final Recurrence Score

Breast Oncotype Dx- Patient Report

The patient report includesRecurrence Score (RS)Average 10-year distant

recurrence rate for that RSGraph of 10-year recurrence

risk as a function of RS in tamoxifen-treated patients

The report is sent toTreating physicianSubmitting pathologist

Advantages of clinical prediction models compared with simple risk classification in oncology practice

Risk Prediction Models (RPM) can improve predictive accuracy Can incorporate multiple variables

RPM allow incorporation of novel predictors Multiple biomarkers

RPM can aid in the choice of cut-points for decision-making Allows examination of risk/recurrence as continuous

RPM can aid in individualized decision-making

Vickers AJ, CA Cancer J Clin. 2011

Challenges

“Too many models, not enough independent validation” (Vickers AJ, CA Cancer J Clin. 2011)

oWhat current models require validation?

oWhat quantitative criteria should be used to assess the performance of risk models for various purposes?

Challenges

Clinical Usefulness

o Can we improve the accuracy of cancer screening risk prediction models at the individual level with the addition of risk/genetic factors?

o What level of accuracy is needed to be useful in clinical decision-making and what are the best ways to measure it?

Challenges

Poor integration into clinical practice

o How should cancer risk prediction models be disseminated to health care providers (e.g. EHRs), patients, and the public?

o How do we best convey risk and uncertainty to health care providers, patients, and the public?

o How can they be used effectively to improve cancer education, risk communication and the development of decision aids?

Graphical Representations of Risk

National Breast Cancer Centrehttp://www.nbcc.org.au/risk/understandingrisk.html

Harvard Universityhttp://www.diseaseriskindex.harvard.edu/update/english

National Cancer Institute

Website Resources

NCI’s Cancer Risk Prediction Resources http://epi.grants.cancer.gov/cancer_risk_prediction/

Bibliography of Risk Prediction ModelsFunding OpportunitiesWorkshop and WebinarsFunded Projects

Cancer Prognostic Resources: A catalog of Interactive Cancer Prognostic Toolshttp://cancercalculators.org/index.aspx

Cancer Intervention and Surveillance Modeling Networkhttp://cisnet.cancer.gov/

END

Ezaz G et al. J Am Heart Assoc 2014 Feb 28;3(1):e000472. doi: 10.1161/JAHA.113.000472.

Risk prediction model for heart failure and cardiomyopathy after adjuvant trastuzumab therapy for breast cancer

Too many models, not enough independent validation

Obtain data to develop and validate more accurate and useful risk models

Extend existing models with data sources that include diverse racial and ethnic groups

Develop risk models with cancer treatment outcomes and non-cancer outcomes

Future Research

Clinical and Public Health Usefulness

Develop risk models that can help identify which population subgroups would benefit (or not benefit) from specific screening and risk reducing interventions

Develop risk models that can help identify which patient subgroups would benefit (or not benefit) from specific cancer therapies weighing both treatment response and toxicity

Future Research

Poor integration into clinical practice

Promote effective cancer risk communication and decision making

Incorporate patient preferences into models for use in clinical decision-making

Create simple, user-friendly models for healthcare providers to facilitate decision-making and referrals

Future Research