outcome measures & statistical analyses · outcome measures & statistical analyses . june...

ICTR The Johns Hopkins Institute for Clinical and Translational Research

Preparing for ClinicalTrials.gov Results Reporting and Module 3:

Outcome Measures & Statistical Analyses June 22, 2014 ICTR Research Navigators

ICTR Module 3

Outcome Measures & Statistical Analyses

ICTR Module 3: Outcome Measures & Statistical Analyses

FDAAA*: Outcome Measures

“…a table of values for each of the primary and secondary outcome measures for each arm of the clinical trial…including the results of scientifically appropriate tests of the statistical significance of such outcome measures.” [Sec. 282(j)(3)(C)(ii)]

*Food and Drug Administration Amendments Act of 2007


Outcome Measures Purpose

The Outcome Measures module displays the results and associated statistical analyses for each pre-specified primary and secondary outcome measure. Other outcome measures may also be included.

See ClinicalTrials.gov Outcome Measures and Statistical Analyses Module Presentation, Deborah A. Zarin, M.D. at http://prsinfo.clinicaltrials.gov/webinars/module7/index.html

http://prsinfo.clinicaltrials.gov/webinars/module7/index.html


Outcome Measures – Data Elements

• To set up table – Arm/Group title* and Description – Number of Participants Analyzed*

• To Describe specific Outcome Measures – Outcome Measure Title* – Outcome Measure Description – Unit of Measure* – Outcome Measure Time Frame* – Measure Type (e.g., mean, median)* – Measure of Dispersion/Precision (e.g., Standard Deviation)*

• Data*

*Items with a red star are required by ClinicalTrials.gov.


Outcome Measures – Data Elements • Outcome Measure Type*

– Options Primary Secondary Other Pre-specified Post-hoc

• Outcome Measure Reporting Status* • Outcome Measure Safety Issue? (Y/N)

*Required by ClinicalTrials.gov http://prsinfo.clinicaltrails.gov/results_definitions.html

Primary or secondary outcome measures are copied from protocol section. If you need to add other outcome measures, you can add them here. Note: outcome measures must be specific, measurable, and have a specific time frame in which they are measured. The following slides will cover this topic in greater detail. Status: indicates whether results have been posted on the pubic ClinicalTrials.gov webpage yet.

Remember that each record is required to have “posted” data for at least one outcome measure. (Also remember posted results has its own tab on the public ClinicalTrials.gov webpage) Here is the excerpt from FDAAA on which outcomes measures are required: http://prsinfo.clinicaltrials.gov/FDAAA801BasicResults.pdf

http://prsinfo.clinicaltrails.gov/results_definitions.html

http://prsinfo.clinicaltrails.gov/results_definitions.html



• The previous slide is a framework for thinking about specificity of the outcome measure described.

• Along the left, you can see that for any level of specificity, a specific time frame is necessary.

• Then going from top to bottom, you can see increasing levels of specificity. The highest level is the domain, and in this case we’re going to report a measure about anxiety. That is not very specific though.

• The next measure is level two and is designed to say what specifically was used to measure anxiety and in

this case the Hamilton Anxiety Rating Scale was used.

• The third is the specific metric, which is what was measured in the individual participants in each arm of the trial. For this example, we are going to look at change from baseline.

• Finally, the question is how are the data across participants in each arm aggregated? In this case it was a categorical measure, and the category chosen was the proportion of subjects in each arm with a decrease of greater than or equal to 50 percent. As you move down the diagram, you are getting more and more specific. It is extremely important that when registering the outcome measures and reporting the results that the information be entered with as much specificity as possible.


Outcome Measure Information Please be specific as possible.

• Title: include the name of the specific measure. Avoid using verbs, that is, do not put “To determine…”

• Time Frame: must have a time point at which the outcome is assessed for the specific metric used (hours, days, weeks, years) Hint: specify which study day it is measured - do not use “until the end of study or death”.

• Description: describes what will be measured, not why it is measured. If the outcome measure is a questionnaire or scale, provide the range and what low or high scores mean.

• Safety Issue: Is this outcome measure assessing a safety issue?


Outcome Measure Information

The former slide shows the data fields that must be entered for an outcome measure. If the outcome measure in the protocol is framed as a hypothesis, including a verb, re-write the outcome measure into a specific measure. (Hint: Look in the statistical section of the protocol to find more specific outcome measures.)

If the Outcome measure is using a scale, the name of the scale should be provided in the Outcome Measure Title. The range and direction of scores (0=worst, 10=best) are provided in the Outcome Measure Description. The Unit of Measure is “units on a scale,” if no other unit was used.

If the Outcome Measure expresses a change, specify two time points in the Time Frame (e.g., week 0 and week 52).


Example of an Outcome Measure What is wrong with this?

Eosinophilic Esophagitis Study Pathology At the end of study

Number of Patients with Complete Histologic Response

Baseline to 6 weeks

The top example is too vague and not specific enough. The bottom example is a better example of how an outcome measure should be expressed. Before this Outcome Measure is complete, you should describe the Dysphagia Questionnaire in Description box, that is, the range and direction of scores.

The criteria used to define “complete response to dysphagia” should be described in the Outcome Measure Description (this may be the specific score achieved on the dysphagia questionnaire).


Get Organized Using the Simple Form

Use one simple form for each Outcome Measure you are entering into the database.


Get Organized Using the Simple Form

• As you start to prepare to enter your data in the form on the previous slide, you may find the Outcome Measure Template helpful as a worksheet to plan data entry.

• This template and others are available at http://prsinfo.clinicaltrials.gov/results_table_layout/ResultSimpleForms.html.

• Where to get information for the module: Protocol (study design, methods, statistical section); manuscript or published journal article (abstract, methods, results [text and tables], statistics section). Note: sometimes p values can be in figures or in figure legends. If a manuscript or article has not been written yet, you might have to work from data tables from the statistician.

http://prsinfo.clinicaltrials.gov/results_table_layout/ResultSimpleForms.html

http://prsinfo.clinicaltrials.gov/results_table_layout/ResultSimpleForms.html


Outcome Measures – Journal Article Format

This is an example of how outcome measures are displayed in journal articles.

This data cannot be entered into ClinicalTrials as one outcome measure, it would require several outcome measures to capture all of the data.

(Note: ClinicalTrials.gov cannot display graphs such as Kaplan-Meier curves, or box and whisker plots. You must obtain the numeric data to enter the outcome measure.)

Pappas PG, Chetchotisakd P, Larsen RA et al. Clin Infect Dis. 2009

© 2009 Infectious Diseases Society of America


Outcome Measures – ClinicalTrials.gov Format, Public View

NCT00145249

This is an example of how a completed outcome measure would look (public view on www.clinicaltrials.gov), as compared to a journal article presentation.


Outcome Measures – ClinicalTrials.gov Format, Public View

This is an continued example of how a completed outcome measure would look, note that the data are presented in a table. It is not possible at this time to present data as graphs, box and whisker plots, etc.

NCT00145249


Outcome Measures – Data Elements – Statistical Analyses*

*The law requires scientifically appropriate tests of statistical significance. Basic Results Data Elements Definitions: http://prsinfo.clinicaltrials.gov/results_definitions.html

http://prsinfo.clinicaltrials.gov/results_definitions.html


Outcome Measures – Data Elements – Statistical Analyses*

• Although the data element of statistical analyses is “optional”, depending on the study protocol and statistical analysis plan it may be “required”.

• Ideally, every protocol should have a statistical analysis plan (SAP). Therefore,

what analyses were planned should be reported in ClinicalTrials.gov. However, not all fields will be required; this is dependent on the statistical test involved. To determine what statistical tests were performed, review the statistical methods section in the protocol, the SAP, and the journal article, if available, or speak to the study statistician. P-values and other statistical tests can be found in the results tables in the journal article.

• For non-statisticians, a good reference book is: Lang, Thomas A, and Secic, Michelle. How to Report Statististics in Medicine: Annotated Guidelines for Authors, Editors, and Reviewers. Philadelphia: American College of Physicians, 2006.


Go to Outcome Overview to Add Statistical Analysis

To enter the statistical analysis, return to outcome measure screen view, and click on “Add Statistical Analysis.”

To enter the statistical analysis, return to outcome measure screen view, and click on “Add Statistical Analysis.”


Add Outcome Statistical Analysis

Generally you must check at least 2 groups, or you can check all groups if appropriate (omnibus analysis). If you get error messages, enter the required data and then save (“OK” button at bottom of screen) and the error messages will disappear.


Add Outcome Statistical Analysis

Choose the statistical test from the drop-down box.

In the next set of boxes, complete the fields appropriate for the statistical test. (Note: If only a p-value is being entered as shown in this example, you only need to enter the p-value and the statistical test of hypothesis. You do not need to enter an estimated value for p-values, this is for other statistical tests.) It is ok to enter a non-statistically significant p-value. If a statistician is available, get one to help you.


Completed Outcome Measure and Statistical Analysis

Choose the statistical test from the drop-down box.

This slide shows a completed outcome measure and the associated statistical analysis.


Outcome Measure and Statistical Analysis– Quick Tips

• Results, including study or outcome conclusions, are not provided in narrative form.

• The Title and Description are comprehensible and descriptive (e.g., Arm A, B, C is not informative).

• For measures obtained using a scale:

• Name of scale is provided in “Measure Title.” • Range and direction of scores (0 = worst; 10 = best) are

indicated in “Measure Description.” • “Unit of Measure” is “units on a scale” if no other unit.

See http://prsinfo.clinicaltrials.gov/ResultsDetailedReviewItems.pdf

http://prsinfo.clinicaltrials.gov/ResultsDetailedReviewItems.pdf

Where science and people connect.

ICTR

24

Module 3: Outcome Measure and Statistical Analysis

This slide set was made possible by a collaboration of CTSA organizations (Mayo Clinic, Partners, University of Michigan Medical School, Duke University) and the National Library of Medicine. The Clinical and Translational Science Awards Program (CTSA) is part of the Roadmap Initiative, Re-Engineering the Clinical Research Enterprise and is funded by the National Center for Advancing Translational Sciences (NCATS), National Institutes of Health (NIH).

outcome measures & statistical analyses · outcome measures & statistical analyses . june...

Documents