study design: the blueprint for success tharyan.pdf · prathap tharyan md, mrcpsych. professor of...
TRANSCRIPT
IASCT: Clinical Trial Interfaces 2011; Mumbai
Study Design: The Blueprint for success
Prathap Tharyan MD, MRCPsych
Professor of Psychiatry & Associate Director, CMC Vellore
Director, South Asian Cochrane Network & Centre
Prof BV Moses & ICMR Centre for Advanced Research & Training in
Evidence‐Informed Healthcare
Christian Medical College, Vellore
IASCT Clinical Trial Interfaces 2011; Mumbai
Don’t Just Do It
Do It RightBecause it matters……
If you want to conduct research……
IASCT Clinical Trial Interfaces 2011; Mumbai
Who needs reliable evidence?
•
Health Professionals at all levels, Health Researchers,
Health Policy Makers, Care Givers and Patients.
EVERYBODY
IASCT Clinical Trial Interfaces 2011; Mumbai
THIS TALK IS DEDICATED TO ALL OUR MOTHERS
IASCT Clinical Trial Interfaces 2011; Mumbai
WHAT ARE WE TRYING TO AVOID?
IASCT Clinical Trial Interfaces 2011; Mumbai
Getting the evidence right: What Is The Truth Here? Look out for my new book, “7 habits of highly successful
and popular people who are
also sensitive boyfriends. ”Biased publications: Cost?
IASCT Clinical Trial Interfaces 2011; Mumbai
He doesn’t really know what I
want….All he thinks of is his book..His friend is so much more
sensitive to my needs
Confound it!What will they both
do when they find
out I’m gay?
IASCT Clinical Trial Interfaces 2011; Mumbai
Getting the evidence right: the effects of chance
What are the chances
of this happening?Is this a coincidence?Does it happen
everyday?Or is this an Al Qaeda
plot?
Surfers rule
Dolphins rule
IASCT Clinical Trial Interfaces 2011; Mumbai
Threats to Internal Validity
•Any
factor
or
process
that
tends
to
deviate
the
results
or
conclusions of a trial systematically away from the truth•Deviation
in
results
can
occur
due
to
systematic
(bias)
or
random errors (chance)•Random errors reduce with increase in sample size; detected
by p values•Bias
can
result
in
overestimates
or
underestimates
of
the
results of a trial (cannot be detected by p values)•Bias
can
occur
due
to
voluntary
or
involuntary
reasons
(not
the same as fraud)
IASCT Clinical Trial Interfaces 2011; Mumbai
The Gold Standard for assessing
the evidence of the efficacy of interventions
Most prone to bias
A study design that minimizes bias, confounding and the play of
chance
Least biased
IASCT Clinical Trial Interfaces 2011; MumbaiPLOS Medicine August 2005
“In a chase for statistical significance”
IASCT Clinical Trial Interfaces 2011; Mumbai
The use of
many small randomized trials with
clinically non‐relevant outcomes,
improper interpretation of statistical significance,
manipulated study design,
biased selection of study populations,
short follow‐up, and
selective and distorted reporting of results
has built and nourished a seemingly evidence‐based myth of
antidepressant effectiveness
Philosophy, Ethics, and Humanities in Medicine 2008, 3:14 doi:10.1186/1747‐5341‐3‐14
IASCT Clinical Trial Interfaces 2011; Mumbai
Getting the evidence straight requires a special effort
Needs a systematic review“A review in which bias has been reduced by the systematic
identification,
critical
appraisal,
and,
if
relevant,
statistical
aggregation
(meta‐analysis)
of
all
relevant
studies
on
a
specific
topic
according
to
a
predetermined
and explicit method.”
Modified from Moher
et al Lancet 1999
IASCT Clinical Trial Interfaces 2011; Mumbai
Getting the evidence straight
What is needed?
•
Ensure all relevant research is considered
•
Critically appraise research for quality (risk of bias) and applicability
•
Synthesize research findings using appropriate methods
IASCT Clinical Trial Interfaces 2011; Mumbai
IASCT Clinical Trial Interfaces 2011; Mumbai
GRADE separates
The overall quality of the evidence
and confidence in the resultStrength of the recommendations
•
Strong recommendations:
most informed patients would
choose the recommended
management
•
Weak recommendations:
patients’
choices will vary
according to their values and
preferences
Grading of Recommendations: Assessment, Development and Evaluation
http://www.gradeworkinggroup.org
IASCT Clinical Trial Interfaces 2011; Mumbai
levels of ‘Quality’ (or confidence in the effect estimate)
•
4 = High
•
3 = Moderate
•
2 = Low
•
1 = Very low
Further research is very unlikely to change our confidence in the estimate of effect
Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate
Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate
We are very uncertain about the estimate
IASCT Clinical Trial Interfaces 2011; Mumbai
Grading of Recommendations Assessment,
Development and Evaluation
5 factors to consider• Study Limitations • Indirectness • Inconsistency • Imprecision • Other
Were the studies free of the risk of bias?Where; in whom; and how were the trials done?Is there unexplained heterogeneity?Is the result statistically and clinically important?Publication Bias? Conflicts of interest? Design
specific issues: cross-over trials etc
Grading of Recommendations Assessment,
Development and Evaluation
IASCT Clinical Trial Interfaces 2011; Mumbai
5 criteria• Limitations (of study design) No serious limitations• Inconsistency (between results) Serious inconsistency (-1)• Indirectness (of evidence) Very serious indirectness (-2)• Imprecision (of measure of effect)• Other (publication bias)
1 Downgraded for serious inconsistency because……….2 Downgraded for very serious indirectness because…….
Quality of Evidence = VERY LOW
IASCT Clinical Trial Interfaces 2011; Mumbai
IASCT Clinical Trial Interfaces 2011; Mumbai
The GRADE approach for guideline panels
• Decide on the overall quality of evidence across
outcomes.
• Include judgments about the underlying values and
preferences of management options & outcomes.
• Decide on the balance of desirable and undesirable
effects.
• Decide on the balance of net benefits and cost.
• Grade the strength of recommendation.
• Formulate a recommendation.
• Implement and evaluate.
IASCT Clinical Trial Interfaces 2011; Mumbai
The single best source of reliable and timely evidence
www.thecochranelibrary.com
Free to all in India thanks to ICMR
IASCT Clinical Trial Interfaces 2011; Mumbai
www.cochrane.org
•Reliable•Independent•Timely
IASCT Clinical Trial Interfaces 2011; Mumbai
IASCT Clinical Trial Interfaces 2011; Mumbai
EVIDENCE BASED MEDICINE
•
The integration of research
evidence, clinical expertise
and patient values
(Sackett
et al, 2000)
Evidence‐Based Medicine Working Group: Evidence‐based medicine. A new
approach to teaching the practice of medicine. JAMA. 1992;268: 2420‐5
http://www.cebm.utoronto.ca/
Is the Evidence Valid?
IASCT Clinical Trial Interfaces 2011; Mumbai
Addressing the gaps between evidence and practice
Practice
Research
Evidenc
e
Getting the evidence straight Getting the evidence used
Glasziou
P, Haynes B. ACP J Club. 2005
“Seeing through a dark glass dimly”
IASCT Clinical Trial Interfaces 2011; Mumbai
Science and ethics are inseparable in research
•
Research
that
does
not
yield
valid
results
is unethical
•
Research, even if valid, that
does not
follow ethical norms is in‐excusable
•
Invalid and unethical research cause harms:•
from wrong evaluations or estimations;
•
ineffective or harmful treatments;
•
denial of effective interventions and
•
betrayal
of
the
trust of research
participants
and
users
of healthcare
IASCT Clinical Trial Interfaces 2011; Mumbai
The current status of medical research
•
Not all research is meant to improve the lives of patients
•
Not all research is conducted using methods that eliminate
bias and the effects of confounders and hence provide
reliable results
•
Not all research is actually needed
•
Not all research that reports favourable
results actually has
favourable
results
•
Not all reported research reflects all research that has
been conducted
•
Not all clinical guideline developers understand or use
evidence
•
Most clinicians believe what they read
IASCT Clinical Trial Interfaces 2011; Mumbai
Science & Ethics as it is today
Research SocietyEthical
oversight
IASCT Clinical Trial Interfaces 2011; Mumbai
Science & Ethics as it should be
ResponsibleResearch
Responsive Society
Ethical oversight
IASCT Clinical Trial Interfaces 2011; Mumbai
What contributes to ethical and valid research?
IASCT Clinical Trial Interfaces 2011; Mumbai
AREAS OF CONCERN WITH CLINICAL TRIALS
Industry Trials
Investigator initiated trials
IASCT Clinical Trial Interfaces 2011; Mumbai
Industry trials
•
Social relevance vs. stock market relevance•
Patent expiring soon‐
replace with “me too”
drug
•
Disease mongering‐
to create a market
•
Advancing science (EBM) vs. advancing profits (MBM)•
Selection of participants
•
Choice of comparator (no equipoise)
•
Choice of outcomes
•
Statistical vs. clinical significance
•
Duration of follow up
•
Surrogate outcomes/ composite outcomes
•
Multiple outcomes/subgroups
•
Selective reporting
•
Studies
•
Outcomes/data from outcomes
•
Interpreting results
IASCT Clinical Trial Interfaces 2011; Mumbai
Investigator‐
initiated trials
•
Social relevance vs. academic relevance
•
No protocol
•
Inadequate knowledge of research methods
•
No sample size calculation
•
Quality of data collection
•
Improper study designs
•
Improper analyses‐
over‐reliance on p values
•
Not publishing (after presenting in a conference/ completing a thesis)
•
Inadequate reporting of data and outcomes
•
Selective reporting of outcomes and data
IASCT Clinical Trial Interfaces 2011; Mumbai
THE EVIDENCE FOR THESE ASSERTIONS
IASCT Clinical Trial Interfaces 2011; Mumbai
Months0 12 24 36 48 60 72 84 96 108 120
Prop
ortio
n pu
blis
hed
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
month 0 12 24 36 48 60 72 84 96 108 120# published 57 354 865 248 193 74 27 20 27 2 2# remaining 3,138 2,972 2,572 1,744 1,548 1,133 1,006 734 674 100 98
Cumulative Time from Presentation to Full Publication of RCTs
Source: Scherer et al, 2005. Systematic review of 79 reports. Cochrane Library
IASCT Clinical Trial Interfaces 2011; Mumbai
Research funded by drug companies was less likely to be
published than research funded by other sources.
Studies sponsored by pharmaceutical companies were more
likely to have outcomes favouring the sponsor than were
studies with other sponsors (odds ratio 4.05; 95% confidence
interval 2.98 to 5.51; 18 comparisons).
BMJ, 2003
IASCT Clinical Trial Interfaces 2011; Mumbai
Selective outcome reportingComparison of 102 trial protocols submitted in 1994‐1995 to and
approved by the scientific‐ethics committees for Copenhagen and
Frederiksberg Denmark, with 122 subsequent publications
Chan AW et al. JAMA 2004
Findings•
Nearly two‐thirds of the studies had a change in at least one
primary outcome between the protocol and publication•
Statistically significant outcomes had a higher likelihood of
being reported compared to non‐significant findings
IASCT Clinical Trial Interfaces 2011; Mumbai
Suppression of unfavourable
results
IASCT Clinical Trial Interfaces 2011; Mumbai
A systematic review and meta‐analysis that avoids publication bias
IASCT Clinical Trial Interfaces 2011; Mumbai
The Emperor’s New Drugs: Kirsch et al 2002
•
Analysis of the efficacy data from 47 RCTs
submitted to the U.S. FDA for approval of the 6 most widely
prescribed antidepressants approved between 1987 and 1999 (fluoxetine, paroxetine, sertraline, nefazodone,
venlafaxine, citalopram)
•
Approximately 80% of the response to medication was duplicated in placebo control groups, and the mean difference between drug and placebo was
approximately 2 points on the 17‐item (50‐pt) and 21‐ item (62‐pt) HAM D Scale.
•
Improvement at the highest doses of medication was not different from improvement at the lowest doses.
IASCT Clinical Trial Interfaces 2011; Mumbai
PLoS
Med 2008; 5(2): e45. doi:10.1371/journal. pmed.0050045
•A previously published meta‐analysis (Kirsch et al, 2002) of the
published and unpublished trials on SSRIs
submitted to the FDA during
licensing has indicated that these drugs have only a marginal clinical
benefit.
•On average, the SSRIs
improved the Hamilton Rating Scale for
Depression score of patients by 1.8 points more than the placebo,
whereas NICE has defined a significant clinical benefit for
antidepressants as a drug–placebo difference in the improvement of
the HRSD score of 3 points.
IASCT Clinical Trial Interfaces 2011; MumbaiPLoS
Med 2008; 5(2): e45. doi:10.1371/journal.
•However, average improvement scores may obscure beneficial effects
between different groups of patient, so in the meta‐analysis in this
paper, the researchers investigated whether the baseline severity of
depression affects antidepressant efficacy.
IASCT Clinical Trial Interfaces 2011; Mumbai
•Data on all the clinical trials submitted to the FDA for the licensing of
fluoxetine, venlafaxine, nefazodone, and paroxetine.
• Meta‐analytic techniques to investigate whether the initial severity of
depression affected the HRSD improvement scores for the drug and
placebo groups in these trials.
•Confirmed first that the overall effect of these new generation of
antidepressants was below the recommended criteria for clinical
significance.
IASCT Clinical Trial Interfaces 2011; Mumbai
•Virtually no difference in the improvement scores for drug and placebo
in patients with moderate depression and only a small and clinically
insignificant difference among patients with very severe depression•The difference in improvement between the antidepressant and
placebo reached clinical significance, however, in patients with
initial
HRSD scores of more than 28—that is, in the most severely depressed
patients. •Additional analyses indicated that the apparent clinical effectiveness of
the antidepressants among these most severely depressed patients
reflected a decreased responsiveness to placebo rather than an
increased responsiveness to antidepressants.
IASCT Clinical Trial Interfaces 2011; Mumbai
• Conclusions: There is little reason to prescribe new‐generation antidepressant medications to any but the most severely depressed
patients unless alternative treatments have been ineffective.
IASCT Clinical Trial Interfaces 2011; Mumbai
OUCH! THAT HURTSWhat do we say to that?
IASCT Clinical Trial Interfaces 2011; Mumbai
•
Among
74
FDA‐registered
studies,
31%,
accounting
for
3449
study participants, were not published.
•
Whether
and
how
the
studies
were
published
were
associated
with the study outcome.
•
A
total
of
37
studies
viewed
by
the
FDA
as
having
positive
results
were
published;
One
study
viewed
as
positive
was
not
published.
•
Studies
viewed
by
the
FDA
as
having
negative
or
questionable
results
were,
with
3
exceptions,
either
not
published
(22
studies)
or
published
in
a
way
that,
in
the
authors
opinion,
conveyed a positive outcome (11 studies).
IASCT Clinical Trial Interfaces 2011; Mumbai
•
According to
the
published
literature,
it
appeared that 94% of the trials conducted were positive.
•
By
contrast,
the
FDA
analysis
showed
that
51% were positive.
•
Separate meta‐analyses of the
FDA
and
journal data sets showed that the increase in effect size ranged from 11 to 69% for individual drugs and was 32% overall.
IASCT Clinical Trial Interfaces 2011; Mumbai
•
The drugs were bupropion
SR, citalopram, duloxetine, escitalopram, fluoxetine, mirtazapine, nefazodone,
paroxetine, sertraline, venlafaxine
and venlafaxine
XR
IASCT Clinical Trial Interfaces 2011; Mumbai
One question
•
Is a difference in 2‐3 points on the HDRS a clinically important outcome for
people with depression?
IASCT Clinical Trial Interfaces 2011; Mumbai
UNREAL TRIALS FOR THE REAL WORLD
Exclusion criteria in clinical trials
IASCT Clinical Trial Interfaces 2011; Mumbai
A Reevaluation of the Exclusion Criteria Used
in Antidepressant Efficacy Trials
Posternak
et al, Am J Psychiatry 2002; 159:191–200)
IASCT Clinical Trial Interfaces 2011; Mumbai
WHY THE DIFFERENCE BETWEEN WHAT WE BELIEVE AND WHAT WE
NOW READ?
What is the truth?
Is meta‐analysis rubbish or there more to it?
IASCT Clinical Trial Interfaces 2011; Mumbai
What is needed are higher evidence standards
While placebo controls are needed due to the high placebo
response rates in depression, comparisons with active drugs
mostly show no differences for any class of antidepressant
Use of clinically relevant outcomes ( clinical improvement,
employment, QOL, etc) rather than rating scales.
Pragmatic trials with fewer exclusion criteria
Prospective registration of trials and making data sets of results
available
large long‐term trials and
careful prospective meta‐analyses of individual‐level data
Philosophy, Ethics, and Humanities in Medicine 2008, 3:14 doi:10.1186/1747‐5341‐3‐14
IASCT Clinical Trial Interfaces 2011; Mumbai
Outcome measures used in psychiatric trials
IASCT Clinical Trial Interfaces 2011; Mumbai
Quality of Schizophrenia trials
•
Overall, the quality of reporting was poor•
Score of 3 or > was predefined as better
quality ‐
33% of North American RCTs
‐ 36% of European RCTs
‐ 43% of ROW RCTs
(46% from Canada, Middle East & Asia) Thornley
& Adams. BMJ 1998
IASCT Clinical Trial Interfaces 2011; Mumbai
Quality in Schizophrenia RCTs
•
Average number of sample = 65 with no change over time
•
For an outcome to show clinically significant improvement of 20% a study would need 150
participants in each arm •
Only 3% of RCTs
were of this size or greater
Thornley
& Adams. BMJ 1998
IASCT Clinical Trial Interfaces 2011; Mumbai
Sample size in 2000 trials Thornley
1998
IASCT Clinical Trial Interfaces 2011; Mumbai
Rating scales in Mental Health
No other branch of the health sciences uses as many rating
scales as does mental health
Hardly any clinicians use rating scales in clinical practice to
determine outcomes
Continuous outcomes (Mean & SD) from rating scales are
assessed by significance tests and generate p values
Many researchers pay obeisance to the almighty p value
The almighty p value can be very deceptive as statistically
significant results from rating scales may prove the effects of
interventions are not due to chance but do not indicate how
effective the interventions are likely to be (statistical versus
clinical significance)
IASCT Clinical Trial Interfaces 2011; Mumbai
What do changes in rating scales mean clinically?
Clinicians tend to assess:severity as mild /moderate /severe andimprovement as worse, no change, mildly better,
moderately better, much better, completely well
Similar to the Clinical Global Impressions‐
Severity &Improvement (CGI)
IASCT Clinical Trial Interfaces 2011; Mumbai
What do changes in rating scales mean clinically?
IASCT Clinical Trial Interfaces 2011; Mumbai
What do changes in rating scales mean clinically?
IASCT Clinical Trial Interfaces 2011; Mumbai
Why not use better anchored versions of the CGI for research?
•
There is a need for revising outcome measures used in
psychiatric research to be more clinically meaningful and
more reflective of clinical practice and use
outcomes of
importance to clinicians and patients.
IASCT Clinical Trial Interfaces 2011; Mumbai
Improving the standards of clinical trials
Catch them younger
IASCT Clinical Trial Interfaces 2011; Mumbai
www.consort‐statement.org
IASCT Clinical Trial Interfaces 2011; Mumbai
Ethical Considerations
IASCT Clinical Trial Interfaces 2011; Mumbai
CRITICAL ISSUES ON THE DESIGN AND REPORTING OF RCTS
IASCT Clinical Trial Interfaces 2011; Mumbai
Ingredients for a good protocol
•
Is there a justification for the trial based on the results of a
good quality systematic review?
•
Does the protocol contain a well constructed research
question with details of participants, interventions,
comparisons (including routes and doses) and outcomes,
(primary and secondary) (PICO)?
•
Does the protocol describe adequately the methods to
generate the randomization sequence, conceal allocation,
blind participants and outcome assessors and evaluate
blinding (if appropriate), and methods to deal with trial
attrition?
IASCT Clinical Trial Interfaces 2011; Mumbai
Target Population
Allocation
Follow up Follow up
Exposed to intervention
Not exposed to intervention
Intervention group Control group
Blinding of investigator/
subjectBlinding ofoutcome
assessment
Intention-to-treatFollow up
Allocation concealment
Method to minimize
Selection
Performance
Detection
Attrition
OutcomeOutcome
Biases
IASCT Clinical Trial Interfaces 2011; Mumbai
Selection bias
•
systematic differences in participant characteristics
at the
start of a trial
•
occurs at the time of allocation into comparison groups
Intervention group Control group
IASCT Clinical Trial Interfaces 2011; Mumbai
Selection bias
•
If groups differ at baseline in characteristics that alter
prognosis, this will influence differences in outcomes
between interventions
•
Inadequately concealed allocation in trials
overestimate treatment effects by:
•
~ 30% ( 95% CI 21% ‐
38%) (Schulz et al, 1994)
•
Most important source of bias
Schulz KF et al. JAMA 1994; 272:125‐128
IASCT Clinical Trial Interfaces 2011; Mumbai
Allocation concealment vs
blinding
time
randomisation
Concealment of allocation Blinding
Selection bias Performance bias & outcome detection bias
•Blinding may not be always possible but allocation concealment is always possible (even in surgical trials)•Without allocation concealment, blinding may be difficult
IASCT Clinical Trial Interfaces 2011; Mumbai
The effect of blinding on outcome
•
Inadequate
blinding
leads
to
performance
bias
(differential
interventions)
as
well
as
detection
bias
and
differential placebo effects
•
Inadequately
blinded
or
open
trials
tend
to
over‐estimate
results than adequately blinded trials by•
17% (Schulz et al, 1995)
•
35% (1% to 57%) (Juni
1999)
Schulz KF et al. JAMA 1995; 273: 408‐412
IASCT Clinical Trial Interfaces 2011; Mumbai
Attrition bias
Systematic differences in the loss of participants to follow
up
between groups (and from start of trial)
•
Participants lost to follow up, or not included in the outcome
assessment, could be different from those who remained in
the trial
Occurs over the duration of follow up
Avoiding attrition bias
Completeness of follow up
describe proportion of participants lost to follow‐up (with
reasons)
IASCT Clinical Trial Interfaces 2011; Mumbai
Dealing with missing data•
Analysis of ‘completers’
only
•
Intention to treat•
For continuous data
•
Imputation of simple mean
•
Imputation of regression mean
•
Last observation carried forward
•
For dichotomous data
•
Assuming worst case scenarios
•
Imputing event rates for missing values
•
Sensitivity analysis: comparing results of completers versus ITT analyses
IASCT Clinical Trial Interfaces 2011; Mumbai
Ingredients…continued
•
Does the protocol describe in sufficient detail the estimated sample size for the primary outcome(s) and (if needed, secondary outcomes) and its
justification? Are the outcomes clinically relevant?
•
Does the protocol describe adequately the responsibilities of investigators and methods to
ensure integrity of data collected and rules for interim analysis?
•
Does the protocol ensure that all elements that are required to be reported by the CONSORT and ICMJE guidelines will be collected and recorded?
IASCT Clinical Trial Interfaces 2011; Mumbai
Ingredients‐
continued
•
Are ethical issues addressed adequately in accordance with international and local
regulations and requirements?
•
Has ethics committee approval been obtained?
•
Is the trial registered prospectively in a publicly accessible Trials Register? (Avoiding reporting
biases)
IASCT Clinical Trial Interfaces 2011; Mumbai
Ingredients‐
when reporting results
•
Is the trial registration number reported?
•
Have deviations from the protocol been reported?
•
Is there a description or diagram of participant flow though
the various stages of the trial?
•
Have absolute values been reported in addition to
proportions for binary outcomes?
•
Have means as well as standard deviations (or standard
errors) and numbers of participants for each intervention
been reported for continuous outcomes?
•
Have effect measures and confidence intervals been
reported in addition to (or instead of) p values?
IASCT Clinical Trial Interfaces 2011; Mumbai
Ingredients‐
when reporting results
•
Have drop‐outs and withdrawals been described? Was an
intention to treat analysis used?
•
Have post‐hoc analyses and sub‐group analyses been kept to
a minimum?
•
Have all important outcomes been reported?
•
Have funding sources and conflicts of interest been
declared?
IASCT Clinical Trial Interfaces 2011; Mumbai
HOW NOT TO DO IT
Assessing the validity of interventional research in India
IASCT Clinical Trial Interfaces 2011; Mumbai
The internal validity of RCTs from India“Sub-optimal …or shall I shall say..Poor show, old chap?”
IASCT Clinical Trial Interfaces 2011; Mumbai
2004 & 2005 compared to 2007 & 2008
•
Of the 65 journals:•
38
(59%)
in
2005
and
37
(57%)
in
2008
mentioned
the
ICMJE
requirements in their instructions for authors,
•
Only
20
(31%)
in
2005
and
22
(33%)
in
2008
specifically
required
authors
to
submit
manuscripts
in
accordance
with
the
CONSORT
statement.
•
Of
151
RCTs
published
in
2004‐2005,
and
145
RCTs
published
in 2007‐2008:•
Only 4/13 (31%) of selected CONSORT items were reported in > 50%
of
trial reports.
George A, Kirubakaran
P, Barnabas J, Tharyan
P. 3rd
South Asian Regional
symposium in Evidence‐informed Healthcare Jan 2010
IASCT Clinical Trial Interfaces 2011; Mumbai
Unhappy Endings
Improved
Improved
Improved
No change
No changeNo change
No change
No change
No change
No change
Worse
Worse
Worse
IASCT Clinical Trial Interfaces 2011; Mumbai
Ethical Considerations
108 (71)107 (73)
103 (68)118 (81)
39 (25)66 (45) *
11 (7)60 (41) *
0 20 40 60 80 100 120
Number of Trials (%)
Ethical Committee ClearenceInformed Consent
Source of FundingConflicts of Interest
Figure - II: Reporting of ICMJE Requirments
2007-2008(145)2004-2005(151)
*Conflicts of Interest
: Odds Ratio
2.4;
95% CI (1.4 ‐
4.0)
*Source of Funding
: Odds Ratio
8.9; 95% CI (4.2 ‐
19.2)
IASCT Clinical Trial Interfaces 2011; Mumbai
More unhappy endings: The internal validity of RCTs
in Mental Health in LMICs
Internal validity measure N=177
Sequence generation 30 (17%)
Allocation concealment
Blinding
21 (12%)
109 (62%)
Assessment of blinding 0 (0%)
IASCT Clinical Trial Interfaces 2011; Mumbai
Poor quality of research
•
Fourth Congress on Peer Review in Biomedical Publication concluded that:
•
Medical journals are full of serious methodological errors
•
Journal editors are giving no time, energy and thought to their craft
•
Studies are published that reach false conclusions
BMJ, 2001
IASCT Clinical Trial Interfaces 2011; Mumbai
Whose fault is this?
•
Editors
•
Peer reviewers
•
Researchers
•
Institutions
•
Institutional review boards•
Ethics committees
•
Research committees
•
Funders of research
•
Educational programmes
and educators
•
Implementers of research evidence
•
Consumers
•
The system of complacency
Chaltha
he!
Its only a PG thesisanyway
IASCT Clinical Trial Interfaces 2011; Mumbai
IF YOU WANT A GREAT ENDING..
YOU NEED A GREAT BEGINNING…
AND MIDDLE…
Improve the design
and conduct of research..
To improve the reporting of results
of research
Ensure that trials are prospectively registered with IRB approval to prevent reporting biases
IASCT Clinical Trial Interfaces 2011; Mumbai
Clinical Trials Registry‐
India
www.ctri.in
The CTRI is a Primary Register of the WHO-ICTRP
IASCT Clinical Trial Interfaces 2011; Mumbai
Mission and Vision
Mission•
Encourage
all
clinical
trials
to
be
prospectively
registered before enrolment of first patient•
To comply with ICMJE and WHO‐ICTRP requirements
Vision•
Ensure registration of all clinical trials prospectively
•
Ensure
voluntary
disclosure
of
all
items
in
the register :
•
To improve transparency
•
To improve internal validity of trials
•
Conform to accepted ethical standards
IASCT Clinical Trial Interfaces 2011; Mumbai
Registration Data Set: CTRI specific items
Trial registration data set with explanations can be downloaded
IASCT Clinical Trial Interfaces 2011; Mumbai
IASCT Clinical Trial Interfaces 2011; Mumbai
Compliance
100% : 13/20
95‐99%: 5/20
90‐95%: 1/20
85‐90%: 1/20
IASCT Clinical Trial Interfaces 2011; Mumbai
Ethics committee
approval:100%
Random sequence
generation: 82.6%
Allocation
concealment: 76%
Blinding 85%
IASCT Clinical Trial Interfaces 2011; Mumbai
Comparing the endings and beginnings
Journals
2004‐2005(N=151 )
Journals
2007‐2008(N=145 )
CTRI2009 Jan
(N=144)
CTRI2010Aug
(N=1002)
Verdict
Random
Sequence
Generation
38% 55% 83% 78%CTRI
better
Concealment of
allocation 16% 21% 76% 84%
CTRI
better
Blinding 64% 46% 85% 88%CTRI
better
IASCT Clinical Trial Interfaces 2011; Mumbai
Central Drugs Standard Control
Organization
Directorate General of Health Services, Ministry of Health and Family Welfare, Government
of India
http://cdsco.nic.in/
IASCT Clinical Trial Interfaces 2011; Mumbai
E‐mail from Dr. Surinder
Singh;
Drug Controller of India
•
“The registration of clinical trials is mandatory in most of the
developed countries and in India it was made Advisory in the
Clinical Trial Permissions issued by office of DCG(I),
•
However the clinical trial registry was made mandatory from
15th
June, 2009 for all the applicants to whom the permission
is granted by office of DCG(I).
•
It may be mentioned here that while the number of trials
registered till Dec, 2008 was 137, the number of trials
registered between Jan, 2009 to Nov, 2009 are 464, out of
which around 373 trials are registered after 15th
June, 2009
after it was made mandatory by the Office of DCG(I).”
IASCT Clinical Trial Interfaces 2011; Mumbai
WHAT’S NEXTWhat about investigator initiated trials?
IASCT Clinical Trial Interfaces 2011; Mumbai
WORKING WITH MEDICAL JOURNAL EDITORS IN INDIA IN
ENDORSING TRIALS REGISTRATION
Meeting with Medical Journal Editors (ICMR)
October 9 2007
IASCT Clinical Trial Interfaces 2011; Mumbai
Editorial policy on trials registration
IASCT Clinical Trial Interfaces 2011; Mumbai
IASCT Clinical Trial Interfaces 2011; Mumbai
Get the Evidence Right
The Devil is in the Detail
Don’t
Just Do It; Do It Right
IASCT Clinical Trial Interfaces 2011; Mumbai
“When you know something, to hold that you know it, and when
you do not know something, to allow that you do not know it;
that is knowledge.”
Confucius (551 BC ‐
479 BC)
IASCT Clinical Trial Interfaces 2011; Mumbai
Join us: www.cochrane‐sacn.org