2013 Nobel Laureates

• Physiology or Medicine: James E. Rothman, Randy W. Schekman and Thomas C. Südhof

"for their discoveries of machinery regulating vesicle traffic, a major transport system in our cells“

• Physics: François Englert and Peter W. Higgs

"for the theoretical discovery of a mechanism that contributes to our understanding of the origin of mass of subatomic particles…"

Randy W. Schekman Peter W. Higgs

• sdf

Life sciences research in 2010

Lancet 2013;382:1286-307 & 2009;374:86–9

US$ 240,000,000,000

85% wasted

Avoidable waste or inefficiency in biomedical research

Are research decisions relevant to users of research?

Appropriate research design, methods, and analysis?

Efficient regulation and management?

Fully accessible research information?

Unbiased and usable research reports?

Research waste

Lancet 2014;383:101–4

The 42 “Wasters”…

A Metin Gülmezoglu, Andrew Vickers, An-Wen Chan, Ben Djulbegovic, David Moher, David W Howells, Davina Ghersi, Douglas G Altman, Elaine Beller, Elina Hemminki, Elizabeth Wager, Fujian Song, H Bart van der Worp, Harlan M Krumholz, Iain Chalmers, Ian Roberts, Isabelle Boutron, Janet Wisely, John P A Ioannidis, Jonathan Grant, Jonathan Kagan, Julian Savulescu, Kay Dickersin, Kenneth F Schulz, Malcolm R Macleod, Mark A Hlatky, Michael B Bracken, Mike Clarke, Muin J Khoury, Patrick Bossuyt, Paul Glasziou, Peter C Gøtzsche, Robert S Phillips, Robert Tibshirani, Rustam Al-Shahi Salman, Sander Greenland, Sandy Oliver, Silvio Garattini, Steven Julious, Susan Michie, Tom Jefferson, Ulrich Dirnagl

1. Setting research priorities

Lancet 2014;383:156–65

1. Setting research priorities

1 intervention used widely

5 resulted in licensed clinical interventions by 2003

101 claimed that new discoveries had clear clinical potential

>25,000 reports in 6 basic science journals 1979-83

Am J Med 2003;114(6):477-84

The inefficiency of basic science research

1. Setting research priorities Thomas Edison

"Young man, why would I feel like a failure? And why

would I ever give up? I now know definitively over 2,000 ways that an

electric light bulb will not work. Success is almost in

my grasp.“

1. Setting research priorities

Lancet 2014;383:156–65

Best basic:applied research funding ratio?

1. Setting research priorities

Lancet 2014;383:156–65

Whose priorities?

1. Setting research priorities

Lancet Neurol 2012;11:209

Involve stakeholders

1. Setting research priorities

Lancet 2014;383:156–65

Set research in the context of systematic reviews

1. Setting research priorities

Lancet 2014;383:156–65

What systematic reviews could have shown…

tPA in animal stroke models prone:supine SIDS:controls

1. Recommendations

• More research on research should be done to identify factors associated with successful replication of basic research and translation to application in health care, and how to achieve the most productive ratio of basic to applied research

• Research funders should make information available about how they decide what research to support, and fund investigations of the effects of initiatives to engage potential users of research in research prioritisation

• Research funders and regulators should demand that proposals for additional primary research are justified by systematic reviews showing what is already known, and increase funding for the required syntheses of existing evidence

• Research funders and research regulators should strengthen and develop sources of information about research that is in progress, ensure that they are used by researchers, insist on publication of protocols at study inception, and encourage collaboration to reduce waste

Lancet 2014;383:156–65

2. Design, conduct and analysis

BMC Med Res Methodology 2004;4:13

Incongruent statistical findings in publications in 2001 (rounding, transcription, or type-setting errors)

38%

62%

25%

75%

2. Design, conduct and analysis Failure to replicate published pre-clinical academic results

64%

36%

89%

11%

Lancet 2014;383:166–75

2. Design, conduct and analysis

Lancet 2014;383:166–75

High effect-to-bias ratio In vivo studies

2. Recommendations

• Make publicly available the full protocols, analysis plans or sequence of analytical choices, and raw data for all designed and undertaken biomedical research

• Maximise the effect-to-bias ratio in research through defensible design and conduct standards, a well trained methodological research workforce, continuing professional development, and involvement of non-conflicted stakeholders

• Reward (with funding, and academic or other recognition) reproducibility practices and reproducible research, and enable an efficient culture for replication of research

Lancet 2014;383:166–75

3. Regulation and management

Lancet 1990;336:846–7

“…the clinician who is convinced that a certain treatment works will almost

never find an ethicist in his path, whereas his colleague who wonders and doubts and wants to learn will

stumble over piles of them”

3. Regulation and management

Lancet 2014;383:176–85

Delays and inconsistency in ethics and governance

3. Regulation and management

Lancet 2014;383:176–85

Is regulation proportionate, when the public approves?

28%

72%

23%

77%

UK National Cancer Registry including postcode, name and address, and sending a letter inviting them to a research study

Finland national biobank of existing diagnostic and research samples

3. Regulation and management

HTA 2007;11:ix–105 & Trials 2013;14:166

RCTs recruited pre-specified sample size

69%

31% 45%

55%

114 RCTs funded by MRC or HTA in the UK in 1994-2003

73 RCTs funded by MRC or HTA in the UK in 2002-2008

3. Regulation and management

Lancet 2014;383:176–85

Better recruitment after UK clinical research networks

3. Recommendations

• People regulating research should use their influence to reduce other causes of waste and inefficiency in research

• Regulators and policy makers should work with researchers, patients, and health professionals to streamline and harmonise the laws, regulations, guidelines, and processes that govern whether and how research can be done, and ensure that they are proportionate to the plausible risks associated with the research

• Researchers and research managers should increase the efficiency of recruitment, retention, data monitoring, and data sharing in research through the use of research designs known to reduce inefficiencies, and do additional research to learn how efficiency can be increased

• Everyone, particularly individuals responsible for health-care systems, can help to improve the efficiency of clinical research by promoting integration of research in everyday clinical practice

Lancet 2014;383:176–85

0%

20%

40%

60%

80%

100%

USA UK

Australia

France

Denmark

Canada

Australia

Spain

UK

USA

Switzerl.

Germany

Spain

Netherl.

1980 - 1996 1997 - 2005

4. Accessible reporting

HTA 2010;14(8):iii, ix-xi, 1-193

Proportion of funded/completed research that is reported

4. Accessible reporting

Lancet 2014;383:257–66

Reporting is selective

4. Accessible reporting

Lancet 2014;383:257–66

Associations with reporting

Oseltamivir Rosiglitazone Rofecoxib/celecoxib etc

4. Accessible reporting Language bias

Chinese Biomedical Literature database

Chinese Medical Current Content

China National Knowledge Infrastructure

Chinese Scientific Journals database

Chinese Medicine Premier

2,500 biomedical journals, <6% indexed in Medline

Health Info Libr J 2008;25:55–61

4. Accessible reporting

Lancet 2014;383:257–66

4. Recommendations

• Institutions and funders should adopt performance metrics that recognise full dissemination of research and reuse of original datasets by external researchers

• Investigators, funders, sponsors, regulators, research ethics committees, and journals should systematically develop and adopt standards for the content of study protocols and full study reports, and for data sharing practices

• Funders, sponsors, regulators, research ethics committees, journals, and legislators should endorse and enforce study registration policies, wide availability of full study information, and sharing of participant-level data for all health research

Lancet 2014;383:257–66

5. Complete & usable reporting

Lancet 2014;383:267–76

5. Complete & usable reporting

Lancet 2014;383:267–76

5. Complete & usable reporting

Lancet 2014;383:267–76

5. Recommendations

• Funders and research institutions must shift research regulations and rewards to align with better and more complete reporting

• Research funders should take responsibility for reporting infrastructure that supports good reporting and archiving

• Funders, institutions, and publishers should improve the capability and capacity of authors and reviewers in high-quality and complete reporting

Lancet 2014;383:267–76

Avoidable waste or inefficiency in biomedical research

Are research decisions relevant to users of research?

Appropriate research design, methods, and analysis?

Efficient regulation and management?

Fully accessible research information?

Unbiased and usable research reports?

Research waste

Lancet 2014;383:101–4

How can, and will, we change?

Implementation Science 2011;6:42