evidence-based funding: supporting biomedical research with … · 2017-05-16 · evidence-based...
TRANSCRIPT
Evidence-Based Funding: Supporting Biomedical Research with Austere Budgets
Michael S Lauer, MD Deputy Director for Extramural Research
National Institutes of Health
Massachusetts General Hospital Research Institute Thursday, November 3, 2016
Massachusetts General Hospital, Simches 3110, Boston, MA Disclosures: None
1
A Problem?
2 Shapiro D, Vrana K. PNAS 2015;112:9496-7
What Should Our Metrics Be?
3 Shapiro D, Vrana K. PNAS 2015;112:9496-7
Better Finish Line
4 Moses H et al. JAMA 2015;313:174-189
Funding
5
Moses H et al. JAMA 2015;313:174-189
Funding …
6 Moses H et al. JAMA 2015;313:174-189
Annual Growth 2004 – 2012: Pharma -0.6% Device +6.2% Biotech +4.6% NIH -1.8%
Workforce
7
Moses H et al. JAMA 2015;313:174-189
Root Causes…
8
“We identified two core problems: • Too many researchers vying for too few dollars. • Too many postdocs competing for too few positions. Most other issues can be viewed as symptoms.”
A Different Metric
9
Changing our funding metric “A question that at first glance may seem trivial but is, I believe, a significant one is whether our key metric for how… we invest in … research should be the number of grants we award or the number of investigators we support.”
Lorsch JR. Mol Biol Cell 2015;26:1578-82
10
Thanks to OER DPEA and SARB
11
Thanks to OER DPEA and SARB
Distribution of Dollars to Awardees …
12
Alberts B et al. PNAS. 2014;111:5773-7
“Agencies should be sensitive to the total numbers of dollars granted to individual laboratories…—although different research activities have different costs—at some point, returns per dollar diminish. We applaud the recent decision by the NIH to examine grant portfolios carefully before increasing direct research support for a laboratory beyond $1M per year.”
13
Thanks to OER DPEA and SARB
14
Thanks to OER DPEA and SARB
15
An Ongoing Conversation
16
Xie Y. Science 2014;344:809-810
Publication and Citations?
17 Shapiro D, Vrana K. PNAS 2015;112:9496-7
So How Do We Measure Research Impact?
• P = productivity – Publish trial results – Highly-cited papers: field-normalized
• Q = quality • R = replication • S = sharing • T = translation
18 Ioannidis JP, Khoury MJ. JAMA 2014 (June 9)
Relative Citation Ratio
19
NIH Office of Portfolio Analysis PLoS Biology (September 6, 2016)
What It Means
20
NIH Office of Portfolio Analysis PLoS Biology (September 6, 2016)
How Do We Know Whether It Means Anything?
21
NIH Office of Portfolio Analysis PLoS Biology (September 6, 2016)
2193 R01 papers 430 HHMI/NIH papers 290 R01 papers
Sample
• All PubMed papers – Published between 1995 and 2014 – With known RCR values (only 0.09% missing)
• Categorized by funding source (SPIRES) – NIH-MGH funded papers: N=24,659 (<1%) – Other NIH funded papers: N=1,415,696 (11%) – Non-NIH papers: N=11,922,575 (89%)
Thanks to OER DPEA and SARB Thanks to OER Data Systems