r&d evaluation at niaid: making the most of expert opinion
DESCRIPTION
R&D Evaluation at NIAID: Making the Most of Expert Opinion. Kevin Wright AEA Annual Conference Nov 4, 2011. Overview. NIAID at a Glance Highlights Organization R&D Evaluation at NIAID Intent Implementation Improvements Looking Ahead Opportunities Threats. NIAID at a Glance. - PowerPoint PPT PresentationTRANSCRIPT
R&D Evaluation at NIAID:Making the Most of Expert Opinion
Kevin WrightAEA Annual Conference
Nov 4, 2011
OverviewNIAID at a Glance– Highlights– Organization
R&D Evaluation at NIAID– Intent– Implementation– Improvements
Looking Ahead– Opportunities– Threats
NIAID at a Glance
3
NIAID R&D Evaluations
Level 3
Level 2
Level 1 NIAID OD
Division 1
Program A
Solicited Unsolicited
Program B
Division 2
Program C Program D
Initial idea
Program DefinitionReviewResearch
RFA/PA written, grants
submitted
Peer review by
study section, Council
Research conducte
d
Scientific results
Division retreat/ program planning
Concept review by
RIC, Council
NIAID Program Cycle
Primary Conclusion of the Interagency Working Group:
“Expert judgment” remains the best available decision support tool for science policy makers, but a nascent community of practice is emerging in the science policy arena that holdsenormous potential to provide rigorous and quantitative decision support tools in thenear future. – The Science of Science Policy: A Federal Research Roadmap, Nov 2008.
Mission
CoreValues
To enhance the quality, utility, transparency and application of evaluation in support of decision making throughout NIAID.
Quality – The Evaluation Section will strive to meet and exceed expectations
Utility – All services and products will be developed and applied where appropriate
Collaborative – The Evaluation Section will listen to and partner with its customers
Innovative – The Evaluation Section will continually seek to enhance products and services
NIAID R&D Evaluations
Evaluation Planning TemplateProgram
TypeProgram
ObjectivesIndicator(s) Data Source Collection
ScheduleBasic Research
Fund research to fill gap
Projects and Pubs
IMPAC II and Pubmed Quarterly
Applied Research
Identify promising candidates
Press Release, Pubs, Pats
Web, Pubmed, USPTO
Annually
Clinical Research
Evaluate candidates
Protocols, Licenses, NDA
ClinicalTrials.gov, Web, FDA Annually
Physical Capacity
Reduce barriers Indirect costs IMPAC II Annually
Intellectual Capacity
Maintain pipeline of investigators
NI/ESI IMPAC II Annually
9
Analytic Tools
Opportunities
• Improving Tools and Resources• Community of Knowledge• Alignment of Research Priorities• Increased Scrutiny of Results
Threats
• Few Success Stories• Inconsistency– Rigor– Use by Decision Makers
• Culture– Management v. Research– Results v. Process
Contact Info
Kevin D. Wright, MPADeputy ChiefStrategic Planning and Evaluation BranchOSPFM, OSMO, NIAID, NIH, DHHS(p) [email protected]