quality analysis of nsf dmp plans - wayne state university
TRANSCRIPT
Quality analysis of
NSF data management plans
written by WSU faculty
Research Data Services
WSU Library System
National Science Foundation
requirement for
data management plans (DMPs)
Beginning January 18, 2011, all proposals submitted to NSF
must include a supplementary document of no more than
two pages describing how the proposal will conform to NSF
policy on the dissemination and sharing of research results.
Usually reviewed as part of the intellectual merit or broader
impacts of the proposal.
2
Other federal funding agencies
will require DMPs in near future
National Institutes of Health (NIH)
Veterans Administration (VA)
Department of Education (ED)
Department of Energy (DOE)
Department of Defense (DoD)
Agency for Healthcare Research and Quality (AHRQ)
National Aeronautics and Space Administration (NASA)
US Geological Survey (USGS)
Environmental Protection Agency (EPA)
Centers for Disease Control and Prevention (CDC)
Food and Drug Association (FDA)
National Institute of Justice (NIJ)
Department of Agriculture (USDA) 3
What is meant by “research data”?
Federal government definition: “ . . . the recorded factual
material commonly accepted in the scientific community as
necessary to validate research findings.”
Includes: spreadsheets, images, audio or video files,
transcripts, protein or gene sequences, spectra, models,
computer code, software, etc.
Excludes: physical objects (e.g., tissue samples, model
organisms), trade secrets, commercial information, protected
medical information
4
DMP content guidelines*
1. Types of data and other materials to be produced in the
course of the project
2. Standards to be used for data and metadata format and
content
3. Policies for access and sharing, including provisions for
appropriate protection of privacy, confidentiality, security,
intellectual property, or other rights or requirements
4. Policies and provisions for re-use, re-distribution, and the
production of derivatives
5. Plans for archiving data and other research products and
for preservation of access to them
*per NSF Grant Proposal Guide (GPG) Chapter II.C.2.j
5
Quality review of DMPs
● 140 DMPs were obtained from SPA
● DMPs were from NSF grants (half funded and half
unfunded) submitted between 2012-2014
● Removed from analysis: 21 “no-data” DMPs (includes
conference, travel, and a few research proposals)
● Final sample size: 119 DMPs
● Each DMP was evaluated by two reviewers, who
reached consensus on disagreements
6
Final sample
College/School # of DMPs
College of Education 2
College of Engineering 61
College of Liberal Arts & Sciences 50
Law School 1
School of Medicine 5
Total 119
7
Yes
(49%)
No/unclear
(51%)
Are the individual(s) responsible for data
management specifically named?
8
How are the data characterized?
Yes
49%
No
51%
Generally
(i.e., content)
(32%)
Specifically
(i.e., container)
(7%)
Both generally
and specifically
(42%)
Not characterized/
unclear
(19%)
9
Yes
49%
No
51%
Is the expected total amount or rate of
data generation specified?
Yes (8%)
No/unclear (92%)
10
Will specific metadata standards and/or
other data descriptive methods be used?
Yes (38%)
No/unclear (62%)
11
Is a method of data backup specified?
Yes
(60%)
No/unclear
(40%)
12
Will any data be shared after the study?
Yes (92%)
No/unclear (8%)
13
How will data be shared after the study?
Best
practice
Invalid
response
14
Are policies for data reuse or
redistribution specified?
Yes (19%)
No/unclear (81%)
15
Do policies for data sharing specify
protections against disclosure of
sensitive information?
No/unclear (80%)
Yes (20%)
16
Do policies for data sharing specify
protections for safeguarding intellectual
property rights?
No/unclear (58%)
Yes (42%)
17
Is the duration of data preservation
specified?
No/unclear (57%)
Yes (43%)
18
Conclusions
DMPs written by WSU faculty vary widely in addressing NSF
guidelines; some are of good quality, but there is substantial
room for improvement.
Increased quality of DMPs could lead to increased funding
for WSU.
19
We can help!
We can consult with individual PIs or grant administrators to
create or refine DMPs. This could be facilitated by:
● Linking to RDS website or materials from SPA website
● Presenting information about DMPs and RDS to PIs during the pre-
award workflow
● Sending email to PIs offering DMP consultation/feedback during pre-
award workflow
rds.wayne.edu
20
We can help!
We can provide DMP training/guidance for groups of faculty
or grant administrators. This would include:
● Raising awareness of funding agency requirements
● Providing a “checklist” of DMP components
● Providing examples of well-written DMPs
rds.wayne.edu
21