working with nsf: writing, reviewing and rotating tanja pietraß

Working with NSF:Writing, Reviewing

and Rotating

Tanja Pietraß

The Writing Process

How to Write a Strong NSF Proposal

Stage 1 – Preparation

•Read the GPG•Follow instructions•Start early!•Respond to review requests!•Ask a mentor for a copy of a proposal that led to an award

How to Write a Strong NSF Proposal

Stage 2 – Research and Writing

•Read the GPG and follow the suggested format•Put your research in context to what’s been/being done•Take care when preparing your broader impact statement – it should not look like an afterthought•For CAREERs: Treat the education plan like a research project•Provide a list of ca. 5 suggested reviewers (provide address/phone/email; you may comment on relationship/expertise)

GPG: A Competitive ProposalGPG: A Competitive Proposal

“… individual projects should include clearly stated goals, specific descriptions of the activities that the PI intends to do, and a plan in place to document the outputs of those activities.”

How to Write a Strong NSF ProposalStage 3 – Review and Submission

•Make it perfect – proof-read!•Have a mentor proof-read your proposal for technical input•Share the proposal with a non-expert for general input (readability, logical construction, no discipline-specific jargon, sufficient level of background)•Check your figures (legibility, numbering, referred to in text?)•Submit early!•Proof-read the final NSF-generated pdf (or better, submit a pdf)

The Review Process

The Life Cycle of an NSF Proposal

Proposal Preparation Processing and NotificationProposal Review and Recommendation

90 Days 6 Months 30 Days

PI has an idea!

Proposal Receiptat NSF

NSF Admin.Review

Award/Decline?

DDConcur

Return Without Review

Award ?

Decline ?

Notification

DGA

Notification

Mail

Panel

Both

Proposal Review




Government Performance and Results Act of 1993 (GPRA)

Target = Process 70% of proposals within 6 months of receipt at NSF

Administrative Review / Compliance



PI has an idea!


NSF Admin.Review

Proposal is checked to verify that it complies with the rules and formatting parameters delineated in the Grant Proposal Guide (GPG)

Common issues include:

On certain issues that involve sections other than the Project Description or Project Summary, the PI may be asked to do a proposal file update (PFU) within 5 business days of notification


Biographic sketch is too long, too many publications Margins or font sizes are too small References are not in the correct format Improper supporting documents uploaded




PI has an idea!


NSF Admin.Review


Does not address Broader Impacts as a separate section within the Project Summary

Has a budget line for a postdoctoral research associate but does not have a one page postdoctoral mentoring plan uploaded in the supporting information section

Does not meet an announced proposal deadline or close of window date (and time, where specified)

Return Without ReviewReturn Without Review

The Proposal:

Is inappropriate for funding by the National Science Foundation (or the CHE Division)

Is a duplicate of, or substantially similar to, a proposal already under consideration by NSF from the same submitter

Was previously reviewed and declined and has not been substantially revised.

Return Without ReviewReturn Without Review

The Proposal:

Read the NSF Grant Proposal Guide (GPG) carefully Check that all of the necessary parts of the proposal are uploaded Make sure that Broader Impacts are addressed in a separate section within the Project Summary (and Project Description) Check that all of the figures appear as you intended in the .pdf file loaded into FastLane Follow any special instructions in a solicitation


Please be sure to:




PI has an idea!


NSF Admin.Review


Mail

Panel

Both

Proposal Review

Proposal ReviewProposal Review

The proposal content sets the direction of the review

Proposers are invited to suggest names of persons they believe are especially well qualified to review the proposal or identify persons they would prefer not to review the proposal

Follow GPG Guidelines and include full contact information

Three Principles of Merit ReviewThree Principles of Merit Review

1. All NSF projects should be of the highest quality and have the potential to advance, if not transform, the frontiers of knowledge.


2. NSF projects, in the aggregate, should contribute more broadly to achieving societal goals. These broader impacts may be accomplished through the research itself, through activities that are directly related to specific research projects, or through activities that are supported by, but are complementary to, the project. The project activities may be based on previously established and/or innovative methods and approaches, but in either case must be well justified.


3. Meaningful assessment and evaluation of NSF funded projects should be based on appropriate metrics, keeping in mind the likely correlation between the effect of broader impacts and the resources provided to implement projects. If the size of the activity is limited, evaluation of that activity in isolation is not likely to be meaningful. Thus, assessing the effectiveness of these activities may best be done at a higher, more aggregated, level than the individual project.

Merit Review CriteriaMerit Review Criteria

• Intellectual Merit: The Intellectual Merit criterion encompasses the potential to advance knowledge

• Broader Impacts: The Broader Impacts criterion encompasses the potential to benefit society and contribute to the achievement of specific, desired societal outcomes.

• What is the potential for the proposed activity to:

a. Advance knowledge and understanding within its own field or across different fields (Intellectual Merit); and

b. Benefit society or advance desired societal outcomes (Broader Impacts)?

• To what extent do the proposed activities suggest and explore creative, original, or potentially transformative concepts?

Elements of Merit ReviewElements of Merit Review

• Is the plan for carrying out the proposed activities well-reasoned, well-organized, and based on a sound rationale? Does the plan incorporate a mechanism to assess success?

• How well qualified is the individual, team, or organization to conduct the proposed activities?

• Are there adequate resources available to the PI (either at the home organization or through collaborations) to carry out the proposed activities?

Elements of Merit ReviewElements of Merit Review

The Intellectual Merit criterion encompasses the potential to advance knowledge.

Translation into questions a peer reviewer would want to know:

What is it about (the research objective)? What is already known and what will your research add

(context)? How will you do it (the technical approach)? Can you do it (you and your facilities)? Is it worth doing (intellectual merit and broader impact)?

How to translate this ….How to translate this ….

The Broader Impacts criterion encompasses the potential to benefit society and contribute to the achievement of specific, desired societal outcomes.

Translation into subjects a peer reviewer would want to know about: Education and training Providing opportunities for underrepresented groups Improving research and education infrastructure Dissemination of research results Communicating the benefits of research to nonscientists

How to translate this ….How to translate this ….




PI has an idea!


NSF Admin.Review

Award/Decline?


DDConcur

Award ?

Decline ?

Notification

DGA

Notification

Mail

Panel

Both

Proposal Review

A Competitive ProposalA Competitive Proposal

A competitive proposal is a good idea, well expressed, with a clear indication of methods for pursuing the idea, evaluating the findings, and making them known to all who need to know.

Decisions are based on the results of merit review and other considerations, such as program budget

Results of Merit Review

0

50

100

150

200

250

1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0

Rating

Act

ion

s

Declined Awarded

Conflicts of Interest (COI)

• Peer Review at NSF needs to be the “gold standard”

• Science ethics– Submitters to the federal government (e.g., NSF) are held to

high standards– A breach of ethics can lead to

• Being barred from submitting proposals

• Fines

• Jail

• Trouble with your employer

– Some ethics violations are a felony

Confidentiality

• NSF Proposals – all information is confidential– Ideas, information, PI names, salaries

– Rankings, reviews, reviewers’ names

• Proposal existence is confidential to PI and institution– Declined proposal is always confidential

– Award abstracts are published on public website

• NSF program staff, reviewers, panelists, Committee of Visitors can see proposals on “need to know” basis

Confidentiality

• Do not reveal your participation in specific review activities (e.g., don’t list on resume)

• Never discuss panel activity or your reviews with anyone outside of NSF-supervised activities

• Remember panel rankings or your review may not lead to an award due to programmatic considerations

• COI identification relies on self-identification and list of collaborators

Conflicts of Interest (COI)

• Institutional COIs

– Current, recent (12 mo) employee or negotiating for employment

– Recent (past 12 mo) honorarium or award

– Reemployment plans, stock, visiting committee member

• PIs, co-PIs, Senior Participants

– PhD or postdoc advisor or student (lifetime)

– Collaborator (past 48 mo), co-editor (past 24 mo)

– Business partner, friend

– Family

• Other COIs– Reviewer expresses a bias against a scientific approach

- A COI is where an individual cannot judge a proposal fairly because of another relationship with that proposal.

Unsure?Ask NSF

How We Deal with COIs at NSF

• Removal - reviewer that has COI is not used

• Recusal - panelist asked to leave the room when the particular proposal is discussed

• NSF Office of General Counsel mediates and can make allowances

• COI forms are signed by panelists and kept on file• Program Directors attend annual COI training• Every division has an Ethics Official

How to Write a Substantive Review

• Intellectual Merit• Broader Impact

– Postdoc Mentoring Plan

– Data Management Plan

• Program Specific Review Criteria

Postdoc Mentoring Plan

• Mandated by America COMPETES Act• Required if postdoc funds are requested• Supplementary document (less than 1 page)• Include, but not limited to:

– Career counseling

– Training in preparation of grant proposals, publications and presentations

– Guidance on ways to improve teaching and mentoring skills

– Guidance on how to effectively collaborate with researchers from diverse backgrounds and disciplinary areas

– Training in responsible professional practices

Data Management Plan

• Required since Jan. 2011• supplementary document; less than 2 pages • describes how the proposal will conform to NSF policy on the

dissemination and sharing of research results• Can’t submit without!• See NSF webpage for FAQs• Specific guidance for CHE:

http://www.nsf.gov/bfa/dias/policy/dmpdocs/che.pdf

Data Management Plan

• the types of data, samples, physical collections, software, curriculum materials, and other materials to be produced in the course of the project;

• the standards to be used for data and metadata format and content (where existing standards are absent or deemed inadequate, this should be documented along with any proposed solutions or remedies);

• policies for access and sharing including provisions for appropriate protection of privacy, confidentiality, security, intellectual property, or other rights or requirements;

• policies and provisions for re-use, re-distribution, and the production of derivatives; and

• plans for archiving data, samples, and other research products, and for preservation of access to them.

Be a Responsible Reviewer• We seek merit review by experts and generalists

– Comments are more important than the box checked (E, V, G, F, P)

• Be constructive– What’s good? What’s important? What’s missing? What

needs improvement?– Strengths & weaknesses– Review the proposal, not the person (except for

qualifications and productivity)– Comment on BOTH standard review criteria and any

additional programmatic criteria

Be a Responsible Reviewer…– OK to comment on clarity, budget, etc.– Avoid ad hominem and identifying comments.

• Respond in a timely manner– Respond quickly even if the answer is “no”– Please suggest alternatives– Most relevant reviewers are asked first– Your fast assistance assures timely response to the

proposer (6-month GPRA – Government Performance and Results Act – target)

– NSF requires 3 reviews

Notification:Declinations

Declinations…. The other story

• Of course, not all stories have happy endings…• Most proposals are declined

– Funding rates in CHE are 20-25%

• So chances are, at some point, you will receive that dreaded email…

• Remember, this happens to nearly everyone, experienced investigator and novice applicant alike…

Please keep in mind that…

• A declination is not a judgment of your value as a scientist

• It is not a judgment about your or your merit!• It is a merely a judgment regarding one

particular proposal in the context of– the other proposals in the pool

– available funding

– and possibly other issues

Nevertheless….

• You wish to increase your competitive edge• Some ideas on how to do this are the subject of

this presentation

What not to do when you are declined

• It is not a good idea to call up your program director right after you learn of your declination

• It is difficult to have a productive discussion until you have had a chance to read and think over your reviews (including panel summary and PO comments) and perhaps discuss them with your colleagues.

• It is really never a good idea to call up and holler at your program director.

What also not to do when you are declined

• It is an extremely bad idea to enlist colleagues, deans, provosts and others to lobby your program director on your behalf.– Actively discourage such behavior on the part of

zealous colleagues.

• Remember, we cannot and we will not discuss your proposal with anyone but you!

• It is also inappropriate to lobby for your own proposal!

Also..

• Please do not try to guess who the reviewers are• 99.9% of the time you will be just plain wrong• In light of this, be careful when you specify

reviewers you do not want us to use….– base this on objective knowledge

– you do not want to run the risk of losing a supportive reviewer

What to do when you are declined

• Email your program director and set up a time for a telephone appointment to discuss your proposal and reviews. Include your proposal number in the request!

• During this appointment, make every effort to listen with an objective mind, whether you agree or not.

• Do not waste time during this appointment arguing with your program director

• Do ask questions!!

Learn to read your reviews objectively (for everyone!)

• The responses to reviews I hear from many declined PIs fall into two categories:

• How could you decline me, the reviews were glowing!!

• Reviewers A, B, and C are obviously incompetent idiots– who clearly did not read my proposal

– who are out to get me

– who know nothing about the field

Try to read reviews objectively

• For this, it is often useful to enlist a colleague, one who will “tell it like it is”

• Try to analyze your reviews as if they were reviews of someone else’s proposal

• Make a list of “to do” items that emerge from the reviews

• If you think a reviewer misunderstood you, try to figure out if you can make your point more clearly

Typical reviewer comments (to help you analyze your reviews)

• The topic: old hat, crowded field, not important to chemistry

• The vision: poorly motivated, not clear where project is going, not clear how pieces hang together, what questions are being asked?

• The prospects for success: not convinced it will work, needs preliminary results, not enough detail for me to evaluate, methods not up to the task, too ambitious, PI does not realize difficulties, no plan B

Responding to reviewer comments

• Sometimes straightforward, e.g., provide more preliminary data, or detail, or a better motivation section, or improve your publication record or develop your broader impact section or proof read more rigorously or make it clear you know the obstacles or address particular concerns.

• Other times it is less so, particularly when the advice is not consistent.

• Learn to read between the lines of the reviews• Enlist help here!!!

Before you revise your proposal

• Ask yourself if you need to make major changes in– topic

– method or approach

• Is a different program more appropriate?• It is often a good idea to discuss these issues

with your program director• Until the proposal is submitted, these

conversations are highly encouraged

Resubmission

• Revise your proposal in light of reviews• Get comments on your revised proposal from

colleagues, including non-experts in your field. (important for experienced PIs, not just novices)

• Do not, repeat, do not simply submit the same proposal with cosmetic changes. This in no way increases its competitive edge.

Multiple Submissions

• If you submit various versions of your proposal to different NSF programs and other agencies you may reduce the chance of getting optimal reviewers for your proposal(s).

• The large increase in multiple submissions puts a huge strain on the reviewer community.

• NSF CHE discourages submission of more than one proposal in one submission window.

Notification:Awards

• Research Opportunity Awards (ROA) - Supplement

Opportunity (NSF-00-144)

• Research Experiences for Teachers (RET)

Supplement (NSF07039)

• Research Experiences for Undergraduate Students (REU) Supplement

• Other Types of Supplemental Support

(Discuss supplement with the cognizant Program

Director)

Supplements

ROA - Research Opportunity Award: Supplement Opportunity

• Enables faculty at predominantly undergraduate institutions (PUI), including community colleges, to pursue research as visiting scientists with NSF-supported investigators at other institutions

• Most frequently, ROA activities are summer experiences but partial support of sabbaticals is sometimes provided

• To improve research & teaching at both the PUI home institution and the host institution

• Making connections with a grantee: Networking; consult NSF Award Abstracts database for current awards in relevant programs

• Supplement requests to NSF can only be submitted by the NSF-supported investigators, not PUI faculty

RET and REU-Supplements• Similar to ROA Supplements

• Research Experience for Teachers (RET) Supplements support K-12 teachers to participate in ongoing, NSF-funded research projects for limited periods

- To enhance the professional development of K-12 teachers at the emerging frontiers of science and to bring new knowledge into the classroom

• Research Experience for Undergraduates (REU) Supplements support undergraduate students to participate in ongoing, NSF-funded research projects

Other Types of Supplemental Support

• In unusual circumstances, small amounts of supplemental funding may be requested to assure adequate completion of the original scope of work.

• NSF will NOT approve requests for supplemental support for such purposes as defraying the costs of increases in salaries, wages or staff benefits or for additional indirect cost reimbursement, whether caused by a change in the indirect cost rate or by changes in direct cost expenditures which affect the indirect cost base.

Renewal Proposal

• A request for additional funding for a support period subsequent to that provided by a standard or continuing grant

• It must be submitted at least six months before additional funding is required or consistent with an established deadline, target date or submission window.

• It competes with all other pending proposals.

Two types of renewal requests:

1. Traditional Renewal

2. Accomplishment-Based Renewal (ABR)

(See Chapter V of NSF Grant Proposal Guide)

Renewal Proposal

Traditional Renewal Proposal

• The box for "Renewal" should be checked on the proposal Cover Sheet.

• It is developed as fully as though the proposer were applying for the first time.

• It covers all the information required in a proposal for a new project, including results from the prior work.

• The 15-page limitation on the project description applies.

• It is appropriate only for an investigator who has made significant contributions, over a number of years, in the area of research addressed by the proposal.

• ABR proposals may not be submitted for consecutive renewals.

• Investigators are strongly urged to contact the cognizant Program Director prior to developing an ABR proposal.

• It must be clearly indicated in the proposal that it is an ABR submission and the box for "Accomplishment-Based Renewal" must be checked on the proposal Cover Sheet.

Accomplishment-Based Renewal (ABR)

Accomplishment-Based Renewal (ABR)

1. The Project Description is replaced with the following items:

a. brief summary (not to exceed four pages) of plans for the proposed support period;

b. information on human resources development at the postdoctoral, graduate and undergraduate levels; and

c. copies of no more than six reprints of publications as supplemental documents.(Of the six publications, two preprints, accepted for publication, may be included.)

2. All other information required (including broader impacts) for NSF proposal submission remains the same.

Technical Reporting Requirement• Information from these reports is used in annual reports

to Congress to demonstrate the Foundation’s performance as mandated by the Government Performance and Results Act (GPRA) of 1993.

• These reports also provide NSF Program Directors and administrative offices with information on the progress of supported projects and the way these funds are used.

• Information in these reports may be made available to the general public.

Annual and Final Project Reports

• Must be submitted via the electronic project reporting system in research.gov

• Annual Report should be submitted at least 90 days prior to the end of the current budget period

• Final Report must be submitted within 90 days following expiration of the grant

• In the case of continuing grants, failure to submit timely annual reports will delay processing of funding increments

(See NSF Award and Administration Guidelines)

Annual and Final Project Reports

• Document progress, outcomes, and important issues

• Highlight important outcomes and issues in the textboxes of the Fastlane project reporting system even though PDF files are uploaded

• Acknowledge funding from NSF and include the NSF “Disclaimer Statement” in all publications

Project Outcomes Report for the General Public

• For proposals submitted or updated (CGI, Suppl.) after Jan. 4, 2010

• Must be submitted electronically via Research.gov

within 90 days following expiration of the grant• Serves as a brief summary, prepared specifically

for the public, of the nature and outcomes of the project

• Will be posted on the NSF website exactly as it is submitted(See NSF Award and Administration Guidelines)

Compliance with Technical Reporting Requirements

Failure to provide these reports on a timely basis will delay NSF review and processing of pending proposals for all identified PIs and co-PIs on a given award.

Post-award Actions: Research Highlights

Proposal

Peer-Review

Funded GrantProposal

Perform Research Formal

Reporting

Highlight ResultsCommunicate to the PublicTell the story of your work!

Research Highlights • Written summaries of results, outcomes, and progress

from NSF investments in research and education activities.

• Brief and engaging stories about a research or educational discovery or result and why it is significant to the field of study and/or for society.

• Used to justify or support budget requests at Federal and NSF levels.

• Shared through various media and serve multiple purposes for informing Foundation stakeholders about progress in advancing discovery, innovation, and education.

• Written in a technical and/or non-technical style depending upon the audience.

Use of Research Highlights

• Electronic Proposal File• Budget Requests to Congress• NSF Advisory

Committees/Committee of Visitors

• NSF Reports• NSF Web pages-Press

Releases-Discoveries• NSF Staff Presentations• NSF Outreach Publications• Research.gov

5

working with nsf: writing, reviewing and rotating tanja pietraß

Documents

reviewthe proposal

nsf grant proposal guide

nsf proposalaward

strong nsf proposalstage

announced proposal deadline

proposal file update

review requests

project description