Welcome to the Review Panel for Critical Resilient Interdependent

Infrastructure Systems and Processes (CRISP)

June 2015

Related'solicita,on:'NSF'154531'

Participating Directorates

Engineering'(ENG)'

Computer'&'Informa6on'Science'&'Engineering'

(CISE)''

Social,'Behavioral'and'

Economic'Sciences''(SBE)'

CRISP'

CRISP Team of Program Officers

Bruce Hamilton (ENG/CBET)

Elise Miller-Hooks (ENG/CIS)

Robert O’Connor (SBE/SES)

Rajiv Ramnath (CISE/ACI) Gurdip Singh

(CISE/CNS)

Sudipta Sarangi (SBE/SES)

Dennis Wenger (ENG/IMEE)

Addi,onal'support'and'par,cipa,on'from'

Tarneisha'Gross,'Rachel'Rosen,'Gabriela'Galeano,'Candace'Owens'

Background •  Anticipated Funding Amount: $20M

–  Type 1 Awards: Projects will be of 3 years in duration with a maximum total budget of $500,000

–  Type 2 Awards: Projects will be of 3-4 years in duration with a total budget ranging from $1 million - $2.5 million

•  Distinguishing Requirement: –  at least one PI or co-PI who is an engineer –  at least one PI or co-PI who is a computer, information

or computational scientist –  at least one PI or co-PI who is a social, economic or

behavioral scientist

Type 1 Awards •  Theory, modeling, data collection and metrics projects

that create: –  knowledge, representations, methodologies, case studies,

approaches

to conceptualize and study interdependent infrastructures as processes, services and systems

•  Objective of team building to help clarify –  basic terminology –  assumptions and premises that enable theories –  model and metric formalizations

for interdependent infrastructures as processes and services

•  Not intended for empirical testing of models/theories

Type 2 Awards •  Interdisciplinary research to conduct:

–  major new interdependent infrastructure research using empirical data

•  Expected to include creation of knowledge, representations, methodologies and approaches to –  conceptualize and study interdependent infrastructures as

processes, services and systems

•  Infrastructures are generally conceived here: –  as networks of systems and processes –  that function collaboratively and synergistically –  that produce & distribute continuous flow of essential goods & services –  as interdependent and connected

•  Examples of interdependencies (1) In a hazard event, emergency services (response/repair) required for restoration of critical services (power/transportation/healthcare…), and critical services enable emergency response/repair activities (2) Manufacturing-Banking/Finance: Demand by manufacturing sector for financing depends on economy which affects loan rates, and economy depends on goods production from manufacturing sector

Interdependent Critical Infrastructure Systems (ICIs)

Crit

ical

Infra

stru

ctur

e S

ecto

rs

US'DHS:'Na,onal'Infrastructure'Protec,on'Program'(NIPP)'

hLp://testdhsgov.edgesuite.net/files/programs/gc_1189168948944.shtm

'

Overarching Goals 1.  Create new approaches/solutions for design/operation of

infrastructures as processes/services 2.  Enhance understanding/design of interdependent critical

infrastructure systems (ICIs) and processes that provide essential goods and services despite disruptions/failures/disturbances from any cause –  natural, technological, organizational or malicious –  various timescales and intensities

3.  Create knowledge for innovation in ICIs to safely, securely, and effectively expand range of goods and services they enable

4.  Improve effectiveness, efficiency, dependability with which they deliver existing goods and services

CRISP Research Objectives 1.  Create new knowledge, approaches, solutions to:

–  increase resilience, performance, readiness in ICIs

2.  Create frameworks/multidisciplinary models of ICIs, processes and services: –  capable of analytical prediction of complex behaviors –  capable of real-time control and dynamic adaptation/reconfiguration –  responsive to system and policy changes

3.  Develop frameworks to understand interdependencies created by interactions between: –  physical, cyber, social, behavioral and economic ICI elements

4.  Understand organizational, social, psychological, legal, economic, technical obstacles to: –  improving ICIs, identifying strategies for overcoming obstacles

NSF Wide Evaluation Criteria

1.  Intellectual Merit (IM): proposal encompasses the potential to advance knowledge

2.  Broader Impacts (BI): proposal encompasses the potential to benefit society and contribute to the achievement of specific, desired societal outcomes.

What is the potential for the proposed activity to:

a)  Advance knowledge and understanding within its own field or across different fields (Intellectual Merit); and

b)  Benefit society or advance desired societal outcomes (Broader Impacts)?

Common Criteria for IM and BI •  To what extent do the proposed activities suggest and

explore creative, original, or potentially transformative concepts?

•  Is the research plan well-reasoned, well-organized, and based on a sound rationale? Does the plan incorporate a mechanism to assess success?

•  How well qualified is the individual, team, or organization to conduct the proposed activities?

•  Are there adequate resources (at home organization or through collaborations) to carry out the proposed activities?

NSF Wide Evaluation Criteria (cont.)

1.  Does the proposal primarily address scientific research on interdependencies among critical infrastructures?

2.  Does the research sufficiently reflect interdisciplinary efforts and help cultivate an interdisciplinary research community in critical infrastructure related to participating directorates?

3.  Do the project personnel have the expertise to conduct necessary interdisciplinary research?

Additional Review Criteria for CRISP

Transformative Research

Transformative research •  generates ideas, discoveries, or tools that

–  radically change our understanding of an important existing scientific or engineering concept or lead to the creation of a new paradigm or field of science or engineering.

Such research is characterized by its challenge to current understanding or its pathway to new frontiers.

NSF is a Government Agency

•  We are required to avoid conflicts of interest.

•  We seek fair and unbiased reviews.

You must not participate in the discussion of any proposal for which you have a conflict in this panel.

Sign and return Conflict-of-Interest form

PERSONAL !  co-author of paper or project

collaborator (48 months) !  co-edited journal or proceedings

(24 months) !  doctoral thesis advisor or

student (life-long) !  family member

INSTITUTIONAL !  current or previous employment

(12 months) or seeking employment

! Honorarium payment of more than $1,500 in past 12 months

!  officer or governing board !  any financial interest

Conflicts of Interest (COI)

Judge Proposals on their Merits

•  Judge each proposal on what it says •  Do not seek external information – no Googling, etc.

•  Judge by review criteria we’ve provided •  Point to specific evidence supporting judgments

Ethics and Anonymity •  Obligation to maintain the confidentiality of the

proposals and the panel discussion •  Reviewers are anonymous; we won’t tell,

you must not tell

•  Your recommendations are advisory and confidential; do not inform PIs of their status

•  Only abstracts and basic data of awarded proposals go into the public domain

•  Declined proposals are not in the public domain; do not distribute, use or quote; delete after use

Confidentiality •  You must maintain the confidentiality of the panel, all

materials/ideas associated with the panels, and any panel recommendations

•  Remote attendees: make sure no one at your location (other than you) can hear and/or see the panel meeting.

•  Use of social media during the panel can violate this requirement

Triage •  If a proposal receives:

–  fewer than two “good” ratings or better –  and fewer than one “very good” or better the panel can choose to triage without discussion

•  A triage decision must be unanimous •  Absolutely no discussion of that proposal can take

place; otherwise, the proposal will need to complete the full review process.

•  If any panelist wishes to discuss a proposal that is eligible for triage, the proposal will complete the full review process.

Panel Discussion •  Panel will consider approximately 30 proposals

–  4 reviews supplied per proposal •  Discussion time: ~15-20 minutes/proposal

–  Not necessary to spend same amount on each •  At end of each discussion, the lead should ask

for consensus rating: –  Highly Competitive (HC) –  Competitive (C) –  Low Competitive (LC) –  Not competitive (NC)

Panelist Roles •  Lead panelist:

–  2- to 3-minute summary of overall proposal –  Seek consensus on initial placement on the board

•  Other reviewers –  Other assigned panelists make comments on proposal’s

strengths and weaknesses –  Reviewers (except scribe) comment in order of highest

to lowest ratings; scribe comments last –  Try not to repeat earlier comments with which you agree

•  Other panelists –  The floor is open to discussion for any unconflicted

panelist, not just those who are designated reviewers

Scribe •  Use provided template to draft panel summary that

reflects key elements of discussion and consensus of panel "  Reviewers may edit for substance and tone "  Comments should be constructive, informative, non-

inflammatory, and non-discriminatory •  Save review as plain text before cutting and pasting

into Fastlane and click ‘Submit draft for comment’ to share with others

•  Submit summary for final approval

Panelist Roles

Changing Individual Reviews

•  You should not change the content of your review by inserting new material into the original review

•  You may fix typos if desired •  You may, if you choose, add new material to the

original review, marked as new material, at the end of the original review

•  Changes in opinions by reviewers from their original written reviews should be noted in the panel summary

Before you leave…

•  Before you leave… –  All competitive (HC-LC) proposals must be ranked –  All panel summaries must be submitted –  COI forms, sign-up, and final panel recommendation

sheet must be signed

Please be sure to register for the panel and sign in on the day of the panel

Thank you for your service to NSF and our larger research communities