PREPARING FOR SITE VISITS:
THE AGONY AND THE AGONYMEREDITH MURR, UC SANTA BARBARA; SHERYL SOUCY-LUBELL, UC DAVIS; BARRY ROWAN, UC SANTA BARBARA
RANDY PHELPS, NATIONAL SCIENCE FOUNDATION
WHAT WE WILL NOT COVER
Anything Pre-notification of site visit:
How to assemble a good team
How to define the mission of the Center
How to construct a good center proposal, etc.
… for excellent guidance on these topics, please see articles in the Research
Development and Grant Writing Newsletter, published by Lucy Deckard and Mike
Cronan of Academic Research Funding Strategies, LLC
USING THE NSF STC AS A CASE STUDY
Site visits come in all shapes and sizes.
Varying in level of guidance, structure, on-site versus reverse-site, etc.
Not practical to talk about all the variations
Key take home message – it takes planning. Lots and lots of planning. More than you
think. It is AGONY. Be prepared.
Many of the issues we encountered in preparing for the NSF STC site visit are
applicable to all site visits.
But, follow the guidance and instructions from the funding agency/program officers.
STC OVERVIEW
SCIENCE AND TECHNOLOGY CENTERS:
INTEGRATIVE PARTNERSHIPS
Large NSF Center program ($50M in FY2012)
Budget $5 million a year for 5 years (renewable up to 10 years)
Funds innovative, potentially transformative, complex research and education
projects that require large-scale, long-term awards.
Expectation for partnerships among academic institutions, national laboratories,
industrial organizations, and/or other public/private entities,
Significant education and diversity mandates
Knowledge transfer, including technology transfer, providing key information to public
policy makers, or dissemination of knowledge from one field of science to another.
HISTORY OF THE STC COMPETITION
Solicitation Released: January 18, 2011
Preliminary Proposal: Due May 30, 2011
269 Submitted, 267 Accepted for Review
Grouped into eleven themes/panels
Panel review occurred September 27-28, 2011
40 Invited back for Full Proposals – Notification October 11, 2011
Full Proposals: Due February 3, 2012
11 Invited for Site Visits
Site Visits: Invited Late June 2012
September – December 2012
Blue Ribbon Panel – January 2013
5 Projects recommended for funding
No awards announced yet
WHAT IS THE MISSION OF THE SITE VISIT
For you to convince the site review team that the project should be
funded
They need to understand the project
They need to be excited about the project – that it has “Impact” and is
“Transformative”
They need to believe the project can be accomplished
They need to believe this is the right team to tackle the project
They need to think there are adequate resources and knowledge base available
KEY INGREDIENTS
The science has to be transformative – already assessed at earlier stages, but still important
PI – seen as a strong and competent leader
The whole team is/appears to be on the same page
Integration of all of the presentations – the mission, education, knowledge transfer and scientific themes
To win, everything has to be perfect
Digest the hints from the full proposal reviews and respond
SITE REVIEW – ADDITIONAL REVIEW CRITERIA
Is the budget appropriate for the scale, scope and complexity of the proposed
Center's activities?
Does the proposed Center management demonstrate the vision, experience, and
capacity to manage a complex, multifaceted, and innovative enterprise that integrates
research, education, diversity and knowledge transfer?
Is the proposed management plan likely to be effective?
Is the role of the external advisory board clearly and appropriately defined?
Is there an adequate succession plan for the leadership of the Center?
Are intellectual property issues adequately addressed?
PREPARATION
STC COMPETITION 2012:
GUIDELINES FOR THE SITE VISIT (FROM NSF)
STC COMPETITION 2012:
GUIDELINES FOR THE SITE VISIT - HIGHLIGHTS
The duration of the presentations and the agenda are fixed and are the same for each
site visit.
In planning the agenda, please allocate 30% of the time of each presentation for answering
any questions that the Site Visitors may have.
Do not plan any lab/facilities visits or demos.
Please make sure that you have appropriate representatives of your university administration present.
you must provide a password protected URL where you will post your proposal, all reviews, your
response to the reviews, Site Visit Agenda, list of participants, and presentation slides.
You must designate an administrative liaison person representing your team and e-mail his/her name
and contact information (including e-mail, tel. and fax nos.)
SITE VISIT AGENDA
Day 0
1:00 p.m. – 6:00 p.m. Site Review Team arrives at Hotel
6:30 p.m. – 8:30 p.m. Site Review Team Meeting
Day 1
7:00 a.m. – 8:00 a.m. Travel to Site Review Location, Light Breakfast
8:00 a.m. – 12 noon Introductions
STC Rationale and Goals
Research
(10:00 a.m.-10:20 a.m.) NSF Executive Session/Break
Research
Facilities and Physical Infrastructure (as appropriate)
(12 noon – 12:30 p.m.) NSF Executive Session
12:30 p.m. – 1:30 p.m. Lunch – Discussion with students (present students of the PI, co-PIs and participants)
AGENDA, CONTINUED
Day 1 Continued
1:30 p.m. – 3:00 p.m. Integrating Research and Education
Developing Human Resources
(3:00 p.m. – 3:30 p.m.) NSF Executive Session/Break
3:30 p.m. – 4:15 p.m. Partnerships and Knowledge Transfer
4:15 p.m. – 5:15 p.m. Administration and Management Plans
5:15 p.m. – 5:30 p.m. Wrap-up
(5:15 p.m. – 6:15 p.m.) NSF Executive Session
6:15.p.m. – 6:35 pm Critical feedback to the Pis, list of questions that require clarification on Day 2 at 9am
7:30 p.m. – 7:45 p.m. Transportation of the Site Visit Review Team to Dinner
AGENDA, CONTINUED
Day 2
7:00 a.m. – 8:00 a.m. Travel to Site Visit Location, Light Breakfast
8:00a.m. – 9:00 a.m. Meeting with Administrators Only (no PIs)/Institutional Support
9:00 a.m. – 10:00 a.m. Summary/Proposing team’s response to Critical Feedback
10:00 a.m. – 4:00 p.m. Site Review Team prepares Site Visit Report
Working lunch provided
THE NITTY-GRITTY
TO-DO CHECKLIST, A SELECTION (SEE FULL VERSION ON-LINE)
Administrative
Contact Information for everyone involved
Date Reservations – for site visit and red team review
Practice Schedule – how many in person, how many remote
Travel and Practices
Room – where is the site visit going to be held? Practices?
Food (for site visit and practices)
Reserve hotel room blocks
Technology
Logos and Powerpoint templates
Designate a laptop and projector for presentations
Website
Have multiple people keeping track of these deadlines so they don’t fall through the cracks
COMPLEMENT OF TEAMS – GEOGRAPHY COMPLICATES PRACTICE
All STC proposals have
One Lead Institution
Multiple collaborating Institutions (some could be international)
Geographic and time zones – hard to arrange schedules
Multiple external partners – travel remote practices
PEOPLE – FOR UCSB PROPOSAL
Research/Proposal team – 21 (from 7 different institutions)
Research Development at UCSB – 3
Honorary Research Development – 1
Administration – who came to site visit
UCSB Chancellor, EVC, Vice Chancellor for Research, Dean of Science, Dean of Engineering, Dean of Graduate
Division
Collaborating Institution #1, Vice President for Research
Collaborating Institution #2, Dean of Engineering
Graduate students and postdocs – from all participating universities
MAP OF ROOMS
ESB 1001 ESB 2001 ESB 2003
Capacity 100 Capacity 45 Capacity 25
Arranged with round tables Sides of room lined with chairs
Breakfast, lunch, PI meeting room All Presentations Site Review Team Breakout Room
PRACTICE SCHEDULE -VISUAL
Person 1
Person 2
Person 3
Person 4
Person 5
Person 6
Person 7
Person 8
Person 9
Person 10
Person 11
Person 12
Person 13
RED TEAM REVIEW
3 weeks before the site visit
Red Team Panelists
Science experts (from Advisory Board)
Education experts (from Advisory Board)
NSF expert (former STC program officer)
Campus leadership
All proposal team members AND grads came to UCSB for the red team.
FOOD AND THE NSF CONSTRAINTS
Money constraints (upper limit, hard in Santa Barbara!!!)
Location constraints (requested walking distance – nothing closer than
20 minute walk)
Conflicting advice about logistics???
Reservations
Payment – snacks but not meals, can only pay credit card or cash
DESIGN ELEMENTS TO MAKE LOOK COHESIVE
Logos
PI pictures
Group pictures
Powerpoint templates
WHAT WENT INTO BINDERS
Agenda
Presentation
Proposal
Review
Response
Pouch with pens, highlighter, page tags, and memory stick with all media
files
WHAT WE SPENT – TOTAL $36,000
Office Supplies: $500
WebEx: $200
Mock Site Visit
Catering: $4,000
Room Rental: $1,500
Travel: $5,500
Printing Binders: $1,500
Site Visit:
Catering: $12,000
Travel: $10,000
Media: $700
Vans: $200
Does NOT include extra person power – at least one-
half a FTE for 3 months
UCSB absorbed most of the costs – but that
discussion should occur early in the process
Travel reimbursements – good to have a standard
packet and it is clear what is needed (grad students are
bad at saving receipts).
LESSONS FROM THE SITE VISIT
YOUR NEW BEST FRIEND – THE TIME TIMER
Site visitors are happiest when they are
engaged and can participate (ask
questions)
Need to build in time for this – and
stick to it!!!
Practice transitions – they take time
too
Need strategy for cutting off questions
if they go long
BEWARE THE NIGHT OF DAY 1
Answer questions – until 2am – plan on it!
We were given 12 questions
Prepared a 10-page response
Have someone for a snack/beer run
Present answers in morning session
Opportunity to revise answers in the afternoon
Deadline was 4pm to give final answers to site visit panel
everyone needs to stay until 5pm on the second day – no noon flights
UCSB SITE VISIT REPORT – SUMMARY EVALUATION
We received the site visit report at 4:30pm on Day 2
Major Strengths
The Center envisions “Grand Challenge” research of the highest technological and societal significance.
The large-scale funding provided by the Center structure is required to accomplish the Center vision, i.e. to accelerate the development of solutions to the pending end-of-electronics-scaling challenges.
The Research themes are integrated and address transformational research goals.
The proposal is well focused, and the scope and objectives are well formulated.
High societal impact if successful; high transformative potential for US positioning in global semiconductor industry.
High potential for Legacy through technical contributions/spinoff projects across multiple disciplines.
Balanced risk portfolio with fundamental and practical objectives
Exceedingly high level of technical expertise and unique infrastructure with institutional integration;
Builds on USCB excellence in electronic devices and materials.
Exceptionally strong support from Deans of principal institutes.
Strong education and outreach supported by novel assessment strategy.
Major Weaknesses
No major weaknesses identified.
OUR SECRET WEAPONS
Our PI
Strong, Brilliant, Organized
Barry
Dedicated person for all logistics and general good nature (solid 3 months, 1st month ~50%, 2nd and 3rd month ~90%)
Barry was exceptional – you may need 2 people!!!
This was a strategic opportunity for the Office of Research
From UC Davis
Week leading up – social event at PIs house – good thing!!!!
Professional pictures taken at practice – so all looked same
SURPRISES – PLANNING IS NEVER PERFECT
Printer in the executive session room
Power and wi-fi access important for all site visit reviewers
Bad thermostat
Light yellowing in projector
Shuttle did not operate early enough to get proposal team members to campus
Binders were HUGE!!! And heavily used by site reviewers
Site Visit team never went to the room we reserved for them – the project team had to leave the main room.
Luckily we had another area big enough to fit us all.
More drinks – the sodas went fast
Don’t choose Monday and Tuesday for site visit dates. Last minute things on weekend.
THINGS THAT WENT WELL THAT WERE OUT OF OUR CONTROL
We had a great Chair of the site visit team
Food was good
Weather was good
No one was sick
No natural disasters (week before there was a fire in SB, two weeks
after was Sandy)
LOOKING FORWARD
Research Development Team – now has the experience of what a good
site visit looks like and can advise other groups
Research Development Team – received a lot of good publicity and
recognition, resulting in more requests for our services
ERCs anyone?
TAKE HOME MESSAGES
Preparing for a site visit is a BIG project, requiring more time and effort than you
imagine
As soon as you find out, evaluate your administrative needs – hire a person if feasible
Find the money to fund travel, food, etc. Who pays?
Who is the “presenter” – always theme leader? Other considerations? Language
Knowledge transfer? Diversity?
PI has to be seen as leader
Practice and preparation are the keys to success
Best way for the team to be cohesive and on the same page is for them to be interacting frequently
In-person practices important so that ALL team members know ALL parts of the project –
including assessment, education, knowledge transfer, etc.
Checklist for Preparing for Site Visits
Category Task When to Start Internal Deadline NSF Deadline
ADMINISTRATIVE
Contact information for everyone involved immediately
Designate administrative liaison immediately asap 6 weeks
Create budget for site visit and prep immediately within 1-2 days
Budget approval and spending account 3 months before within a week
Decide which materials should go in binders 2 months before within 2 weeks
Order office supplies for binders 1.5 months before
1 month before
Coordinate with printer (deadline for binders) 1 month before 2 weeks before
Dinner reservations for review team 1 month before 3 weeks before
Prepare info sheets on review team 2 weeks before 1 week before
Name badges for mock site visit 1 week before mock site visit
Food cost for review team 3 weeks before 2 weeks before ~1 week
Name badges for site visit 1 week before site visit
Assemble slides and prepare for printing 2 weeks before 1 week before
Take individual and group photos of project team
3-4 weeks before
1 week before
Put together binders 1 week before 2 days
Travel reimbursements for mock site visit when received within 2 weeks
Travel reimbursements for site visit when received within 2 weeks
TRAVEL AND IN-PERSON PRACTICES
Schedule Site Visit based on project team, Administration, and NSF Availability
immediately
Schedule presentation practice (based on availability of participants)
immediately within a week
Schedule mock site visit immediately within a week
Reserve rooms for site visit immediately within a week
Reserve rooms for mock site visit immediately within 2 weeks
Reserve rooms for practices as soon as dates are known
at least 2 weeks prior
Designate a person to time the presentations at the site visit
2 months before 3 weeks before
Reserve hotel rooms for mock site visit 2 months before mock
2 months before mock
Reserve hotel rooms for site visit 3 months before 3 months before 6 weeks
Reserve meeting room at hotel 3 months before 3 months before 6 weeks
Reserve transportation for review team 2-3 months before
2 months before 6 weeks
Travel information for mock and actual site visit 2 months before 6 weeks before 6 weeks
Travel reimbursement procedures 1 month before mock
3 weeks before mock
Catering for Mock site visit (know your policy) 1 month before mock
3 weeks before mock
Catering for site visit (know your policy) 2 months before 1 month before
Reserve furniture (tables, chairs, etc.) 2 months before 1 month before
Update all files on website 3 weeks before keep up to date 2 weeks before
TECHNOLOGY
Logos and presentation templates 3 months before 3 weeks prior to mock
Dropbox (or similar) for all involved (offline copy)
3 months before asap
Access to a speakerphone, phone lines, slide remote, laser pointer, etc.
3 months before a week prior to first practice
Designate a laptop for presentations 3 months before a week prior to first practice
Dedicated webinar and teleconference service 3 months before a week prior to first practice
Finalize agenda for NSF 1 month before prior to NSF deadline
3 weeks
Website – password protected 2 months before 1 month before 2 weeks
Wireless passwords (if needed) 1 month before 2 weeks before
Media rental (projector, laptops, power, etc) 1 month before 2 weeks before
Practice Schedule for UC Davis
Notification: June 28
Preliminary meeting of leadership with each Science, Ed, KT team: Weeks of July 2-13
Preliminary meeting with campus executives: July 16
Site visit date determined: July 16
Second meeting of Science, Ed, KT teams: Weeks of July 18-25
Logo and slide master finalized: Aug 3
First draft of presentations due to leadership: Aug 7
Second meeting with campus executives to discuss institutional match: Aug 13
Written response to reviewers: August 20
Confirmation from Mock Site Visit reviewers: Aug 22
Second draft of presentations due to leadership: Aug 27
Practice runs within Science, Ed, KT teams: Weeks of Aug 27-Sep 6
Third meeting with campus executives to discuss institutional match: Sept 4
Mock visit #1: Sept 7
PI travel to east coast partner institutions to meet with leaders and administration: Sep 8-12
All binder materials finalized except for slide presentations: Sep 12
Mock visit #2: Sept 21
Binders finalized: Sep 26
Dress Rehearsal: Sept 29
Site Visit: Oct 1-2
Practice Schedule of UC Santa Barbara Entire Center: October 5 and 6 at UCSB (on-site, in person): red team reviewers attending October 24 at UCSB (on-site, in person) Rationale and Goals, Theme 1, Theme 2 and Theme 3, Facilities: Phone conferences, plan for all day Dates: August 31 and September 19 Education, Outreach and Diversity: Phone conferences, plan for all day Dates: September 4, September 20 Knowledge Transfer, Industry Programs, Management, Evaluation: Phone conferences, plan for all day Dates: Dates: September 7, September 21
Person 1
Person 2
Person 3
Person 4
Person 5
Person 6
Person 7
Person 8
Person 9
Person 10
Person 11
Person 12
Person 13