Clinical and TranslationalScience Institute / CTSIat the University of California, San Francisco

Enabling new models of pre-competitive collaboration in proposal development and team creation

Context

The Internet allows for new ways to access a wider set of individuals and enable their contributions to various activities

Explicit

Implicit

Crowdsourcing

Most current approaches fall under the category: Crowdsourcing

Crowdsourcing = Outsourcing tasks or challenges to the broadest possible community, mediated by the Internet

Crowdsourcing to improve proposals

Old

RFP

Review

New?

RFP

Review

Pre-reviews?

Add collaborators?

Improve submission?

Remove redundant submission?

“Open Proposal”

Could we get a wider set of individuals to help improve ideas online and/or join teams they may not have otherwise?

“Open Proposal”

a process and online tool that enables broad collaboration in proposal development and team creation

“Open Proposal”

General Process:

Announce

Request for Proposals

Deadline #1

Submit proposal online in an Open Forum

Deadline #2

Comment, join teams

Revise proposal for final version

Decision

Selected proposals announced

• No new proposals

• Broadest possible community comments on proposals and joins teams

• Broadest possible community submits proposals

• Commenting open • Proposals reviewed internally

SUBMISSION PHASE IMPROVEMENT PHASE REVIEW PHASE

Open Proposal - Example

• Eligibility: All faculty and staff at UCSF

• Campus Coordination: RFP coordinated with UCSFs umbrella intramural funding organization, RAP.

• Promotion: Promoted via RAP and independently through multiple channels

• Review Process: Managed by CTSI

CTSI Pilot Awards to Improve the Conduct of Research (Feb-Apr 2012)

CTSI Pilot Awards: An Open Proposal RFP

Three Phases

Submission Phase: 5 weeks

Improvement Phase: 2 weeks

Internal Review: 2 weeks

Submission (~5 weeks)

• Applicants submit idea online with opportunity to • receive input (via on- and/or off-line commenting)• look for collaborators

• CTSI board members and program directors review and make connections where possible

CTSI Pilot Awards: An Open Proposal RFP

Submission (~5 weeks) cont’d

• Proposal submission = simple, online, “one-page” format including• Rationale• Plan• Criteria and metrics for success• Approximate cost and brief justification • List of collaborators

CTSI Pilot Awards: An Open Proposal RFP

Improvement (2 weeks)• Applicants and community

• browse proposals & comment to improve

• indicate interest to collaborate

• CTSI board members review, comment & solicit input from experts in community

• Applicants

• review comments on their proposals & revise as needed

• required to post online constructive criticism on another proposal at least once

CTSI Pilot Awards: An Open Proposal RFP

Internal Review (2 weeks)• Initial review and scoring of proposals by board – ID’d those most likely

to be funded

• Reached out to applicants with requests for additional information (e.g. budget) and proposed adjustments to proposals (e.g. scope, focus, partnerships)

• Compiled reviewer comments and applicant responses for final review/award identification

CTSI Pilot Awards: An Open Proposal RFP

Open Proposal – Basic Statistics

28 proposals submitted, 8 awarded  • 4 were from CTSI, 4 were not

Total $ awarded is approximately $327K • Award Range: $16,340 to $85,904• Average award: $43,411

Open Proposals – How We Did

Open Proposal – Succeeded in being interactive

Open Proposals – Improved proposals

• Iteration and improvement of proposals based on comments

• Withdrawal: one applicant voluntarily withdrew after learning from a commenter about 2 resources already in existence that fulfilled the goal of the proposed resource!

Open Proposals – How We Did

• Comment

• Author

Open Proposals – What we learned about the process

Need dedicated resource for forum management and monitoring• Encouraging commenting, soliciting input, fielding questions, etc.

More time needed with authors once proposals filtered for those most likely to be funded

Recurring issue – how to match proposers and ideas with the expertise they really need?

Extra slides