TeraGrid Allocations Discussion

John TownsDirector, Persistent Infrastructure

National Center for Supercomputing Applications

University of Illinois

Please help us with the following questions:

•How well does the current TeraGrid allocation process serve the needs of both the research community and the providers of TeraGrid resources and services?

•Does the SAB have any advice on how this process might evolve as both the nature of research problems and the nature of TeraGrid resources and services evolve?

Overview of Allocations Process (I)

•Who Can Receive Allocations?– any Ph.D.-level

researcher (or NSF-eligible PI) from a U.S. academic or non-profit research institution

– NSF Graduate Research Fellows and Honorable Mention awardees

– any discipline, any institution, any funding agency support

•What Can Be Requested?– three request sizes

•Small, Medium, or Large

– more than 20 resources•Computational•Data Storage•Visualization•Advanced User Support

– no cost to PIs

Overview of Allocations Process (II)

•When Are Requests Accepted?– Small requests:

Anytime•30,000 CPU-hours•5 TB disk•25 TB tape

– Medium requests: Quarterly•500,000 CPU-hours•25 TB disk•100 TB tape

– Large requests: Semi-annually

•Where Do You Apply? – POPS

•https://pops-submit.teragrid.org/

Overview of Allocations Process (III)

•How Do You Get Started?– for Small (start-up) requests, just need an

abstract of the work and a CV for the PI•allow 2 weeks to get everything set up.

– for Medium or Large requests, a proposal is reviewed by independent committee at quarterly meetings.

Medium or Large Allocations

•PIs need to be aware of the lead time for getting an MRAC or LRAC award– Requires a written proposal– Reviewed by domain experts

•LRAC (Large)– Reviewed semi-annually– Awards begin April 1,

Oct. 1

•MRAC (Medium)– Reviewed quarterly– Awards begin Jan. 1,

April 1, July 1, Oct. 1

Using Your Allocation

•New User Packet– provides your

TeraGrid-wide and sSite-specific logins

•TeraGrid User Portal– online hub for

monitoring and using your allocation

– portal.teragrid.org•guests welcome

•User Support– help@teragrid.org

Allocations Policies

•High-level description of the nuggets of the policy•http://www.teragrid.org/userinfo/access/allocationspolicy.php

– PI eligibility– proposal format, page length, etc– review process; review panel– review criteria– award management

•Who sets them and how are they evolved?– ad hoc team to develop recommendations and comments

•John Towns, Nancy Wilkin-Diehr, Ralph Roskies, Dave Hart, et.al.– community input to gain consensus

•PIs, xRAC Panel, NSF Program Officers, RP representatives, etc.– has evolved policies:

•community account/science gateways•storage•support services•what’s next?

Specific Issues (I)

• How does the allocation panel review take into account the review of scientific merit and broader impacts of the proposed work already done by the NSF? How does “Double jeopardy” with separate financial and resource allocation proposals get handled?– policy

•if review of science has been done, it is NOT the purview of xRAC Panel •panel only does this in cases of requests not supported financially from grants that result for proposals that have had such review

– practice of enforcement•remanded at start of every meeting; reigned in during panel reviews

• How does the allocation process take into account the science impact, if at all, of the proposed TG usage? – Actually, that is not really the specific purview of the review panel,

though they do consider it if there is ambiguity amongst the panel on how to handle a request. This gets at the review of the science, which is typically NOT what the xRAC reviews.

– Question for SAB - Should this factor be given more or less (or any) emphasis?

Specific Issues (II)

•How are the allocations reviewers selected? What process or criterion is used to ensure that they have adequate expertise? What term limits (if any) should be applied to their service?– typically look for recommendations of outgoing reviewers– also look for recommendations of NPS program officers– try to assess the work they do from on-line information– do not conduct a formal review process of potential reviewers– from policy document:

•“The committees consist of volunteers who are selected from the faculty and staff of U.S. universities, laboratories and other research institutions. All of the committee members have expertise in some area of computational science or engineering and serve a term of 2-5 years.”

– typically try to keep most reviewer to 2-3 years. – some kept 4-5 years for various reasons (reviewer availability,

quality of reviews, etc.)

Proposal Counts

• Typically 3-6 reviewers per proposal– larger number for really big LRAC requests

• Typically ~35 members of review panel• High reviewer load:

– some reviewers will have 15-20 proposals when MRAC and LRAC coincide

0

20

40

60

80

100

120

Sep-05 Dec-05 Mar-06 Jun-06 Sep-06 Dec-06 Mar-07 Jun-07 Sep-07 Dec-07

# o

f P

rop

os

als

MRAC

LRAC

Specific Issues (III)

•Who should be serving as panel discussion chair at the allocations committee meetings, and what guidelines are in place for the conduct/role of this chairperson?– in practice, selected by Allocations Officers from

panelists– guidance given to chair by Allocations Officers– Invite input from SAB!!

Specific Issues (IV)

•What role, if any, should NSF program directors funding projects in large-scale computation have in the allocations process?– in practice have only been observers in the

process and provided responses on NSF-level policy issues

Specific Issues (V)

•Are the conflict-of-interest policies applied to allocation panelists more or less stringent than those used for NSF panels?– COI policy

•generally follow NSF COI rules –collaborators, advisors, etc.–institutional conflicts

•difference: proposers may be on panel that reviews their proposal

•conflicted reviewers are dismissed from room during discussions involving conflicts

– Question for SAB - Should they be?

Specific Issues (VI)

•How are reviewers' areas of expertise matched to the allocation requests? – reviewers provide list of primary and secondary

FOS– PIs indicate primary and secondary FOS of

proposal– provides first-order matching– Allocations Officers review and address

unassigned proposals

– Question for SAB - Is this the most effective strategy?

Specific Issues (VII)

•How does the allocation process enable individual projects to have dedicated access to large fractions of a given system for significant periods of time? How do TG management practices and metrics likewise enable such partitioning of the systems? – PIs can specifically request– ask PIs to provide 1 one page additional description

•additional information regarding this needs; allow RPs to prepare

– fundamentally, availability is controlled by the RP; nothing specifically requires this•is seen as being in RP’s own best interest to provide •requested by community and typically well justified.

– Question for SAB - Are these sufficient?

Specific Issues (VIII)

•How will the allocation process take into account the jumps in available resources and system size when each Track 2 system and, eventually, the Track 1 system come online? Do the Track 2 and Track 1 systems present any particular challenges to the allocation process?– addressing this and related issues within the TG

Extreme Scalability RAT– fundamentally, current process is sufficient– NSF has initiated “PRAC” solicitation and this will

impact process for Track 1