Presented by: Tony Rimovsky (tony@ncsa.uiuc.edu)

TeraGrid Account Management

Tony Rimovsky, Area Director for Network Operations and Security

tony@ncsa.uiuc.edu

Steven Quinn, Lead for Teragrid Accounting and Account management

squinn@ncsa.uiuc.edu

Presented by: Tony Rimovsky (tony@ncsa.uiuc.edu)

Teragrid Accounting and Account Mgmt

• Teragrid Accounting and Account management– Uses AMIE as a component– Manages account/user information, allocations,

usage– Designed to give sites as much freedom as possible

•Variety of accounting methods •Variety of account management systems•Everyone has slightly different ways they want to do things

– Accommodates NSF Resource Allocation system•Quarterly Allocation meetings

–Large, Medium and Development allocations–Uses POPS system for applying for allocations

Presented by: Tony Rimovsky (tony@ncsa.uiuc.edu)

AM Information Management

• AMIE: Account Management Information Exchange System

•Methods and Software developed under NMI•Uses GGF URWG (Usage Record Working Group) specifications

•AMIE deals with:–Resource allocations

–User information

»Name; Address; DN; etc. –Project/group information

–Usage reports

•Contacts: »Steve Quinn (NCSA): squinn@ncsa.uiuc.edu »Michael Shapiro (NCSA): mshapiro@ncsa.uiuc.edu »http://scv.bu.edu/AMIE/

Presented by: Tony Rimovsky (tony@ncsa.uiuc.edu)

AMIE Architecture

– Transaction-based exchange mechanism•Transaction comprised of sequence of packets (messages) and acknowledgements

– TGCDB sends Requests– Sites send Notifications

Presented by: Tony Rimovsky (tony@ncsa.uiuc.edu)

AMIE Data Representation

• XML schema for Accounting Objects– Machines– Users– Accounts– Allocations– Usage

• AMIE speaks xml, but sites interface to AMIE through AMIE DB

Presented by: Tony Rimovsky (tony@ncsa.uiuc.edu)

TGCDB

AMIE Local DB

Site Accounting

AMIE Transaction

AMIE Local DB

RP Site TGCDB “site”

RP Site (N, N+1, …)

Site and TG independent software process

TG Accounting and Acct Mgmt System

Presented by: Tony Rimovsky (tony@ncsa.uiuc.edu)

TG Accounting and Acct Mgmt SystemLocal Database detail

PBS Shim API Layer

Site written xlation layer

Database

AMIE module

DB to XML

TGCDB “site”

Site Accounting

AM

IE L

oca

l D

ata

base

Presented by: Tony Rimovsky (tony@ncsa.uiuc.edu)

TG Strategy

• Centralized TG AM DB (TGCDB)• Sites' AM Systems remain “essentially”

unchanged• Sites react to AM requests from TGCDB• Sites notify TGCDB of AM actions taken

– Push site project/account/person/usage info to TGCDB

Presented by: Tony Rimovsky (tony@ncsa.uiuc.edu)

Site Prerequisites

• Need basic Accounting/Account Management system– Existing system– New, locally developed system– Packaged Accounting/Account Management system (QBank,

Gold, etc.) configured for local requirements

• Ability to track accounting objects:– projects/accounts/usage/etc.

• Ability to track/record project usage

Presented by: Tony Rimovsky (tony@ncsa.uiuc.edu)

Site Prerequisites (cont.)

•Mechanism to validate project specification at job submission time– Is this a valid project?– Can this user bill against it?– Need to tie projects globally and locally.

– PROJECTS in concept are necessary with Allocation based accounting•Multiple “awards” under a PI•Accounting done per project, NOT per user or PI.

Presented by: Tony Rimovsky (tony@ncsa.uiuc.edu)

Making it work within TG

• Install/Configure AMIE• “Hook” AMIE to local AM system (site responsibility)

– Need to push local project/person/login info to TGCDB– Need to reconcile incoming info against local AM info (“do

we already know about this person in a different context?”)– Some sort of RDBMS needed for local AMIE DB (PostgreSQL

preferred)•Sites read local AMIE DB for TGCDB AM requests•Sites write notifications into AMIE DB. •AMIE interacts with local/remote AMIE DB

Presented by: Tony Rimovsky (tony@ncsa.uiuc.edu)

Making it work within TG (cont.)

• “Hook” AMIE to local AM system (cont.)– AMIE software provided to work with local AMIE

DB •Provides API to read packets from and write packets to the local AMIE DB

• Map TG Project IDs to local project ids– Sites need to accommodate users specifying

TG-wide project ID at job submission time– A “TGProjects” module provided to handle local

site project ID to TG-wide project ID mapping•Works with /etc/tg-projects file, which contains local to TG-wide mapping

Presented by: Tony Rimovsky (tony@ncsa.uiuc.edu)

Making it work within TG (cont.)

•TGProjects module (cont.)•Should be called at job submission time to substitute for user-provided TG-wide project ID

•“qsub wrapper” provided for convenience

•Install/configure TGAccounting •Install/configure various tools (usage; DN management; etc.)

Presented by: Tony Rimovsky (tony@ncsa.uiuc.edu)

Non-cycle based accounting

•The TGCDB and AMIE were both designed to accommodate arbitrary types of resource usage– Both systems support arbitrarily defined “billing

units”– No assumptions inherent about the type of

allocations or usage

•Non-cycle based accounting is more of a policy issue than an implementation issue– How will resources be allocated and charged?– What controls will resources use to react to

usage?

Presented by: Tony Rimovsky (tony@ncsa.uiuc.edu)

Interoperability

•AMIE is designed to work both inter- as well as intra-grid.

•Data can cross administrative domains– This includes both usage and account records– Coordination of identifiers is important

•Another area where policy will be more work than implementation

Presented by: Tony Rimovsky (tony@ncsa.uiuc.edu)

Allocation Model

•NSF/Teragrid/HPC allocation model allows for project based allocations on one or more resources

•Fungible allocations are supposed by allocating on “virtual resources” (ie. grids)

•Teragrid tracks usage at the job level.– AMIE packets include

•User/project/resource identifiers•Submit time•Wall time•Number of nodes•Etc.