dCache/SRM @ OSG

Abhishek Rana

Frank Würthwein

UCSD

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

OSG@UCSD 12/16/04

Frank Würthwein 2

SRM-dCache @ fnal

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

40GB/hourCMS write/hour

this week

CDF reads/daythis year

20TB/day

OSG@UCSD 12/16/04

Frank Würthwein 3

CDF Center @ FNAL

For comparison:Tier-1 requirement as Specified in CMS Computing model is800MB/sec

A day in 2003

Typical month in 2004

900MB/s

OSG@UCSD 12/16/04

Frank Würthwein 4

SRM-dCache scalability

• IO scalability is essentially infinite.– Depends primarily on # of pools and IO per pool.

• Scalability limits lie elsewhere:– Large rate of client requests

• Tight loops that try to open many files will bring system into its knees.

– Large rate of metadata accesses• Limit # of files per directory• Do not allow user access to virtual filesystem

Applications need to avoid “stupid” behavior!

OSG@UCSD 12/16/04

Frank Würthwein 5

The “geek gap”

• SRM-dCache is as complex as it is powerful.– Requires well packaged “cookie cutter”

configurations to minimize knowledge required from site admins.

– Requires some thought about hardware:• How much infrastructure hardware per cluster?• Choose filesystem that doesn’t fragment.• Choose linux kernel that isn’t buggy.• Some performance tuning (p2p, cost function, “quotas”)

OSG@UCSD 12/16/04

Frank Würthwein 6

Deployed flavors

• As cache for mass storage (HPSS,Enstore,OSM, …)

• Virtualizing space on RAID servers– replication only for load balancing

• Virtualizing compute node space– N copies of every file; “lazy” replication– virtual RAID in addition to load balancing.

OSG@UCSD 12/16/04

Frank Würthwein 7

FNAL plans

• Space reservation & quotas

• authorization module

• MIS

• Integration, deployment, operations

OSG@UCSD 12/16/04

Frank Würthwein 8

Reservations & Quotas

• Quota exists as a hard partitioning of pools and types of files that may be loaded into those pools.

• SRM v2 type reservations are being retrofitted into SRM v1.1 by 3/2005.

OSG@UCSD 12/16/04

Frank Würthwein 9

Security & ACLs

• Cron based synch of dcache & VOMS exists today.

• Authorization module with callout to GUMS by 3/2005

OSG@UCSD 12/16/04

Frank Würthwein 10

MIS

• Interface to GRIS & GIIS works

• Plan of Action being developed by 2/2005.– Internal accounting schema must allow

efficient queries.– Must allow VO specific queries– Send info to MonALISA

OSG@UCSD 12/16/04

Frank Würthwein 11

Integration, deployment, operations

• ~1/2005: Deploy resilient dCache at CMS sites & start operations challenge from Tier-1 to Tier-2. (UCSD will allow some VOs on part of its cluster)

• ~3/2005: Re-deploy new version incl. reservations, security, MIS enhancements.

OSG@UCSD 12/16/04

Frank Würthwein 12

Concerns by fkw

• Application’s use of SRM-dCache– dCache specific scalability issues– “Expectation management”

• Devil’s in the details– Personally, I would proceed well planned &

carefully integrate a few sites at a time rather than a large free for all.