petabyte storage facility at rhic razvan popescu - brookhaven national laboratory
out of 23
Post on 22-Dec-2015
Embed Size (px)
- Slide 1
- PetaByte Storage Facility at RHIC Razvan Popescu - Brookhaven National Laboratory
- Slide 2
- CHEP 2000 -- PadovaPetaByte Storage Facility at RHIC2 Who are we? t Relativistic Heavy-Ion Collider @ BNL Four experiments: Phenix, Star, Phobos, Brahms. 1.5PB per year. ~500MB/sec. >20,000SpecInt95. t Startup in May 2000 at 50% capacity and ramp up to nominal parameters in 1 year.
- Slide 3
- CHEP 2000 -- PadovaPetaByte Storage Facility at RHIC3 Overview t Data Types: Raw: very large volume (1.2PB/yr.), average bandwidth (50MB/s). DST: average volume (500TB), large bandwidth (200MB/s). mDST: low volume (
- CHEP 2000 -- PadovaPetaByte Storage Facility at RHIC16 HPSS Performance t 80 MB/sec for the disk subsystem. t ~1 CPU per 40MB/sec for TCPIP Gbit traffic @ 1500MTU or 90MB/sec @ 9000MTU t >9MB/sec per SD-3 transport. t ~10MB/sec per 9840 transport.
- Slide 17
- CHEP 2000 -- PadovaPetaByte Storage Facility at RHIC17 I/O Intensive Systems t Mining and Analysis systems. t High I/O & moderate CPU usage. t To avoid large network traffic merge file servers with HPSS movers: Major problem with HPSS support on non-AIX platforms. Several (Sun) SMP machines or Large (SGI) Modular System.
- Slide 18
- CHEP 2000 -- PadovaPetaByte Storage Facility at RHIC18 Problems t Short lifecycle of the SD-3 heads. ~ 500 hours < 2 months @ average usage. (6 of 10 drives in 10 months). Built a monitoring tool to try to predict transport failure (based of soft error frequency). t Low throughput interface (F/W) for SD-3: high slot consumption. t SD-3 production discontinued?! t 9840 ???
- Slide 19
- CHEP 2000 -- PadovaPetaByte Storage Facility at RHIC19 Issues t Tested the two tape layer hierarchies: Cartridge based migration. Manually scheduled reclaim. t Work with large files. Preferable ~1GB. Tolerable >200MB. Is this true with 9840 tape transports? t Dont think at NFS. Wait for DFS/GPFS? We use exclusively pftp.
- Slide 20
- CHEP 2000 -- PadovaPetaByte Storage Facility at RHIC20 Issues t Guarantee avail. of resources for specific user groups: Separate PVRs & movers. Total exposure to single-mach. failure ! t Reliability: Distribute resources across movers share movers (acceptable?). Inter-mover traffic: 1 CPU per 40MB/sec TCPIP per adapter: Expensive!!!
- Slide 21
- CHEP 2000 -- PadovaPetaByte Storage Facility at RHIC21 Inter-Mover Traffic - Solutions t Affinity. Limited applicability. t Diskless hierarchies (not for DFS/GPFS). Not for SD-3. Not enough tests on 9840. t High performance networking: SP switch. (This is your friend.) IBM only. t Lighter protocol: HIPPI. Expensive hardware. t Multiply attached storage (SAN). Most promising! See STKs talk. Requires HPSS modifications.
- Slide 22
- CHEP 2000 -- PadovaPetaByte Storage Facility at RHIC22 Summary t HPSS works for us. t Buy an SP2 and the SP switch. Simplified admin. Fast interconnect. Ready for GPFS. t Keep an eye on the STKs SAN/RAIT. t Avoid SD-3. (not a risk anymore) t Avoid small file access. At least for the moment.
- Slide 23
- Thank you! Razvan Popescu firstname.lastname@example.org
View more >