HEPiX, CASPUR, April 3-7, 2006 – Steve McDonald

Steven McDonaldTRIUMF Network & Computing Services

steven.mcdonald@triumf.ca

Canada’s National Laboratory for sub-atomic physics

• Located in Vancouver, Canada (west coast)

• ~ 400 Full Time Staff & Physicists

• 2nd furthest T1 site from CERN ~168ms RRT

HEPiX, CASPUR, April 3-7, 2006 – Steve McDonald

* *******

*

*

**

Centre - Country ALICE ATLAS CMS LHCb MB/s

ASCC - Taipei X X 110

CNAF - Italy X X X X 220

PIC - Spain X X X 200

CC-IN2P3 - France X X X X 220

FZK - Germany X X X X 220

RAL - England X X X X 220

BNL –United States X 154

FNAL – United States X 50

TRIUMF - Canada X 75

NIKHEF - Netherlands X X X 175

NORDIC GRID X X X 90

# TIER1 Sites 6 10 7 7 1,600

TRIUMF

Assumes all ATLAS T1 are equal

HEPiX, CASPUR, April 3-7, 2006 – Steve McDonald

TRIUMF-CERN 1GbE Lightpath(s)

• TRIUMF• BCNET• CANARIE• SURFnet• CERN

• 1st dedicated GbE circuit • established April 18th 2005

• 2nd dedicated GbE circuit established July 19th 2005

HEPiX, CASPUR, April 3-7, 2006 – Steve McDonald

SC3 throughput results over2 x 1G lightpaths

TRIUMF 90-140 MB/s is typical

HEPiX, CASPUR, April 3-7, 2006 – Steve McDonald

Sept 1 ‘05

3rd week Sept ‘05

End Oct ‘05

Amsterdam

HEPiX, CASPUR, April 3-7, 2006 – Steve McDonald

CWDM & Lightpaths at TRIUMF

/CA*net4

10 GbE @1550nm - CERN T0

2x 1 GbE @1530nm - Canada T2’s2x 1 GbE @1510nm - ? T2 & T12x 1 GbE @1690nm - CAnet4 research

Aug 2002

TRIUMF Local loop to Gigapop & CA*net4 PoP ~ 22km

HEPiX, CASPUR, April 3-7, 2006 – Steve McDonald

TRIUMF WAN CWDM Equipment

BCNET 22km

10GbEFoundry Switch -> CERN

MRV CWDM

1610 nm1590 nm1570 nm1550 nm

Potential to Add 2 more1GbE channels

Single Pair Fiber4 1GbE channelsPassport 8600• ORAN• WESTGRID• 2x CERN

SFP4 Port

Optical Mux2x GbE

TDM

HEPiX, CASPUR, April 3-7, 2006 – Steve McDonald

ATLAS Tier2 in Canada

TRIUMFTier 1

EasternTier2

CERN TIER 0

10G

1G

Dedicated e2e LightpathsCan be lightpathsor on the routed 1Gbps CA*net4research network

~ ?SARA/NIKHEF

Tier 1

There will be two Tier 2’s Centres in Canada, one in the East and one in the West. Each centre will be a distributed across several sites, with one contact point

WesternTier2

1G

Tier2’s

HEPiX, CASPUR, April 3-7, 2006 – Steve McDonald

TRIUMF Tier1 → Canada Tier2’s

Participating Canadian Tier2 sites in SC3Western Tier2’s

Simon Fraser UniversityUniversity of AlbertaUniversity of Victoria

Eastern Tier2’sUniversity of Toronto

T1-T2 network activity Aug 17 - Aug 24 : All four Tier 2 sites participating

Already linked via a $40MDistributed Computing GridInfrastructure

Plans are underway to includeVictoria in an expanded grid

HEPiX, CASPUR, April 3-7, 2006 – Steve McDonald

Transfer results over 10GbE circuit

HEPiX, CASPUR, April 3-7, 2006 – Steve McDonald

HEPiX, CASPUR, April 3-7, 2006 – Steve McDonald

10 GbE Lightpath to CERN Summary

TRIUMF CERN

Atlantic Crossing

√ √√ √ √√

X

X

√

HEPiX, CASPUR, April 3-7, 2006 – Steve McDonald

ATLAS Tier1 resources distribution

Tier 1 Center % of pp Resources

ASGC 7.7

BNL 24

CNAF 7.5

FZK 10.5

IN2P3 13.5

PIC 5.5

RAL 7.5

TRIUMF 5.3

NDGF 5.5

ATLAS T1-T1 Peering

FZK & IN2P3 BNL

CNAF RAL

NIKEF/SARA TRIUMF & ASGC

NDGF PIC

TRIUMF ~ .6T1 original networkingspecification called for ~75 MB/sec per ATLAS T1 with x6 safety factor implies3.6 Gbps

For TRIUMF T1 ~2-3 Gbps

Consider 10G LAN Phy/SONET mappingTransatlantic 10G link

HEPiX, CASPUR, April 3-7, 2006 – Steve McDonald

1G HEPnet daisy chainrouted

TRIUMF partitions APN and establishes cross connects with 3rd parties APNs

UoToronto PhysicsTier 2

UoVictoria PhysicsTier 2

TRIUMFTier 1

CERNTier 0

Amsterdam

New York

Chicago

Toronto

Vancouver

Victoria

FERMITier 1 Brookhaven

Tier 1

UBC Physics

UA Physics UoT

Physics

Carleton Physics UdM

Physics

CA*net 4

Edmonton

Ottawa

To other physics users at smaller universities Geneav

CWDMCWDM

1G Tier 2 East

Optionalinterfaces

Note: Typical View on TRIUMF UCLP GUI

1G Tier 2 West

5G Tier 0 data

2G Tier 1 data