![Page 1: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/1.jpg)
National Backbone Extension:Rolling out Multisite Gigabit connections over
Fibre Optics
Brian Boyle & Mike Norris
![Page 2: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/2.jpg)
Topics1. NBE – in context2. Aims of the project
3. What has been achieved
4. Under the bonnet
5. The bigger picture
6. Summary
7. Next steps
8. Discussion2
![Page 3: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/3.jpg)
NBE – in context• Building on the NBN• New requirements – hybrid networking• New paradigm of service and support• Proposal and resources• Support from stakeholders and clients
3
![Page 4: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/4.jpg)
Bandwidth
BA
C
Users
2Mbit/s Gigabit
Bandwidth Use
![Page 5: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/5.jpg)
HPC facility
ICHEC traffic
ICHEC latency
![Page 6: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/6.jpg)
General Internet
Daily traffic
Yearly traffic
![Page 7: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/7.jpg)
NBE – the project• Project plan and management• Inventory of sites• Criteria, priorities• Surveys and schedules• Procurement, multivendor scenario
7
![Page 8: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/8.jpg)
Project organisationJohn Boland CEO Project sponsor
Steering group Victor Reijs Mike NorrisJohn Boland
Eoin KennyProject Manger
WG4Network
management/provisionCormac/Daniel
Out sourcing company Brian N
Out of Band
WG5Config tool
Cillian Sharkey
WG3Network Design
Eoin KennyDaniel LeteOwen Byrne
WG6 PoP Planning
StephenColm
WG2Fibre procurement
Owen ByrneLouis TwomeyPaul Cosgrove
WG1Client LiasonMike NorrisBrian Boyle
WG0Project management
Eoin Kenny
WG7DWDM Network
SiemensEoin
![Page 9: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/9.jpg)
Project goals
• A network for connecting clients, with the following features
• High bandwidth• Multi-service• Resilient• Scaleable• Flexible• Secure
9
![Page 10: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/10.jpg)
Goals – Multi-service
– Service orientated connectivity, e.g:• Private secure connection per Institute to An Chéim• Interconnectivity with the Government VPN
– Not just focused on internet reachability– Capable of delivering access to multiple services using
the same infrastructure– Advanced networking protocols supported – IPv6,
multicast
10
![Page 11: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/11.jpg)
Goals - Resilience
• Seek to build in circuit, equipment, path, and routing diversity
• Equivalent service levels on primary and secondary• Using multiple ducts, termination points, equipment• Using a robust routing technology (BGP) • Can make use of existing connectivity via other service
providers
11
![Page 12: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/12.jpg)
Goals - Scalability• Can deliver multiple 1 Gigabit/s circuits• Can operate at line speed end to end• Capable of offering even higher speeds subject to
requirements (10 Gbps, higher)• 10 Gbit/s access for clients being explored• Technology refresh available for 40 Gbit/s…• …and similar for 100 Gbps in development
12
![Page 13: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/13.jpg)
Goals - Flexibility• Not subject to telco business case scenarios• Not tied by contract to any particular network
implementation technology or model• Can use a mix of fibre and traditional managed Ethernet
circuits• Fully compatible with existing networks (no big bang
changes)• Capable of delivering wavelengths to clients if the need is
proven
13
![Page 14: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/14.jpg)
Client perspective• Client priorities are key throughout• Aim is to keep change to client sites minimal• HEAnet installs managed routers and switches in
addition to current network equipment• When ready, services can be tested and migrated
individually• No Big Bang change• No change without explicit client agreement
14
![Page 15: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/15.jpg)
Typical installation• Research of available providers (fibre, managed circuits),
usually via Tender process• Client site survey by provider with client
Paths identified, leading to “way leave” contracts• Civil works commence• Fibres are pulled into building, and subsequently spliced• Routers and switches are delivered, configured by HEAnet• Services can be tested and used by client
15
![Page 16: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/16.jpg)
New link services• Point-to-point Ethernet links (p2p)
– 1 Gbit/s, Ethernet framed– Ethernet Port Service– Ethernet VLAN Service– No aggregation– Automatic monitoring/graphing
16
![Page 17: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/17.jpg)
Ethernet Port and VLAN services
![Page 18: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/18.jpg)
Some examples• Services
- connectivity- institutional access and security- p2p
• Applications- Oireachtas- ICHEC- An Chéim- UCLP
• Client examples- DCU
18
![Page 19: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/19.jpg)
LAN Extensions – Ryan Academy• Centralise services, both voice and data• Servers centrally located at DCU• IP Telephony Extension to Nortel Meridian• Seamless telephony integration• Security cameras and DVR remote view and monitoring• Buildings Management System• High-End Video Conferencing > 2 Meg/sec• Reliability• No onsite technical support ( low maintenance )
19
![Page 20: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/20.jpg)
LAN Extensions – Mater Dei Institute
• Sister College of DCU• All Servers consolidated in DCU• Previous difficulties with server location in MD
( Dust and no Aircon )• Minimal on-site technical support
( mainly helpdesk 2/3 days per week )• No requirement for Sysadmin• Reliability of service
20
![Page 21: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/21.jpg)
Under the bonnet• DWDM, CWDM• Ethernet switches• MPLS• New routing strategies• Out of band access for network management• CPE, routers• Outsourcing of L2 ops management• Provisioning tool• Contributions, SLA
21
![Page 22: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/22.jpg)
Multiple sources
• We make use of many fibre providers to connect at the lowest cost for our requirements– E-net– Dublin City Council– ESBT– BT Ireland– NTL– Aurora
22
![Page 23: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/23.jpg)
Fibre network end 2006 – approx 2300km of fibre
![Page 24: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/24.jpg)
MAN design13/01/06
National Backbone(10G)
Core Ring(10G)
1490nm
Galway PoP
Clinical Science(UCHG)
1490nm 1530nm 1610nm
1610nm1590nm1570nm
1570nm
1550nm1530nm1510nm
1510nm
1470nm
1470nm
mux/demux
mux/demux
Templates
Galway Regional λ Planning { } everything between these will form part of the future MAN ring. Currently only a point-to-point between NUIG and the Clinical Science building exists. This will later form part of the proposed ring.· 1470nm will be used initially to provide a 1 Gbit/s point-to-point link between UCHG and NUIG. This
will provide a resilient connection when the MAN ring is complete.· 1490nm will be used to provide a 1 Gbit/s resilient link between GMIT and NUIG (upon ring closure)· 1510nm will be used to provide a 1 Gbit/s resilient link between GMIT and NUIG (upon ring closure)· 10G resilient link between NUIG and ESB Telecoms aggregating n * 1Gbit/s from Access Ring· 1530nm will be used to provide a 1 Gbit/s resilient link between NUIG and NUIG’s Research Institute
(upon ring closure)
1550nm
GMIT
1590nm
1490nm
3750 Switch
mux/demuxmux/demux
AdvaFSP
ESBT PoP
Cisco7606
3750 Stack
MAN Ring(n * 1Gbit/s)
10G
1Gig TX
Regional
National
CAO
3750 Switch
1510nm
NUIGResearch Institute
3750 Switch
1530nm
3750 Switch
1470nm
![Page 25: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/25.jpg)
Transmission technologies
• We use a heterogeneous mix of technologies to achieve our goals:– CWDM in metropolitan areas gives us multiplexed fibre
services for short hops– DWDM between regions to give the same for longer hops– MPLS network to give multi-service offering– Multiple physical connections to add resiliency
25
![Page 26: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/26.jpg)
HEAnet backbone network topology
![Page 27: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/27.jpg)
How does it do it?• Circuits are managed using an automated
provisioning system• Based on client requirements, the system is
used to configure routers, service monitoring portal
• Aim to have client use the tool directly, with administrator later checking/approving circuit change requests
27
![Page 28: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/28.jpg)
Network provisioning tool
![Page 29: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/29.jpg)
The bigger picture
• NIBEST – NW link with NIRAN• UKERNA DF – SuperJANET 5 links• GÉANT – hybrid networking• Transit – preparing for the next step• INEX – peerings and more
29
![Page 30: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/30.jpg)
INEX traffic growth
• HEAnet-INEX (IPv4)
• Total INEX
![Page 31: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/31.jpg)
![Page 32: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/32.jpg)
IP bandwidth
1
10
100
1,000
10,000
100,000
IP b
andw
idth
(Mbp
s)
Access b/w
External b/w
![Page 33: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/33.jpg)
IP bandwidth
0
5,000
10,000
15,000
20,000
25,000
30,000
35,000
40,000
45,000
IP b
andw
idth
(Mbp
s)
Access b/w
External b/w
![Page 34: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/34.jpg)
Summary of changesBefore After
•Network consists of managed links connecting regional POPs•Clients used managed links to connect to PoPs•Service monitored by HEAnet (IP) and telco providers (physical)
•More PoPs, connected by Dark Fibre•Clients used Dark Fibre and/or managed links to connect to PoPs•Service monitored by HEAnet (IP, Ethernet) and Fibre providers (physical)
![Page 35: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/35.jpg)
Benefits
• Improved bandwidth• Guaranteed uncontended bandwidth service
levels• Reduced cost• Flexibility• Improved access to HEAnet services
35
![Page 36: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/36.jpg)
Outline of servicesNETWORK
MANAGEMENTHOSTING
SERVICESMULTIMEDIA
APPLICATIONS
Network Operations Centre, Helpdesk
Co-location - equipment hosting
Hi-quality IP Video-conferencing
Up-to-minute traffic and performance
Web hosting, Database hosting, Email, Domains...
Video & audio streaming over
Internet
Spam Control
tools, MAPS
Anti-virus trends
Telco liaison, fault resolution and client
support
Hot Standby, failover website facility
High quality streamed content
(e.g. Oireachtas Live, IPTV)
Public Key Infrastructure
Authenticate Authorise
Accounting
Performance and quality maintenance
Mirroring, Multimedia content central
storage
Schools connectivity to backbone
SECURITY
Computer Emergency Response Team
Security Scanning Service
![Page 37: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/37.jpg)
Phase 2 progress, 2nd Nov
Confirmed 7 Cancelled 4 In progress 17 Completed 13
Total number of links 43
37
![Page 38: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/38.jpg)
Dark fibre statistics
• No. of DF providers 5• Aggregate length of DF pair 2,300 km• No. of primary client sites
with DF links 18 (of 40)• Total sites with DF connections 31
38
![Page 39: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/39.jpg)
Next steps
• New model, service levels• Phase 3• Client service reviews• Review of research infrastructures• Integrated e-infrastructures
39
![Page 40: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/40.jpg)
Questions?
![Page 41: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/41.jpg)
Project history• In 2004, we commenced implementation of dark
fibre point to point links• Followed by additional of CWDM and DWDM
equipment to increase service possibilities• Commenced work on an automated provisioning
tool to create and manage configurations• Quickly grew into a heterogeneous network
combining a mix of technologies and providers
41
![Page 42: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/42.jpg)
Scalable capacity• Backbone – 10 Gbit/s• Can add more 10 Gbit/s using existing
infrastructure• Client connections at 1 Gbit/s
42
![Page 43: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/43.jpg)
What has been achieved
• High levels of service to clients- high bandwidth- resilience – path, equipment, service
• Scalability now and in the future• Underpins e-infrastructure• Complex and flexible• Rich mix of services• Secure
43
![Page 44: National Backbone Extension: Rolling out Multisite Gigabit connections over Fibre Optics](https://reader036.vdocuments.mx/reader036/viewer/2022062816/5681591d550346895dc64642/html5/thumbnails/44.jpg)
Trends since 1991
1
10
100
1,000
10,000
100,000
1,000,000
Jan-
91
Jan-
93
Jan-
95
Jan-
97
Jan-
99
Jan-
01
Jan-
03
Jan-
05
Valu
e re
lativ
e to
199
1
HEAnet traffic
Access bandwidth
External bandwidth
HEAnet staff