technical overview - france-ixa juniper ex9200 and dwdm muxs in stock ready to deploy able to deploy...
TRANSCRIPT
![Page 1: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/1.jpg)
Technical Overview By Simon MUYAL
30th September 2016
![Page 2: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/2.jpg)
Agenda
1
![Page 3: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/3.jpg)
Backbone infrastructure update Network stability and availability Paris and Marseille upgrades
Agenda France-IX Technical update
2
Deployments SLA probes Flow statistics
Next steps for 100G optical transport 100G density on IXP fabric More visibility
2017
![Page 4: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/4.jpg)
Backbone infrastructure update
3
![Page 5: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/5.jpg)
More and more stability Network availability (October 2015 to
September 2016)
Network stability
Marseille = 100%
Paris > 99.99%, one outage
4
![Page 6: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/6.jpg)
Hardware
Network stability
No hardware issues observed
5
![Page 7: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/7.jpg)
Software
No software upgrade needed for Brocade JunOS: 2 upgrades
Network stability
Minor issue related to LAG configuration on 100G interfaces
Major issue for visibility : IPFIX bug à Flow statistics unavailable
6
![Page 8: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/8.jpg)
Decembre 2015 A member started spoofing IPv4 adresses from other members Replying to ARP requests, not destined to its IPv4 address Some members have been impacted by this non voluntary « IP spoofing »
Less than 1hour of outage on the last 12 months
8
![Page 9: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/9.jpg)
Improvements done Additional tests made on quarantine VLAN before moving a port into production (ARP PING scan on LAN IP prefix)
Improvements under consideration
Shutdown « automatically » a member port having spoofed an IP address
Less than 1hour of outage on the last 12 months
9
![Page 10: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/10.jpg)
Paris: + 50%
Traffic growing
10
Marseille: +100%
![Page 11: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/11.jpg)
Reports available every quarter
More visibility
11
Sent to Marseille and Paris member’s lists
Contain only technical information
• Current and new deployments • New services • Outages, Maintenances
![Page 12: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/12.jpg)
Additional information on tools portal
More visibility
12
Weathermap, looking glass, …
Flow statistics again available
![Page 13: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/13.jpg)
Why?
Cleaning PoPs and optimizing operations
13
Beautiful but also…
Reducing risk of « accident » when we perform operations on site
Easy to ask for remote Hands&Eyes
![Page 14: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/14.jpg)
Removing old copper cables Redoing fiber cabling properly
Pre-cabling Avoid to go onsite for every customer connection à save time, money à speed-up new connections : Able to deliver a new
port on the same business day
Cleaning PoPs and optimizing operations
14
![Page 15: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/15.jpg)
15/20
![Page 16: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/16.jpg)
16/20
![Page 17: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/17.jpg)
Telehouse 2 upgrade: New Juniper EX9200 installed
• More redundancy, additional capacity • Customers split on 2 devices
Paris infrastructure update
17
TH2 PoPs Edge
1/3
TH2
PAR5 PAR2
PoPs Edge
![Page 18: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/18.jpg)
Iliad DC2 and DC3 upgrade
Paris infrastructure update 2/3
18
Backbone capacity upgrade 200Gbps
• From 70G to 200Gbps
2 Juniper EX9200 installed • 2 Brocade MLX-8
removed
Fast delivery 100G customer’s interfaces available
![Page 19: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/19.jpg)
Several 1G and 10G line cards installed to address new customer requests
A Juniper EX9200 and DWDM MUXs in stock ready to deploy
Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP
Paris infrastructure update 3/3
19
![Page 20: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/20.jpg)
Interxion MRS1 PoP Juniper EX9200 installed
100G ports available @Interxion MRS1 @Jaguar Network MRS01
2x40Gbps of capacity between Interxion and Jaguar PoP
Up to 400Gbps with existing solution
Marseille infrastructure update
20
![Page 21: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/21.jpg)
Deployments
21
![Page 22: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/22.jpg)
SLA probes
Study done to select the most appropriate solution
• Home made solution vs Turnkey solution
• Dedicated probes,
one per PoP
• 10Gbps vs 1Gbps
Main measurements: • Delay • Jitter • Frame loss
• Other KPI like
Reachability • Per customer • Per PoP • Global
22
Study
![Page 23: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/23.jpg)
Physical deployment
Dell R320 with 2x10Gbps interfaces deployed Paris and Marseille (11 PoPs) Connected to the peering LAN à same behaviour than a customer
SLA probes Deployment
23
![Page 24: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/24.jpg)
Operating system: Linux with no virtualisation to optimize performances
Some tests made with KVM à bad results when we try to reach 10Gbps
Ansible used for deploying and maintaining SLA probes
SLA probes Deployment
24
![Page 25: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/25.jpg)
Measurements Scapy library and python scripts used to perform measurements Full-mesh Any-to-Any à Matrix available LAG: how to test all the links belonging to a LAG?
• Hashing : Different IPs SRC/DST and L4 Ports
SLA probes Measurements
25
![Page 26: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/26.jpg)
Storage Influxdb
• Time series DB • Continues queries for aggregation
Visualisation Grafana for internal use Integration on tools.franceix.net soon
• Data available, at least for members
SLA probes Measurements
26
![Page 27: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/27.jpg)
SLA probes
infrastructure
27
Measurements-FranceIXpeeringLAN
DATAexport–OOBLAN
Underdevelopment
![Page 28: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/28.jpg)
SLA probes Visualisation
28
![Page 29: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/29.jpg)
SLA contract available soon: Possibility to have a contract with SLAs
Data visualization available before end of 2016
SLA probes Next steps
29
![Page 30: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/30.jpg)
• Useful for members when they don’t have flow statistics tools – Top talkers
• Useful for global statistics – Compare with SNMP - – IPv6 statistics – …
Flow statistics
30
![Page 31: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/31.jpg)
Flow export: sFlow on Brocade MLX IPFIX on Juniper EX9200 Sampling rate: 1/4096
• Enough for MAC SRC/MAC DST accounting
PMACCT for flow collection: Thanks for this great open source tool!!
Flow statistics
31
![Page 32: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/32.jpg)
InfluxDB for Flow storage Same tool than SLA probles Scalable: >300 members à #10 000 flows to store
RabbitMQ for queuing between PMACCT and infuxDB
Useful if there is an issue on InfluxDB or between PMACCT and influxDB APIs
Visualisation: integrated on tools.franceix.net
Flow statistics
32
![Page 33: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/33.jpg)
Flow statistics
infrastructure
33
sFlow
IPFIX
![Page 34: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/34.jpg)
Flow statistics Visualisation
34
![Page 35: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/35.jpg)
Next steps for 2017
35
![Page 36: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/36.jpg)
Currently: Passive muxes and 10G DWDM SFP+ Cost effective solution
• Passive MUX: very cheap • 10G DWDM SFP+ cheap • Up to 40 channels à 400Gbps
But…
• Difficult to deploy and maintain • Some limits on L3 equipment for aggregating links (LAG) • Poor optical monitoring (DOM integrated in SFP+
transceiver)
Optical infrastructure renewal
36
![Page 37: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/37.jpg)
100G optical solutions are more affordable
Interesting solution for DC interconnects (low distance less than 30km)
Optical infrastructure renewal
37
Study on going to deploy this type of infrastructure beginning of S2
2017
![Page 38: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/38.jpg)
Goal: Increase 100G port density Around 10 x 100G ports deployed before the end 2016 100G port has to be more cost effective
Objective: Deploy a new IP infrastructure in the core during summer 2017
RFP beginning of 2017 Deployment S2 2017
IP infrastructure renewal
38
![Page 39: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/39.jpg)
Available before end of 2016
BGP BH Next hop MAC address associated to this BH NH This MAC address is filtered on the edge
Possibility to use BGP communities selective BH possible
AntiDDOS Blackholing (BH)
39
![Page 40: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/40.jpg)
Dashboard in tools.franceix.net with useful information
Outages Maintenances Network Availability SLA probes information
New members connected Last upgrades …
More visibility
40
![Page 41: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/41.jpg)
Automate the processes
Minimize the time for connecting a new member
Automation
41
Research Build automated APIs for resellers Build automated APIs for marketplace vendors
![Page 43: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/43.jpg)
Config management
Technical update – part 2 by Arnaud
43
Logs TACACS
AS57734
Oxidized
Route servers
![Page 44: Technical Overview - France-IXA Juniper EX9200 and DWDM MUXs in stock ready to deploy Able to deploy a 100G PoP in less than 2 weeks • New PoP or upgrade existing PoP Paris infrastructure](https://reader035.vdocuments.mx/reader035/viewer/2022070705/5e95558c6c9c802d5212197f/html5/thumbnails/44.jpg)
44/20
Thank you for your attention