General Meeting 2014

Technical Overview By Simon Muyal

26th September 2014

Agenda

1/25

Agenda France-IX Technical update

Fibre infrastructure renewal Optical layer: DWDM solution

Issue of 18th of August : Some explanations

2/25

Fibre infrastructure renewal

3/25

Iliad - Vitry TeleCity Courbevoie

Interxion-5

Interxion-2

Telehouse-2 Telecity

Condorcet

Interxion-1

FranceIX PoP - Edge FranceIX PoP - Core nx10 Gbps

Fibre infrastructure renewal

4/25

• Redundancy – each Edge PoP connected to 2 core PoPs – Different paths to avoid SPOFs

• Quality of the fiber Important for 100G

– Short paths ( <20km) even if it is more expensive ( permit to save money for DWDM infrastructure)

– PMD reports

• New fiber infrastructure Makes deployment easier

Fibre infrastructure renewal Main goals

5/25

• Some links provided by DC • For the other links, 3 offers very competitive

Fibre infrastructure renewal Tender and Selection

Deployment: Done in June

3 links 3 links

6/25

Iliad - Vitry TeleCity Courbevoie

Interxion-5

Interxion-2

Telehouse-2 Telecity

Condorcet

Interxion-1

FranceIX PoP - Edge FranceIX PoP - Coeur

Fibre infrastructure renewal

6’/25

Optical layer DWDM solution

7/25

Comparison between

Optical layer: different solutions

100Gbps Active DWDM

(Adva, Alcatel, etc) Too expensive,

even with long term scenarios

40 channels passive MUX with nx10Gbps

100Gbps

Passive MUX + amplifiers Too expensive, even with

long term scenarios

100Gbps

BW (once per fiber) Not scalable,

difficult to maintain

8/25

• 40 channels passive MUX with nx10Gbps – Taking into account short distance between PoPs and the high

price for 100G line cards, this solution is the most appropriate – Starting with 10x10G (100G of capacity) for core backbone links – Up to 400G of capacity, just adding transceivers in the routers

easy to evolve

Optical layer: selected solution

selected for 40Ch-MUX selected for DWDM transceivers

9/25

Backbone infrastructure renewal

10/25

• Replace core switches (3 PoPs): – Support 100Gbps in the main PoPs – Increase 10Gbps density

• Replace edge switches (4 PoPs) by existing core switches (Brocade MLX): – Homogeneous configuration

(VPLS and L2 filtering in the edge) – Increase capacity

• Lease financing

Main goals Backbone infrastructure renewal

11/25

• ALCATEL/LUCENT • BROCADE • CISCO • EXTREME • JUNIPER

Short-list

Tender and selection Backbone infrastructure renewal

12/25

Juniper selected: EX9200, 14 slots

Tender and selection Backbone infrastructure renewal

13/25

Best note taking following criteria

Tender and selection Backbone infrastructure renewal

Price (leasing mandatory)

40%

Technical Installation delays 10% mandatory and

optional criteria

30% Vendor support

20%

POC to validate technical part

14/25

Tender and selection Backbone infrastructure renewal

One week to perform 31 tests: OK - 2 EX9200 - 1 MLX-16 - 1 IXIA - servers

March April

Done in

15/25

Test 1 IPv4 and IPv6 configuration OK Test 2 Jumboframes OK Test 3 ISIS configuration (Multi-Topology ISIS) OK Test 4 MPLS configuration OK Test 5 VPLS configuration OK Test 6 … OK … … …

Rollout schedule

POC

Fiber

DWDM

EX9214 (Core)

Brocade (Edge)

Deployment (October/Nov)

Staging (June/July) Deployment (end of July)

June

End of May/June

March/April

16/25

Iliad - Vitry TeleCity Courbevoie

Interxion-5

Interxion-2

Telehouse-2 Telecity

Condorcet

Interxion-1

Juniper EX 9214 Brocade MLX 16

Fibre infrastructure renewal

Brocade MLX 16 Force 10

16’/25

3 new PoPs available (or coming soon)

DC3 (Vitry)

17/25

Saint-Denis

Aubervilliers

18th of August issue

18/25

Two main issues

12:00 Some flooded traffic, around 300Mbps Some impact for members

15:00

Loop amplifying the 300Mbps of traffic reaching 2Gbps of flooded traffic on some interfaces Big impact for members, Route Servers impacted

19/25

• Flooded traffic appeared again during the week several times – Flow monitoring was disabled

Two main issues

17:00

18:00

Loop stopped

Flooding traffic stopped

20/25

Flooding issue investigation

Some sniffers installed in the backbone • Flooded traffic was unicast traffic (and not broadcast)

Bug detected on Juniper devices: • Root cause: Issue occurs randomly when an

interface is flapping. The issue didn’t appear during the POC.

• Control plane and forwarding plane are not coherent problem when the MAC address entries are replicated to the forwarding engines (PFE) • “Unknown unicast” policer didn’t work as

expected 21/25

Preventing the issue from occurs again

In the meantime, to prevent issue occurs again, we installed an event script to be proactive,

Unknown unicast traffic is checked every 5 seconds

If flooded traffic is not legitime, we clear VPLS entries

corresponding with the MAC addresses where flooded traffic

is observed.

22/25

A fix will be available soon

The fix should be available soon

A fix is currently tested by

in November

23/25

Improvements

On our side: - Route servers: Direct connection to core routers - Clearing abnormal traffic: proxy ARP, RA packets, STP, CDP/FDP, OSPF/ISIS, … FranceIX expects more reactivity from JUNIPER if an issue occurs: - JTAC more available when we open a case - We consider exchanges between Juniper Engineering team (US) and JTAC too slow

24/25

Thank you for your attention