nanog24 miami 12th february 2002

Agnès Pouélé (agnes.pouele@dante.org.uk), Jan Novak (janovak@cisco.com)

Inter-domain Multicast in European Research Networking:

TEN-155 Operational Experience and Deployment on GÉANT

NANOG24

Miami 12th February 2002

Agnes Pouélé, DANTE Ltd.Network Engineer

Jan Novak, Cisco Systems Inc. Network Consulting Engineer

Agenda• DANTE

• TEN-155 Operational Experience– Evolution of the TEN-155 multicast topology from

1998 to 2000– Operational Experience

• GÉANT deployment– Overview of GÉANT Network– GÉANT’s Multicast design– GÉANT Multicast and Unicast Coverage– GÉANT Multicast Service and Monitoring

• Conclusion2

DANTE, TEN-155, GÉANT

• DANTE • DANTE is a not-for-profit company set up in 1993

by European National Research Network organizations.

• TEN-155 • was an ATM based network built initially on OC3

links and then upgraded in 2000 • GÉANT

• successor of TEN-155 • launched in December 2001• 10 Gbps Pan-European Network

Part I TEN-155 Operational Experience

Typical mess of DVMRP tunnels on Sun WSs with usual tunnel routing problems.

Starting pointMBONE TUNNELED TOPOLOGY

Agnès Pouélé (agnes.pouele@dante.org.uk), Jan Novak (janovak@cisco.com) 6

OSLO IETF transmitted over both native STM-1 ATM based network and DVMRP tunnelled infrastructure

1999: PIM-SM and DVMRP set-up

PIM-SM domain

Multicast deployment in TEN-155

• Based on this first set-up and successful operation, we concluded to enable multicast on all production routers in TEN-155.

• Multicast code stable, CPU usage ok, M-BGP ok• PIM-SM “only” for production service

• PIM-SM to DVMRP border works, but not possible to operate routinely (NOC)

• From 1999 to 2000 each country was migrated from the DVMRP cloud to a interconnection with TEN-155 using PIM-SM/MBGP/MSDP

2000: TEN-155 final topology

NLUnited Kingdom

France

Greece

Belgium

Slovenia

Czech Rep

Germany

Switzerland

Portugal

AS8933OSPF + internal MBGP

Poland

Israel

NetherlandsNordics

e x t e r n a l

a l e x t e r n

AT & T

Luxembourg

Hungary

ABILENE

tunnel

unicast/mcast linededicated mcast

TEN-155 Operational ExperiencePerformance Impact

• Parameters to be considered for the exploitation of Multicast– CPU - parameters

• about 1600 forwarding (mroute) entries• max 8 outgoing interfaces (average 2-3)• max 20 Mbit/s of data forwarded by one router resulting in 5-10% of CPU increase (mainly PIM)

TEN-155 Operational ExperiencePerformance Impact

• Parameters to be considered for the exploitation of Multicast– Memory

• about 3000 SA messages in the cache• about 1600 forwarding entries (mroute)• about 10 000 routes in the MBGP table

• These parameters didn’t represent a significant memory usage

Usage of MSDP – new component – new storms

• Default peer in redundant topology

• “Redundant” mesh-groups

MSDP peers default peer

Group A

Group C Group B

Impact of a MSDP storm on the CPU load

Usage of MSDP – early implementation problems

• Cisco IOS 12.0.6S and lower– ghost SA entries in the MSDP cache

• SA messages recreated by the incoming (S,G) joins

• FIXED– Origination of SA messages only when source registered

to the originating RP

– “ip msdp redistribute” command – without arguments• re-originated all known SAs

• caused huge increase of the SA counts worldwide• FIXED

TEN-155 MSDP monitoring

•MSDP usage monitoring •number of RPs, average 90 (40 EU)•number of groups•number of pairs source, group (S,G)•number of SA messages per minute

•Monitoring set-up

WS - DANTE DE.TEN-155router

•Software – adapted C++ MSDP •implementation of Steve Rubin

MSDP peering

TEN-155 MSDP monitoring

Number of RPs announced to TEN-155 before and after the loss of US connectivity

number of RPs

TEN-155 MBGP monitoring

•MBGP monitoring •stability/updates•number routes, max about 9000 (760 EU)•number of ASNs, average 240 (80 EU)

•Monitoring set-up

WS - DE PoP DE.TEN-155router

•Software - Merit’s MRTD - •modified SAFI definition for MBGP

MBGP peering

TEN-155 MBGP monitoring

•In blue, •the number of routes originated from one AS

•In green, •the number of updates originated by the same AS

stability/updates

TEN-155 Multicast Monitoring tools

• http://www.dante.net/pubs/dip/40/40.html

• http://www.dante.net/pubs/dip/41/41.html

• http://www.dante.net/mbone/

• http://sigma.dante.org.uk/stats/mrtg/msdp/data/

• http://www.dante.net/mbone/mbgp

The graph values from mbgp and msdp monitoring are historical values.

TEN-155 Operational ExperienceConclusion

• Move to native and SM multicast in national networks (NRENs)

• Unicast and multicast non congruent• MSDP peer doesn’t need to be RP.• Concept of two BGP tables and “multi-protocol” RPF

check often still misunderstood.• Inter-domain Multicast debugging

=> Almost impossible to fix problems in just one week

PART IINEXT GENERATION

GÉANT DEPLOYMENT

Agnès Pouélé (agnes.pouele@dante.org.uk), Jan Novak (janovak@cisco.com) 20

www.dante.net/nep/GÉANT-MULTICAST/map.html

GÉANT Services • GÉANT Standard IP Service

– IP traffic from NREN to NREN and Research peerings.

• Multicast Service (rolling out now)

• Replacement of TEN-155 Managed Bandwidth Service– GÉANT Premium IP Service – Layer-2 VPNs (forthcoming)

• Upcoming

• Security and Dos attack detection, IPV621

GÉANT’s Customers and Other Peerings

• GÉANT Unicast Customers– 27 countries in Europe

• GÉANT Multicast Customers – 24 countries in Europe

• GÉANT Unicast and Multicast research and commercial peerings– Abilene, Canarie and ESnet via GTREN– Infonet, UUNET

GÉANT Multicast Customers

NREN ACCESS MCAST Type

Austria POS STM-4 NO Mcast

Slovenia POS STM-4* NATIVE

Belgium POS STM-16 NATIVE

Croatia ATM E3 NATIVE

Czech Rep. POS STM-16 NATIVE

Cyprus E3 TBA

Germany POS STM-16 NATIVE

ESTONIA POS STM-1 NATIVE

Portugal POS STM-4 NATIVE

Greece POS STM-16* NATIVE

Ireland POS STM-1 NATIVE

Hungary POS STM-16 NATIVE

Italy POS STM-16 NATIVE

Israel T3 NATIVE

United Kingdom POS STM-16 NATIVE

Latvia E3 NATIVE

Lithuania T3NATIVE

Norway, Sweden, Finland,Island

POS STM-16 NATIVE

Poland POS STM-16 NATIVE

Spain POS STM-16 NATIVE

France POS STM-16 NATIVE

Luxembourg POS STM-1 NATIVE

Roumania E3 NATIVE

Slovak Rep. POS STM-1 NATIVE

Netherlands POS STM-16 NATIVE

Switzerland POS STM-16 NATIVE/TBA

Bulgaria E3 TBA

GÉANT Multicast Implementation

• Current design built on– The experience gained from TEN-155– Guidelines from multicast experts– Juniper Laboratory tests – GÉANT Multicast services

–Multicast transit domain NRENs to NRENs–Multicast transit domain NRENs to other

PEERS–Beacon monitored backbone

• GÉANT Multicast routing policy at:–

http://www.dante.net/nep/GEANT-MULTICAST/routing-policy.html

Manganese

Iridium

HeliumGallium

Sulfur

HostYXenon

Network 10.2/16 –AS 102

Network 10.0/16-AS 100

Network 10.1/16-AS 101

Network 10.3/16 –AS 103

Group 233.1.10.1Source 10.1.10.2

Source 10.2.20.2Group 233.2.20.1 Group 233.3.30.1

Source 10.3.30.2

Group 233.0.1.1Source 10.0.1.2

Group 233.0.3.1Source 10.3.30.2

TEST-BED LAB 26/27th Nov 2001

M-BGP peering

MSDP peering

I-MBGP full mesh

•Junos 5.0R3.3

• mesh group

I-MSDP full mesh

PIM SMv2 GÉANT domain

• GÉANT single PIM-SMv2– Version 2 , enabled on all interfaces.

• Three Rendez-vous Points with private anycast address. – backup for internal sources and receivers.– Private Anycast address (filtered out)– closest RP based on the OSPF cost

• All other interconnected administrative domain have to be PIM-SM v2 enabled with their dedicated RP.

PHYSICAL TOPOLOGY: RP

IUCCJanet NY4-1 Nordunet

Latnet

Litnet

Posnan

DFNNY4-2

Infonet

Multicast access

Rendez-vous Point

Cesnet

HungarnetRoEduNet

Switch&Cern

Renater

Rediris

Surfnet

Belnet

Restena

Heanet

Carnet

Aconet

Unicom-bCynet

STM64/OC192STM16/OC48STM4/OC12STM1/OC3

GÉANT Design: MBGP and MSDP• MBGP

• Separate multicast routing table (inet.2) • Currently congruent BGP and MBGP topology in

Europe

• iMSDP • MSDP is fully meshed between 19 PoPs• Use of mesh group• i-MSDP Peering with loopback addresses (Not

the anycast address !!)

• eMSDP• NREN <---> GÉANT Access Router

MSDP peering Logical view

iMSDP Mesh Group

iMBGP Full Mesh

Iucc Janet Ny4-1

Renater

Switch&Cern

Rediris

RoEdunet

Sanet Posnan

Cesnet

Infonet

Nordunet

Latnet

Litnet

Carnet

CynetUnicom-b

Restena

beBelnet

Hungarnet

Heanet

Surfnetnl

Aconet

External MSDP peering NRN ----- GÉANT router

Abilene

www.dante.net/nep/GEANT-MULTICAST/map.html

MSDP SA Filtering

• A list of filtered SA is defined at:– http://www.dante.net/nep/GEANT-MULTICAST/

deployment-msdp.html

• We filter the recommended list.

• We authorise 239.194.0.0/16 from the IPV4 Organisation Local Scope through GÉANT.

GÉANTAS20965

Third party providerEuropean Distributed Access

Commodity Internet Access

GTRENRESEARCH Peerings

STM-16

... ...

Abilene

www.dante.net/nep/gtren.html

GÉANT Multicast and Unicast Coverage

GÉANT Multicast Service

• Access to the service– Via the primary access to GÉANT– Via a GRE tunnel (currently nobody)– Support of PIM-SM v2 only

• Operational procedures (rolling out now)– Goal: same level of service as Unicast.

• Troubleshooting – Extension of the trouble ticket systems to multicast

incidents

GÉANT multicast monitoring

• Beacon– Tool initially developed by Kai Chen from

NLANR• dast.nlanr.net/projects/Beacon/

• Relies on a number of Agents spread over the network which simultaneously send and receive multicast packets carrying a packet sequence number and a timestamp.

• Communicates with a central server which displays matrices of Agents via web pages.

GÉANT multicast monitoring• Recommended by TF-NGN group

– (www.dante.net/tf-ngn)

• To use from day 1– monitoring of multicast inside and outside of GÉANT.

• Server code enhanced with historical functionality http://noc.man.poznan.pl/noc/index/strony (Menu item

“Applications”)

romradz@man.poznan.pl

Multicast Beacon Agent written in C http://www.cesnet.cz/tf-ngn/multicast/

GÉANT multicast monitoring• Beacon’s matrices

– One for the internal sources of GÉANT• Each GÉANT POP has a beacon agent installed

– One for the external sources of GÉANT• http://beaconserver.geant.net:19999/

– We have assigned two multicast groups from GLOP range [RFC2770] for each matrix

• Parameters monitored• Loss• Delay• Jitter

Beacon internal/external matrix

www.dante.net/nep/GÉANT-MULTICAST/deployment-beacon.html

Other Monitoring Tools

• Per group monitoring from TEN-155– Based on the IETF IP-MROUTE MIB – Shows traffic per multicast group per interface– Under installation on ws1.se.geant.net

• MSDP and MBGP monitoring tools– Not yet available

CONCLUSION

• Conclusion– From TEN-155 to GÉANT Unicast and multicast

moves to a congruent topologyacross Europe and towards research peerings.

• Deployment status

• Links– http://www.dante.net/nep/GEANT-MULTICAST/– http://beaconserver.geant.net:19999/

We would like to thank and acknowledge the help of the people who worked and are working with

us on these projects, mainly from all EU and US research networks

THANKS

Questions ?

Peering relationship Address DescriptionAll 224.0.1.2/32 SGI "Dogfight" gameAll 224.0.1.3/32 RWHODAll 224.0.1.8/32 Sun's NIS+All 224.0.1.22/32 SVRLOCAll 224.0.1.24/32 MICROSOFT-DSAll 224.0.1.25/32 NBC-proAll 224.0.1.35/32 SVRLOC-DAAll 224.0.1.39/32 Cisco's Rendezvous Point Announcement ProtocolAll 224.0.1.40/32 Cisco's Rendezvous Point Discovery ProtocolAll 224.0.1.60/32 HP's Device Discovery ProtocolAll 224.0.2.1/32 rwho group (BSD)All 224.0.2.2/32 Sun's Remote Procedure Call ProtocolAll 229.55.150.208/32 Norton "Ghost" disk duplication softwareAll 234.42.42.42/32 ImageCast disk duplication softwareAll 234.142.142.142/32 ImageCast disk duplication softwareAll 232.0.0.0/8 Default SSM-range. Do not do MSDP in this range.Out of GÉANT 239.0.0.0/8 Administratively Scoped IPv4 Group Addresses

In GÉANT

239.0.0.0/8 with exception of 239.194.0.0/16

Administratively Scoped IPv4 Group Addresses. Th 239.194.0.0/16 address address range is reserved for GÉANT only multicast traffic.

All 10.0.0.0/8 Private addresses (RFC 1918)All 172.16.0.0/12 Private addresses (RFC 1918)All 192.168.0.0/16 Private addresses (RFC 1918)All 127.0.0.1/8 Loopback address

SA Filter list

nanog24 miami 12th february 2002

Documents

miami, november 12th, 2019 - corporate jet investor ·...

12th annual offshorealert conference agenda: may 4-6, 2014,...

ool · 2021. 1. 20. · 901 s. miami avenue, suite 300,...

milwaukee chicago icpr corridor: hiawatha service …...arun...

px ease join aia miami e mcad invite you to … · px_ease...

2019-2020 media kit · 2019-11-12 · key biscayne medley...

shops at beacon lakes - stiles · 2017. 3. 29. · miami...

rana florida · contemporary art will gather in miami beach...

sibergen services - miami web design, miami hyper-v, miami...

sibergen services miami web design, miami hyper-v, miami...

facultÉ de mÉdecine de tours · sylvester cancer center,...

diseno web miami,paginas web miami ,servicios web en miami

miami-dade county public schoolsmas.dadeschools.net ›...

dentists miami | miami dentists | dentists in miami | miami...

nhssf 2020 request for proposal issue date: september 15,...

miami-miami beach-kendall, florida

seminario internacional de miami miami international...

equinix ixp update – nanog24 lane patterson equinix, inc....

miami heat miami herald academic sportsmanship honorees ·...

detailed analysis of isis routing protocol on the qwest...