internet exchange points (ixps) philip smith e2 workshop, afnog 2005
TRANSCRIPT
Internet Exchange Points (IXPs)
Philip Smith
E2 Workshop, AfNOG 2005
Objectives To be able to explain what is an
exchange point To be able to explain why ISPs
participate in IXPs To understand why IXPs are important To review some current IXP designs
used today To think about how to set up an
exchange point in your environment
Introduction to Internet Exchange Points
A bit of history What are they? Why use them?
A Bit of History…
End of NSFnet – one major backbone move towards commercial Internet
private companies selling their bandwidth need for coordination of routing
exchange between providers Traffic from ISP A needs to get to ISP B
Routing Arbiter project created to facilitate this
What is an Exchange Point Network Access Points (NAPs)
established at end of NSFnet original “exchange points”
Major providers connect their networks and exchange traffic
High-speed network or ethernet switch Simple concept – any place where
providers come together to exchange traffic
Internet Exchange Points
ISPs connect at Exchange Points or Network Access Points to exchange traffic
XP 1 XP 2
ISP A
ISP B
Conceptual Diagram of an IXP
ISP Router
ISP Router
ISP Router
Exchange Point Medium
Why IXPs?
Multiple service providers Each with Internet connectivity
InternetInternet
AA BB
Why IXPs?
Is not cost effective Backhaul issue causes cost to
both partiesInternetInternet
AA BB
Why IXPs?
Domestic Interconnection
InternetInternet
AA BB
Exchange Structures layer 2 models (the NAP or IXP)
tailored bilateral policies
Why use an IXP?
PEERING Shared medium vs. point-to-point Shared
can exchange traffic with multiple peers at one location via one interface
Point-to-Point for high volumes of traffic
Why use an IXP?
KEEP LOCAL TRAFFIC LOCAL!!! ISPs within a region peer with each
other at local exchange No need to have traffic go overseas
only to come back Much reduced latency and increased
performance
Why use an IXP?
SAVES MONEY!!! Traffic going overseas means transit
charges paid to your upstream ISP Money stays in local economy
Used to provide better local infrastructure and services for customers
Customers pay less for Internet access
Therefore more customers sign up ISP has more customers, better business
Why use an IXP?
VASTLY IMPROVES PERFORMANCE!!! Network RTTs between organisations
in the local economy is measured in milliseconds, not seconds
Packet loss becomes virtually non-existent
Customers use the Internet for more products, services, and activities
Why use an IXP? Countries or regions with a successful
IXP have a successful Internet economy Local traffic stays local Money spent on local ‘net infrastructure Service Quality not an issue
All this attracts businesses, customers, and content providers
The Need for Exchanges
AS 2
AS 1
US
AFRICA
Local traffic travels via the US
The Need for Exchanges
AS 2
AS 1
US
AFRICA
Keep Local Traffic Local!
Exchange Point Design
Ethernet switch Has superseded all other types of
network devices for an IXP From the cheapest and smallest 12 or
24 port 10/100 switch To the largest 32 port 10GigEthernet
switch
What can go wrong? Too many exchange points in one
region competing exchanges defeats the
purpose Becomes expensive for ISPs to
connect to all of them
An IXP is not a competition, it’s not a business
What can go wrong? IXPs try to compete with their
membership IXPs run as a closed privileged club For example:
Interconnecting IXP locations with their own transit service
Providing access to end users rather than just Service Providers
Restrictive membership criteria (closed shop) Interfering with ISP business decisions e.g.
Mandatory Multi-Lateral Peering
Exchange Point policies/politics AUPs
Acceptable Use Policy Minimal rules for connection
Fees? Some IXPs charge no fee Other IXPs charge cost recovery A few IXPs are commercial
Nobody is obliged to peer Agreements left to ISPs, not mandated by
IXP
Exchange Point etiquette
Don’t point default route at another IXP participant
Be aware of third-party next-hop Only announce your aggregate
routes Filter! Filter! Filter!
And do reverse path check
Exchange Point examples
LINX in London, UK Ethernet switches
AMS-IX in Amsterdam, NL Ethernet switches
JPNAP in Tokyo, Japan Ethernet switches
Exchange Points in Africa CR-IX – Cairo, Egypt iBiX – Ibadan, Nigeria JINX – Johannesburg, South Africa KINIX – Kinshasa, Dem Rep of Congo KIXP – Nairobi, Kenya MOZIX – Maputo, Mozambique RINEX – Kigali, Rwanda SZIXP – Mbabane, Swaziland TIX – Dar es Salaam, Tanzania UiXP – Kampala, Uganda
Source: http://www.nsrc.org/AFRICA/afr_ix.html
Features of IXPs Redundancy
multiple switches Support
NOC to provide 24x7 support for problems at the exchange
DNS, Route Collector, Content & NTP servers CCTLD servers Content redistribution systems such as Akamai Route Collector – Routing Table view
Features of IXPs Location
neutral co-location facilities Address space
Peering LAN AS
If using Route Server (Route servers) Statistics
Traffic data – for membership
More info about IXPs
http://www.ep.net/ep-main.html Excellent resource for ip address
allocation for exchanges, locations of XPs in the world, AUPs and other policies
http://www.pch.net/documents Another excellent resource of IXP
locations, papers, IXP statistics, etc
Things to think about...
Do you need to be at an Exchange Point?
Would you want to start an Exchange Point?
Would keeping local traffic local benefit your ISP?
Would your environment (politically, etc.) support an Exchange Point?
Discussion
How would you build an exchange point in your environment?
Who would connect? What services would you provide? What policies would you enforce? What does your environment look
like? Is it feasible to set up an IXP?
Important to Remember...
Exchange Points can be as simple as an ethernet HUB!!!!
Keeping local traffic local improves performance cheaper often simple to do!
Exercise
Building an IXP
AS 1
BGP to provider
AS 3
AS 5
AS 7
AS 9
AS 2
AS 4
AS 6
AS 8
AS 10
AS100 AS200
AS 1
AS 3
AS 5
AS 7
AS 9
AS 2
AS 4
AS 6
AS 8
AS 10
196.200.220.224/28
SWITCH
Ethernet to IXP
Introduction to Route Collectors
Route Collector Background
What is a Route Collector? Features of a Route Collector Purpose of a Route Collector IXP Design with a Route Collector
What is a Route Collector?
Usually a router or Unix box running BGP
Gathers routing information from service provider routers at an IXP
Does not forward packets
Purpose of a Route Collector
To provide a public view of the Routing Information available at the IXP Useful existing members to check
functionality of BGP filters Useful for prospective members to check
value of joining the IXP Useful for the Internet Operations
community for troubleshooting purposes E.g. www.traceroute.org
Route Collector at an IXP
IXP R3R2
R1
ROUTE Collector
Route Collector Requirements Router or Unix system running BGP Peers eBGP with every IXP member
Accepts everything; Gives nothing Uses a private ASN Connects to IXP Transit LAN
“Back end” connection Second Ethernet globally routed Connection to IXP Website for public
access
Route Collector Implementation
Most IXPs now implement some form of Route Collector
Benefits already mentioned Great public relations tool Unsophisticated requirements
Just runs BGP
Introduction to Route Servers
Route Collector plus more
Route Server Background
What is a Route Server? Features of a Route Server Advantages of using a Route
Server Exchange Point Design with a
Route Server
What is a Route Server?
All the features of a Route Collector But also:
Announces routes to participating IXP members according to their routing policy definitions
Implemented using the same specification as for a Route Collector
Features of a Route Server Helps scale routing Simplifies Routing Processes on ISP
Routers Insertion of RS Autonomous System
Number in the Routing Path Handling of Multi-Exit Discriminator Route Flap Damping Uses Policy registered in IRR
Diagram of N-squared Mesh
With the Route Servers
RS based Exchange Point Routing Flow
TRAFFIC FLOW ROUTING INFORMATION FLOW
Advantages of Using a Route Server
Helps scale Routing Separation of Routing and
Forwarding Simplify Routing Configuration
Management on ISPs routers Enforce Good Routing Engineering Helps prevent the spread of bogus
routing information!
Disadvantages of Using a Route Server
ISPs lose direct policy control Dependent on 3rd party for
configuration and troubleshooting Insertion of RS Autonomous
System Number in the Routing Path
Peering with the Route Servers
Any ISP attached to an IXP can peer with the Route Servers
ISP must register their policy in the Internet Routing Registry Most IXPs who provide the RS facility
also provide a local IRR for policy registration
Must use BGP
Things to think about...
Would using a route server benefit you? Helpful when BGP knowledge is
limited Avoids having to maintain a large
number of eBGP peers
Introduction to the IRR
The Internet Routing Registry
What is the Routing Registry Contact names, email addresses and
telephone numbers for an AS Routing policy for an AS (what other
ASes does it connect to, which routes do they exchange)
Information about routes (most important is which AS originates the route)
Several other types of information
What is the Routing Registry?
Distributed database collectively known as Internet Routing Registry (IRR) APNIC, RIPE, ARIN, RADB, etc http://www.irr.net/docs/list.html
Providers register routing policy Used for planning, debugging and
generating backbone router configs
What is the Routing Registry?
Can be used by anyone worldwide debugging configuring engineering routing addressing
What happens if I don’t use the IRR
Routing Horror Stories AS7007 announcing bogus routes
Inconsistent policy at network borders Peers and upstreams need physical
notification of policy changes Mistakes easily made
So, I need to use the database because…..
Filters generated off the IRR protect against inaccurate routing information
Makes troubleshooting and debugging easier
Keep track of policy Security Filter! Filter! Filter!!
Why Bother using the IRR?
View of global routing policy in a single cooperatively maintained database
to improve integrity of Internet’s routing
generate router configs protect against inaccurate routing info
distribution verification of Internet routing
Why Bother using the IRR?
Many providers require that you register your policy (or they won’t peer with you)
Describing Policy
Use the policy languages to describe your relationship with other Peers routes importing routes exporting specific policies
interfaces, MEDs, communities
register routes with origin AS
Querying the Database
whois -h whois.ripe.net AS702 whois -h whois.ripe.net AS1849-
MAINT whois -h whois.ripe.net 158.43.0.0
How to Register your IRR policy
Register one or more maintainers Register AS and policy information Register Routes Describes your import and export
policy At the very least, provides contact
information
Router Configuration
Currently configs by hand - slow and inaccurate
Configuring routers using the IRR lots of tools available!!! IRRToolSet maintained by ISC
route and Aspath filters. Import and export
Filtering is a good thing...
Router Configuration
IRR
Configuration Machine
IRR Database
Server
Router
How do I use the IRR to generate configurations
Tools available to generate config files for most BGP implementations
IRRToolSet http://www.isc.org/sw/IRRToolSet/ Started off as RAToolSet as a project of
ISI Moved to RIPE NCC custodianship and
became IRRToolSet Enhanced to support RPSL (RFC2622)
Now maintained by ISC
How do I participate?
Set up your own registry Private for your ISP? Community for the region? Download the software (from ISC)
Use one of the many public IRR systems Ask AfriNIC to set one up?
Things to think about...
How would you register your policy? Try to describe it in an aut-num object
How would registering your policy benefit you? The community?