isdx: an industrial-scale software-defined ixp · bgp session sdn switch sdx controller sdx 3 . sdx...
TRANSCRIPT
![Page 1: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/1.jpg)
iSDX: An Industrial-Scale Software-Defined IXP
Nick Feamster Princeton University
http://sdx.cs.princeton.edu/
Arpit Gupta, Robert MacDavid, Rüdiger Birkner, Marco Canini, Jennifer Rexford, Laurent Vanbever
![Page 2: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/2.jpg)
Internet Exchange Points (IXPs)
AS A Router
AS C Router
AS B Router
BGP Session
Switching Fabric IXP
Route Server
2
![Page 3: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/3.jpg)
Software Defined IXPs (SDXs)
AS A Router
AS C Router
AS B Router
BGP Session
SDN Switch
SDX Controller
SDX
3
![Page 4: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/4.jpg)
SDX Creates New Possibilities
• More flexible business relationships – Make peering decisions based on time of day, volume of
traffic & nature of application
• More direct & flexible traffic control – Define fine-grained traffic engineering policies
• Better security – Prefer “more secure” routes – Automatically black hole attack traffic
4
![Page 5: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/5.jpg)
Three Years of Research: We Can Now Support Industry Scale
5
100 200 300 400 500
Participants
103
104
105
106
107
108
109
Forw
ardin
gT
able
Entr
ies
Unoptimized
MDS SDX-Central
iSDX
Optimal
BGP routes and updates for large EU IXP in a commodity hardware switch.
![Page 6: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/6.jpg)
iSDX Evaluation: Summary
• Data Plane State: – Requires 65K < 100K forwarding table entries
• Data Plane Update Rate: – Requires 0 < 2500 updates/second
• Other Goals: – Processes BGP update bursts in real time (50 ms) – Requires only 360 BGP Next Hops compared to 25K
from previous solutions
6
![Page 7: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/7.jpg)
Constraints (and Insight)
7
Devices Operations Data Plane Performance
State (# entries)
Update Rate (flow-mods/s)
Match-Action on Multiple Headers 100K 2,500
Matches on IP Prefixes only ~1M N/A
Insight: Optimize the use of available resources on each device.
![Page 8: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/8.jpg)
AS A IXP Fabric
AS B
AS C announces
10/8, 40/8
AS D announces 10/8, 40/8, 80/8
AS E announces
80/8
Simple Example
dPort = 443 → fwd(C) dPort = 22 → fwd(C)
dPort = 80 → fwd(E)
SDX Controller
8
![Page 9: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/9.jpg)
Forwarding Table Entries at SDX
SDN Policies # Forwarding Table Entries
dPort = 443 → fwd(C) 1
dPort = 22 → fwd(C) 1
dPort = 80 → fwd(E) 1
Number of forwarding table entries for A & B’s Outbound SDN Policies
AS A
AS B
9
![Page 10: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/10.jpg)
Number of Forwarding Entries
10
Simple Example Large IXP
Baseline 3 62K
• Data from Large IXP: – BGP RIBs & Updates from 511 participants – 96 million peering routes for 300K IP prefixes – 25K BGP updates for 2-hour duration
Scales, but is not congruent with BGP!
![Page 11: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/11.jpg)
AS A IXP Fabric AS C announces
10/8, 40/8
Congruence with BGP Policies
dPort = 443 → fwd(C)
p dIP = 80.0.0.1, dPort = 443
Problem: Need to ensure p is not forwarded to C.
SDX Controller
11
![Page 12: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/12.jpg)
AS A IXP Fabric AS C announces
10/8, 40/8
Solution: SDN Policy Augmentation
dIP ∈ {10/8, 40/8} ⋀ dPort = 443 → fwd(C)
p dIP = 80.0.0.1, dPort = 443
Match on prefixes advertised by C. SDX Controller
12
![Page 13: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/13.jpg)
Data Plane State Explosion!
SDN policy augmentation increases forwarding table entries.
SDN Policies # Forwarding Table
Entries
10/8 40/8 80/8
dPort = 443 → fwd(C) 1 1 0
dPort = 22 → fwd(C) 1 1 0
dPort = 443 → fwd(D) 1 1 1
4
3
13
![Page 14: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/14.jpg)
Number of Forwarding Entries
Simple Example Large IXP
Baseline 3 62K
Policy Augmentation 7 68M
Cannot support forwarding table entries and update rate.
14
![Page 15: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/15.jpg)
Three Insights (and Optimizations)
• Many prefix, policy combinations have exactly the same forwarding decision – Optimization: Forwarding equivalence
• Per-participant forwarding decisions have even more commonality – Optimization: Independent forwarding equivalence
• Advertisements can be encoded as FEC entries – Optimization: Reachability encoding
15
![Page 16: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/16.jpg)
Forwarding Equivalence Classes
SDN Policies # Forwarding Table
Entries
10/8 40/8 80/8
dPort = 443 → fwd(C) 1 1 0
dPort = 22 → fwd(C) 1 1 0
dPort = 443 → fwd(D) 1 1 1
10/8, 40/8 exhibit similar forwarding behavior.
16
![Page 17: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/17.jpg)
Applying Forwarding Equivalence
10/8
40/8
80/8
forward to BGP Next Hop
AS S
AS C announces 10/8, 40/8 AS D
announces 10/8, 40/8, 80/8
IXP Fabric
Single BGP next hop for 10/8, 40/8
17
dPort = 443 → fwd(C)
![Page 18: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/18.jpg)
Applying Forwarding Equivalence
10/8
40/8
80/8
forward to BGP Next Hop
AS S IXP Fabric
fwd(C)
match on BGP Next Hop
AS C announces 10/8, 40/8 AS D
announces 10/8, 40/8, 80/8 Flow rules at SDX match on
BGP next hops. 18
dPort = 443 → fwd(C)
![Page 19: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/19.jpg)
Number of Forwarding Entries
Simple Example Large IXP
Baseline 3 62K
Policy Augmentation 7 68M
*FEC Computation 4 21M
Still not possible to support forwarding table entries and update rate.
[*SIGCOMM’14]
19
![Page 20: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/20.jpg)
Three Insights (and Optimizations)
• Many prefix, policy combinations have exactly the same forwarding decision – Optimization: Forwarding equivalence
• Per-participant forwarding decisions have even more commonality – Optimization: Independent forwarding equivalence
• Advertisements can be encoded as FEC entries – Optimization: Reachability encoding
20
![Page 21: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/21.jpg)
SDN Policies # Forwarding Table
Entries
{10/8, 40/8} 80/8
dPort = 443 → fwd(C) 1 0
dPort = 22 → fwd(C) 1 0
dPort = 443 → fwd(D) 1 1
Independent FEC computation is more efficient.
What If Each Participant Computes FEC Independently?
21
![Page 22: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/22.jpg)
Independent FEC Computation
• Large number of SDX participants – Many different policies on groups of prefixes – Leads to a large number of small FECs of prefixes
• Compute FECs independently – Separate computation per participant – Leads to small number of large FECs, and less
frequent recomputation – Enables “scale out” of the FEC computation
22
![Page 23: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/23.jpg)
SDN Policies # Forwarding Table
Entries
{10/8, 40/8} 80/8
dPort = 443 → fwd(C) 1 0
dPort = 22 → fwd(C) 1 0
dPort = 443 → fwd(D) 1
Idea: Each participant independently computes FECs.
Independent FEC Computation
2
1 23
![Page 24: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/24.jpg)
Number of Entries
Simple Example Large IXP
Baseline 3 62K
Policy Augmentation 7 68M
FEC Computation 4 21M
Independent FEC Computation 3 763K
Still not possible to support forwarding table entries and update rate.
24
![Page 25: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/25.jpg)
Three Insights (and Optimizations)
• Many prefix, policy combinations have exactly the same forwarding decision – Optimization: Forwarding equivalence
• Per-participant forwarding decisions have even more commonality – Optimization: Independent forwarding equivalence
• Advertisements can be encoded as FEC entries – Optimization: Reachability encoding
25
![Page 26: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/26.jpg)
BGP & SDN Coupling
SDN Policies # Forwarding Table
Entries
10/8 40/8 80/8
dPort = 443 → fwd(C) 1 1 0
dPort = 22 → fwd(C) 1 1 0
dPort = 443 → fwd(D) 1 à 0 1 1
Incoming BGP Update: {AS D withdraws route for prefix 10/8}
26
![Page 27: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/27.jpg)
Decoupling BGP from SDN
• Apply advances in commodity hardware switches – Support for Bitmask Matching (OpenFlow 1.3)
• Extend BGP “next hop” encoding – So far: encode FECs (single field) – Idea: encode reachability encoding
• Changing only the BGP announcements – No need to update the SDX data plane!
27
![Page 28: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/28.jpg)
Reachability Encoding
10/8
40/8
80/8
forward to BGP Next Hop
Reachable via AS C
AS S
AS C announces
10/8, 40/8
AS D announces 10/8, 40/8, 80/8
IXP Fabric
Reachable via AS D
Dedicate one bit per participant
dPort = 443 → fwd(C)
28
![Page 29: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/29.jpg)
Reachability Encoding
Reachable via AS C
Reachable via AS D
AS S
AS C announces
10/8, 40/8
AS D announces 10/8, 40/8, 80/8
10/8
40/8
80/8
fwd(C)
match on Reachability Bitmask
IXP Fabric
forward to BGP Next Hop
Independent of BGP Dynamics
dPort = 443 → fwd(C)
29
![Page 30: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/30.jpg)
Reachability Encoding
SDN Policies # Forwarding Table
Entries
C
dPort = 443 → fwd(C) 1
dPort = 22 → fwd(C) 1
dPort = 443 → fwd(D) 1
Reduces Data Plane State
2
1
30
![Page 31: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/31.jpg)
Number of Forwarding Entries Simple
Example Large IXP
Baseline 3 62K
Policy Augmentation 7 68M
FEC Computation 4 21M
Independent FEC Computation 3 763K
Reachability Encoding 3 65K
We can now run SDX over commodity hardware switches.
31
![Page 32: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/32.jpg)
We Can Do This at Industry-Scale!
32
100 200 300 400 500
Participants
103
104
105
106
107
108
109
Forw
ardin
gT
able
Entr
ies
Unoptimized
MDS SDX-Central
iSDX
Optimal
BGP routes and updates for large EU IXP in a commodity hardware switch.
![Page 33: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/33.jpg)
iSDX Evaluation: Summary
• Data Plane State: – Requires 65K < 100K forwarding table entries
• Data Plane Update Rate: – Requires 0 < 2500 updates/second
• Other Goals: – Processes BGP update bursts in real time (50 ms) – Requires only 360 BGP Next Hops compared to 25K
from previous solutions
33
![Page 34: iSDX: An Industrial-Scale Software-Defined IXP · BGP Session SDN Switch SDX Controller SDX 3 . SDX Creates New Possibilities • More flexible business ... traffic & nature of application](https://reader035.vdocuments.mx/reader035/viewer/2022063013/5fcb7e0851d137390b477cbb/html5/thumbnails/34.jpg)
You Can Run iSDX Today • Running code
– Vagrant & Docker based setup – Instructions to run with Hardware Switches
• Ongoing efforts – Hosted by Open Networking Foundation – Deployment
• Inter-agency exchange • IXPs in Europe & Asia
34
http://sdx.cs.princeton.edu