accessing cloud with disaggregated software-defined router
TRANSCRIPT
Accessing Cloud with Disaggregated Software-Defined Router
Hua Shao, Xiaoliang Wang, Yuanwei Lu, Yanbo Yu, Shengli Zheng, Youjian Zhao
NSDIApril, 2021
| Table of Contents
• Background and Motivation
• System Architecture
• Design Details
• Experience and Future Work
2
Cloud Network
Enterprise Branch
CVM
CVM
CDB
US VPC
Private Connection
| Tencent Cloud Access Network Overview
CVM
CVM
CDB
SH VPCCustom IDC
ISPs
3
Users
CVM
CVM
CDB
HK VPC
Access
Site
Access
Site
Enterprise Branch
Private ConnectionCustom IDC
ISPsUsers
54+AZs
27+Regions
4
| Challenges and Motivation
Traditional Commodity Router：
• Hardware & Software Vendor Lock-in• Slow feature velocity• Hard to Scale
IS-IS
RSVP-TE
MP-BGP
VPC C
EnterpriseA
VPC B
Region
B
C
A
User Requirement：
• Massive forwarding table, VRFs, channels• Roll out network features fast
• Scale for rapidly growth of traffic
4
Line Card Line Card Line Card
| Overview
PrimaryProcessor
SecondaryProcessor
Switching Fabric
5
External Peer
ControllerConsole
Routing Plane
Control Plane
Access Plane
Forwarding Plane
External Peer
FIB/ARP
FIB/ARP
Commodity Router Disaggregated Software-defined Router (DSR)
| Architecture of DSR
Access Module
BGP
NGW
RNSO
ExternalRouter
GNSOOSS/BSS
VPC
NGWForwarding Module
BGPRouting Module
RNSOControl Module
BGP/BFD
FIB/ARPconfig/monitor
T-GRE VxLAN
GNSOOrchestrator
RPC
config/monitor
FIB/ARP
| Scalability
7
CS
AS
CS
AS
CS CS
AS AS
NGWFCR
AS AS
RNSO
AS AS
GNSONGWForwarding
Plane
FCRRouting Plane
RNSOControl Plane
GNSOOrchestrator
• Each component scales independently
• Each network can be operated independently
• 3.2Tbps forwarding capacity
eBGP
eBGPeBGPeBGPeBGP
Node
Cluster
Node Node Node
Cluster
Node Node
VIP 2 VIP 3
VIP 1
| Reliability
• Single node/path failure will not affect the system
• Forwarding Path Failure Detection
• Data Plane supports Non-stop forwarding (NSF)
• Routing Plane supports Non-Stop Routing (NSR)
External
Router1
External
Router2
Routing
Module 1
Routing
Module 2Routing Plane
Control Plane
Forwarding Plane
NGWNGW
Data Module
NGWNGW
Control Module
8
| Customer Access (Private-Connection GW & VPNGW)
DSR
DSR
VPC 10.0.0.0/16
Interoperating with both External Network and SDN-Based Network at large scale
BGP Session
EA
BGP Session
Internet
CustomerRouter
Traditional NetworkSDN-Based Network
9
IPSEC VPN
Private Connection
| End-user Access (Content Provider)
Large scale forwarding table (10M) and flexible Traffic Engineering
EA2DSRISP Router2
BGP Session
VPC1 115.159.246.0/24
VPC2 116.150.247.0/24
EA1DSRISP Router1
BGP Session
VxLAN Fabric
10
| Flexibility - FW Service
• Support >100k flex rules for FWpurpose
Data Plane
DSR
VPC
VxLANFabric
FW Service
ExternalRouter
EA
<DIP> --> <FW, VNI><SIP> --> <FW, VNI>
11
| Flexibility - DDoS Service
SDR
VPC
DDoS Service
EA
180.10.1.1/32, DDoS
ExternalRouter
BGP route 180.10.1.1/32
Data Plane
12
| Flexibility - DDoS Service
• Redirect attack traffic to DDoS serviceefficiently
SDR
VPC
DDoS Service
EA180.10.1.1/32, DDoS0.0.0.0/0, DP
ExternalRouter
BGP route 180.10.1.1/32
Data Plane Only processing the real traffic
13
| Operationality - Monitoring
Operational Experiences
• 3 Levels Data Plane Probing
• Critical resources monitoring
• Various statistics and events
Forwarding Plane Cluster
core0
server0
core0 corex
RMOS
core0 core0 corex
server1
Cluster Level
Heath check
Server Level
Heath check
Core Level
Heath check
14
Future Works• End-to-End network quality detection
and analysis system for overlay network
• Simulation and verification system todetect and fix abnormal behaviors inadvance
| Conclusion
• Disaggregate functionalities into individualcomponents
• High scalability of each components at each level
• Fast features velocity via software programming
switch switch …
DataPlane
DataPlane
…
ControlPlane
ControlPlane
…
Orches-trator
Orches-trator
…
Scalability
Flex
ibili
ty
RoutingPlane
RoutingPlane
…
15
16
Thanks
Q&A