Nagoya Institute of Technology, Japan.
Keisuke Matsuo Ryota Kawashima Hiroshi Matsuo Nagoya Institute of Technology, Japan. The Goals Proposal Realizing broadcast/multicast in virtual networks
Reducing traffic amount in substrate networks Broadcast Multicast Virtual Network VM VM VM VM Proposal PM PM PM PM Physical Network PM : Physical Machine Outline Backgrounds Related Work Proposal Evaluation 1
Network Virtualization 2 Related Work IP multicast in substrate networks Source-based Unicast Replication 3 Proposal 4 Evaluation Multi-tenant Datacenter Networks
Overlay-based network virtualization Each tenant can have its own virtual networks Virtual Networks Tenant A VM VM Tenant C VM VM Tenant B VM VM VM VM VM Physical Network Broadcast on Virtual Networks
Multiple VMs of different tenant run on same PMs Physical Switch Virtual Switch VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM Sender Physical Machine Generate many wasteful packets VM : Tenant A VM : The Other Tenant Broadcast on Virtual Networks
VM VM VM VM PM PM PM PM PM PM PM PM Physical Network PM : Physical Machine B/Mcast Support on Major Services
Amazon VPC, Microsoft Azure Broadcast/Multicast do not be supported VMware NSX Supporting broadcast/multicast Unicast Replication methods Major Broadcast Protocols
Physical networks use various protocols ARP RARP Gratuitous ARP DHCP Broadcast communication on Virtual Networks is not allowed Functionality of virtual networks is poor Major Multicast-related Protocols
Routing protocol OSPF Construction of routing table Providing High-Availability VRRP Heartbeat messages among servers Without IP Multicast, we cant construct practical virtual networks Outline Backgrounds Related Work Proposal Evaluation 1
Network Virtualization 2 Related Work IP multicast in substrate networks Source-based Unicast Replication 3 Proposal 4 Evaluation IP Multicast in Substrate Networks
VXLAN Mapping virtual networks to IP multicast groups Virtual Switch VM VM VM VM VM VM VM Multicast group A Virtual Network A VM VM VM VM VM VM VM VM VM Multicast group B Multicast group C Virtual Network C Virtual Network B Physical Machine IP Multicast in Substrate Networks
Virtual Network VM VM VM VM VM VM VM router router router PM PM PM PM PM PM PM PM Physical Network Introducing and managing multicast routers cost a lot Source-based Unicast Replication (SUR)
Duplicates the packet Replaces dest addresses Hot-spot Controller Hot-spot VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM Sender Hot-spot in both upstream and sender-side links VM : Tenant A VM : The Other Tenant Source-based Unicast Replication
Virtual Network VM VM VM VM VM VM VM Controller Hot-spot Hot-spot PM PM PM PM PM PM PM PM Physical Network Outline Backgrounds Related Work Proposal Evaluation 1
Network Virtualization 2 Related Work IP multicast in substrate networks Source-based Unicast Replication 3 Proposal 4 Evaluation (Broadcast, Unknown unicast and Multicast)
Our Proposal TE-Cast: Topology Embedded xCast Supporting broadcast/multicast in virtual networks Reducing traffic amount in substrate networks Logically grouped virtual switches Encapsulated BUM packets with topology information On-the-fly packet forwarding on virtual switches (Broadcast, Unknown unicast and Multicast) Proposal Overview Our proposal can reduce traffic amount
Sends network topology information to the Sender-side switch Controller Group1 Group2 Group3 Creates packets containing the topology information Group4 VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM Sender Our proposal can reduce traffic amount in both upstream and sender-side links : Representative switch VM : Tenant A VM : The Other Tenant Proposal Overview Virtual Network Physical Network Controller PM PM PM
VM VM VM VM VM VM VM Controller PM PM PM PM PM PM PM PM Physical Network Key Features of Our Proposal
Grouping Virtual Switches Getting Topology Information Forwarding Mechanism Multicast Support Grouping Virtual Switches
Reduction of traffic amount Grouping virtual switches based on network distance Ex. server rack unit based Representative virtual switch Different switch can be elected for each flow Packet processing load can be distributed Group1 Server rack 5 1 2 3 4 Representative Key Features of Our Proposal
Grouping Virtual Switches Getting Topology Information Forwarding Mechanism Multicast Support Getting Topology Information
Using OpenFlow protocol Network topology information dl_dst = ff:ff:ff:ff:ff:ff Actions = CONTROLLER Database OFPT_PACKET_IN OFPT_FLOW_MOD OFPT_VENDOR Controller VM VM Tenant ID, IP address, MAC address, switch port number Key Features of Our Proposal
Grouping Virtual Switches Getting Topology Information Forwarding Mechanism Multicast Support Extracts the original packet
Packet Encapsulation Extracts the original packet Encapsulation Physical Network VM VM Receiver Sender Ethernet (Physical) IP Proposal UDP Physical Network Header Ethernet (Virtual) Payload Original Packet Packet Delivery within a Group
Analyzing the topology information 2 PM1 PM2 Representative Non Representative 1 3 Group1 VM1 VM2 VM3 VM4 Ethernet (Physical) IP (Physical) UDP (Physical) Proposal Ethernet (Virtual) Payload Addresses for PM2 Entry for VM1 Total length Tenant ID Type IPPM1 MACPM1 IPVM1 MACVM1 Output port Num IPPM2 MACPM2 IPVM3 MACVM3 Entry for VM3 Key Features of Our Proposal
Grouping Virtual Switches Getting Topology Information Forwarding Mechanism Multicast Support IP Multicast on Virtual Networks
Multicast routers are not required Virtual Network VM Manage Multicast Group VM VM VM VM VM VM Controller PM PM PM PM PM PM PM PM Physical Network How to Manage Multicast Groups
Controllers manage multicast groups Snooping IGMP packets IGMP Join message Actions = CONTROLLER Adds this VM to the multicast group Controller OFPT_FLOW_MOD OFPT_PACKET_IN VM VM VRRP (Multicast) Providing High-Availability function Virtual Network
Virtual IP address Multicast address Providing High-Availability function Virtual Network VM VM VM VM VM Multicast VM Backup Master Controller PM PM PM PM Physical Network OSPF (Multicast) Each router constructs routing tables Virtual Network
Multicast address OSPF Virtual Network VM VM VM OSPF Virtual Router VM VM Controller PM PM PM PM Physical Network Floodlight Controller
Implementation Open vSwitch Controller Virtual switches Open vSwitch 2.3.1 ofproto Get topology information datapath Deliver packets in groups Controllers Floodlight 0.90 Management function of topology information Userspace ofproto Kernel Module VM datapath Physical NIC OpenFlow Protocol Internal Services Core Services TE-Cast Applications App. Floodlight Controller Pros and Cons Pros Supporting Broadcast/Multicast on virtual networks ( Comparing with VXLAN ) Multicast routers are not required On-the-fly packet forwarding on virtual switches ( Comparing with the Source Unicast Replication ) Mitigating hot-spot links Cons Hop counts for packet delivery can increase The processing load of the controller increase Outline Backgrounds Related Work Proposal Evaluation 1
Network Virtualization 2 Related Work IP multicast in substrate networks Source-based Unicast Replication 3 Proposal 4 Evaluation Evaluation Comparing with Source Unicast Replication
Packet transfer delay Packet amounts in links Physical topology & machine specifications Physical Switch H 1GbE network 1GbE network Floodlight Mininet OS CentOS 6.6 (2.6.32) CPU Intel Core i GHz Memory 16GB OS CentOS 7.0 (3.10.0) CPU Intel Core i GHz Memory 16GB Network Environment Communications on tenant A Group1 Group2 Group3
Bridge Floodlight Bridge Bridge Bridge Group1 Group2 Group3 H1 H2 H3 H4 H5 H6 H7 H8 H9 H10 H11 H12 H13 H14 H15 H16 VM : Tenant A VM : The Other Tenant Result of Packet Transfer Delay
Floodlight Bridge Bridge Bridge Bridge Representative Non Representative H1 H2 H3 H4 H5 H6 H7 H8 H9 H10 H11 H12 H13 H14 H15 H16 Responder Responder Sender ARP response time Pattern 1: Default Responder: Host 1 Pattern 2: Source Unicast Replication, Proposal (Representative) Via controller Pattern 3: Proposal (Non Representative) Responder: Host 5 Via controller + Representative switch [packets] Delay [ms] Result of Packet Amounts in Links
Floodlight Bridge 1 12 19 Bridge Bridge Bridge 2 3 13 14 20 21 4 5 6 7 8 9 10 11 15 16 17 18 22 23 24 25 H1 H2 H3 H4 H5 H6 H7 H8 H9 H10 H11 H12 H13 H14 H15 H16 The number of BUM packets in each link Every host of tenant A transmits broadcast Pattern 1: Default Pattern 2: Source Unicast Replication Pattern 3: Proposal Relative packet amounts 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 ~ 25 Our proposal reduced traffic amount in upstream and sender-side links Ratio of reduced packets SUR 20% Proposal 44% There were no wasteful packets Conclusion Broadcast/Multicast in virtual networks
Waste of the Network Bandwidth Proposal: TE-Cast (Topology Embedded xCast) Supporting Broadcast/Multicast on virtual Networks Reducing packet amounts Realizing on-the-fly virtual switches Future work A cache mechanism of topology information Reducing the network delay Distribution of processing load of the controller Using distributed controllers Result of Packet Amounts in Links
SUR Proposal Bridge 1 12 19 Bridge Bridge Bridge 2 3 13 14 20 21 4 5 6 7 8 9 10 11 15 16 17 18 22 23 24 25 H1 H2 H3 H4 H5 H6 H7 H8 H9 H10 H11 H12 H13 H14 H15 H16 Relative packet amounts 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 ~ 25 Result of Packet Amounts in Links
SUR Proposal Bridge 1 12 19 Bridge Bridge Bridge 2 3 13 14 20 21 4 5 6 7 8 9 10 11 15 16 17 18 22 23 24 25 H1 H2 H3 H4 H5 H6 H7 H8 H9 H10 H11 H12 H13 H14 H15 H16 Relative packet amounts 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 ~ 25 Create Multicast Group
Supporting Multicast Controllers construct multicast groups Snooping IGMP packets Controller Create Multicast Group IGMP Join message OFPT_PACKET_IN VM VM 1 Match ! Entry 1 Entry 1 copy Entry 2 Entry n Entry list for Tenant A Multicast group 1