open vswitch and the intelligent edge · an intelligent edge • we view the hypervisor as the edge...
TRANSCRIPT
![Page 1: Open vSwitch and the Intelligent Edge · An Intelligent Edge • We view the hypervisor as the edge of the network • An intelligent edge is in a unique position (the “Goldilocks](https://reader030.vdocuments.mx/reader030/viewer/2022041004/5ea7eb2cc73c4f3d7c1ccbd5/html5/thumbnails/1.jpg)
© 2014 VMware Inc. All rights reserved.
Open vSwitch and the Intelligent Edge Justin Pettit OpenStack 2014 Atlanta
![Page 2: Open vSwitch and the Intelligent Edge · An Intelligent Edge • We view the hypervisor as the edge of the network • An intelligent edge is in a unique position (the “Goldilocks](https://reader030.vdocuments.mx/reader030/viewer/2022041004/5ea7eb2cc73c4f3d7c1ccbd5/html5/thumbnails/2.jpg)
Hypervisor as Edge
2
Open vSwitch
VM1
Hypervisor
VM2 VM3
![Page 3: Open vSwitch and the Intelligent Edge · An Intelligent Edge • We view the hypervisor as the edge of the network • An intelligent edge is in a unique position (the “Goldilocks](https://reader030.vdocuments.mx/reader030/viewer/2022041004/5ea7eb2cc73c4f3d7c1ccbd5/html5/thumbnails/3.jpg)
An Intelligent Edge • We view the hypervisor as the edge of the network • An intelligent edge is in a unique position (the “Goldilocks Zone”)
– Greater context than in-network devices • Without tags, network must rely on fields that are easily spoofed • Tags provide limited amount of context
– Reduced risk of attack than an agent running in the guest • Policies enforced in the hypervisor – outside of the guest
– Enforce policies earlier • Clouds typically have over-subscribed links and untrusted sources
• Different parts of the system can coordinate with each other
• Can affect many things – Networking – Security
![Page 4: Open vSwitch and the Intelligent Edge · An Intelligent Edge • We view the hypervisor as the edge of the network • An intelligent edge is in a unique position (the “Goldilocks](https://reader030.vdocuments.mx/reader030/viewer/2022041004/5ea7eb2cc73c4f3d7c1ccbd5/html5/thumbnails/4.jpg)
Network Control and Visibility • In an ideal location • Able to infer state by observing, or probe state with introspection
• Mapping of logical to physical before going into the fabric • Can modify behavior
– Enforce policy at tunnel ingress and egress – Modify bits in the inner or outer packet – TCP Pacing – TCP De-synchronization – Flowlets
4
![Page 5: Open vSwitch and the Intelligent Edge · An Intelligent Edge • We view the hypervisor as the edge of the network • An intelligent edge is in a unique position (the “Goldilocks](https://reader030.vdocuments.mx/reader030/viewer/2022041004/5ea7eb2cc73c4f3d7c1ccbd5/html5/thumbnails/5.jpg)
Inferring State • Sees every packet and knows local source
– Learn MAC and IP on first use – IGMP and DHCP snooping – Which pairs are communicating – Flow characteristics
5
![Page 6: Open vSwitch and the Intelligent Edge · An Intelligent Edge • We view the hypervisor as the edge of the network • An intelligent edge is in a unique position (the “Goldilocks](https://reader030.vdocuments.mx/reader030/viewer/2022041004/5ea7eb2cc73c4f3d7c1ccbd5/html5/thumbnails/6.jpg)
Guest Introspection • An agent runs in the VM that communicates with a daemon in the hypervisor • Types of data retrieved
– Users – Identity for both inbound and outbound network connections – Identity (user and version/hash) of processes – Data transfer rates – Socket queue depth – System characteristics
6
![Page 7: Open vSwitch and the Intelligent Edge · An Intelligent Edge • We view the hypervisor as the edge of the network • An intelligent edge is in a unique position (the “Goldilocks](https://reader030.vdocuments.mx/reader030/viewer/2022041004/5ea7eb2cc73c4f3d7c1ccbd5/html5/thumbnails/7.jpg)
Applications for Greater State • QoS • Load-balancing
• Selecting traffic to be sent to middlebox (NFV) • Better firewalls
• Elephant flow detection and handling
7
![Page 8: Open vSwitch and the Intelligent Edge · An Intelligent Edge • We view the hypervisor as the edge of the network • An intelligent edge is in a unique position (the “Goldilocks](https://reader030.vdocuments.mx/reader030/viewer/2022041004/5ea7eb2cc73c4f3d7c1ccbd5/html5/thumbnails/8.jpg)
Security
![Page 9: Open vSwitch and the Intelligent Edge · An Intelligent Edge • We view the hypervisor as the edge of the network • An intelligent edge is in a unique position (the “Goldilocks](https://reader030.vdocuments.mx/reader030/viewer/2022041004/5ea7eb2cc73c4f3d7c1ccbd5/html5/thumbnails/9.jpg)
Implementing a Firewall • Currently, two ways to implement a firewall in OVS
– Match on TCP flags (Enforce policy on SYN, allow ACK|RST) • Pro: Fast • Con: Allows non-established flow through with ACK or RST set, only TCP
– Use “learn” action to setup new flow in reverse direction • Pro: More “correct” • Con: Forces every new flow to OVS userspace, reducing flow setup by orders of magnitude
– Neither approach supports “related” flows or TCP window enforcement
9
![Page 10: Open vSwitch and the Intelligent Edge · An Intelligent Edge • We view the hypervisor as the edge of the network • An intelligent edge is in a unique position (the “Goldilocks](https://reader030.vdocuments.mx/reader030/viewer/2022041004/5ea7eb2cc73c4f3d7c1ccbd5/html5/thumbnails/10.jpg)
Connection Tracking • We are adding the ability to use the conntrack module from Linux
– Stateful tracking of flows – Supports ALGs to punch holes for related “data” channels
• FTP • TFTP • SIP
• Implement a distributed firewall with enforcement at the edge – Better performance – Better visibility
• Introduce new OpenFlow extensions: – Action to send to conntrack – Match fields on state of connection
• Have prototype working. Expect to ship as part of OVS by end of year
10
![Page 11: Open vSwitch and the Intelligent Edge · An Intelligent Edge • We view the hypervisor as the edge of the network • An intelligent edge is in a unique position (the “Goldilocks](https://reader030.vdocuments.mx/reader030/viewer/2022041004/5ea7eb2cc73c4f3d7c1ccbd5/html5/thumbnails/11.jpg)
Guest Introspection + Connection Tracking • Possible to implement an advanced firewall
– Know precisely what user is generating traffic – Know precisely what application and version is generating traffic
11
![Page 12: Open vSwitch and the Intelligent Edge · An Intelligent Edge • We view the hypervisor as the edge of the network • An intelligent edge is in a unique position (the “Goldilocks](https://reader030.vdocuments.mx/reader030/viewer/2022041004/5ea7eb2cc73c4f3d7c1ccbd5/html5/thumbnails/12.jpg)
Elephant Flows
![Page 13: Open vSwitch and the Intelligent Edge · An Intelligent Edge • We view the hypervisor as the edge of the network • An intelligent edge is in a unique position (the “Goldilocks](https://reader030.vdocuments.mx/reader030/viewer/2022041004/5ea7eb2cc73c4f3d7c1ccbd5/html5/thumbnails/13.jpg)
Elephants versus Mice • Majority of flow are short-lived (mice), but majority of packets are long-lived (elephants) • Mice tend to be bursty and latency-sensitive
• Elephants tend to transfer large amount of data and less concerned about latency • Elephants can fill up network buffers, which introduce latency for mice
• At the edge, we are able to affect the underlay based on the overlay
![Page 14: Open vSwitch and the Intelligent Edge · An Intelligent Edge • We view the hypervisor as the edge of the network • An intelligent edge is in a unique position (the “Goldilocks](https://reader030.vdocuments.mx/reader030/viewer/2022041004/5ea7eb2cc73c4f3d7c1ccbd5/html5/thumbnails/14.jpg)
Detection and Action • Multiple mechanisms for detection:
– Rate and time – Large segments (TCP only) – Guest introspection
• Multiple mechanisms for action: – Put mice and elephants into different queues – Route elephants differently from mice – Send elephants along a separate physical network – Intelligent underlay
14
![Page 15: Open vSwitch and the Intelligent Edge · An Intelligent Edge • We view the hypervisor as the edge of the network • An intelligent edge is in a unique position (the “Goldilocks](https://reader030.vdocuments.mx/reader030/viewer/2022041004/5ea7eb2cc73c4f3d7c1ccbd5/html5/thumbnails/15.jpg)
NSX Deployment
15
Open vSwitch
VM1 192.168.0.1
HV1 17.0.0.1
Open vSwitch
VM2 192.168.0.2
HV2 17.0.0.2 VXLAN Tunnel
NSX Control Cluster
![Page 16: Open vSwitch and the Intelligent Edge · An Intelligent Edge • We view the hypervisor as the edge of the network • An intelligent edge is in a unique position (the “Goldilocks](https://reader030.vdocuments.mx/reader030/viewer/2022041004/5ea7eb2cc73c4f3d7c1ccbd5/html5/thumbnails/16.jpg)
Handling Elephants in NSX • Open vSwitch is at an optimal location at the edge
– Has flow-level view of all the hypervisor’s traffic – Knows mapping between logical and physical addresses
• Detection and action occur separately, so can evolve independently
• Supported detection mechanisms: – Rate and time – Large segments
• Supported actions: – Mark DSCP bits in (outer) IP header – Add elephant flows to OVSDB column for underlay agent
16
![Page 17: Open vSwitch and the Intelligent Edge · An Intelligent Edge • We view the hypervisor as the edge of the network • An intelligent edge is in a unique position (the “Goldilocks](https://reader030.vdocuments.mx/reader030/viewer/2022041004/5ea7eb2cc73c4f3d7c1ccbd5/html5/thumbnails/17.jpg)
Elephant Flows with SDN Controller • OVS identifies elephants as the appear on the wire through OVSDB • An agent monitors OVSDB and makes appropriate API calls to the SDN controller
• Shown as a VMware-HP Technology Preview
17
HP SDN Controller
HP Switch 1 HP Switch 2 HV1 HV2
NSX Elephant Agent
![Page 18: Open vSwitch and the Intelligent Edge · An Intelligent Edge • We view the hypervisor as the edge of the network • An intelligent edge is in a unique position (the “Goldilocks](https://reader030.vdocuments.mx/reader030/viewer/2022041004/5ea7eb2cc73c4f3d7c1ccbd5/html5/thumbnails/18.jpg)
Elephant Flows with DSCP Marking • Signaling of elephants occur at the hypervisor by marking the (outer) IP header • Switches configured to handle elephant-marked packets appropriately
• Working on an Internet Draft for recommended DSCP values
18
Switch 1 Switch 2 HV1 HV2
![Page 19: Open vSwitch and the Intelligent Edge · An Intelligent Edge • We view the hypervisor as the edge of the network • An intelligent edge is in a unique position (the “Goldilocks](https://reader030.vdocuments.mx/reader030/viewer/2022041004/5ea7eb2cc73c4f3d7c1ccbd5/html5/thumbnails/19.jpg)
Testing Results with Cumulus Networks • Used a modified OVS that detects elephant flows by counting the number of bytes each flow
generates. When the user-configurable threshold is crossed, elephants are marked with a particular DSCP value.
• The Cumulus switches place elephant marked flows into an alternate queue
19
![Page 20: Open vSwitch and the Intelligent Edge · An Intelligent Edge • We view the hypervisor as the edge of the network • An intelligent edge is in a unique position (the “Goldilocks](https://reader030.vdocuments.mx/reader030/viewer/2022041004/5ea7eb2cc73c4f3d7c1ccbd5/html5/thumbnails/20.jpg)
Test Topology • Sources
– VMs connected via vSwitch • 10G connection to network
• Network Paths – 1G “normal” link
• easy to congest with VM traffic sources
– 10G “alternative” link
• Sink – bare metal server
• 10G connection from network
cumulusnetworks.com 20
1G
vswitch
n
vm
vnic
10G
10G
10G
vm
vnic
vm
vnic
![Page 21: Open vSwitch and the Intelligent Edge · An Intelligent Edge • We view the hypervisor as the edge of the network • An intelligent edge is in a unique position (the “Goldilocks](https://reader030.vdocuments.mx/reader030/viewer/2022041004/5ea7eb2cc73c4f3d7c1ccbd5/html5/thumbnails/21.jpg)
Traffic Generation and Result Measurement • Generators
– elephants – nuttcp • fixed time transfers, 4M window
– mice – small (10ms) interval pings • mimics tcp-acks, lock release, small db transations
• Results – elephants
• realized bandwidth, drops
– mice • mean-time-to-completion, drops
cumulusnetworks.com 21
![Page 22: Open vSwitch and the Intelligent Edge · An Intelligent Edge • We view the hypervisor as the edge of the network • An intelligent edge is in a unique position (the “Goldilocks](https://reader030.vdocuments.mx/reader030/viewer/2022041004/5ea7eb2cc73c4f3d7c1ccbd5/html5/thumbnails/22.jpg)
Results – flow statistic detection & alternate queue reaction
22
0
1
2
3
4
5
6
7
8
9
10
500
550
600
650
700
750
800
850
900
950
1000
1 11 21 31 41 51 61 71 81 91 101 111 121 131
Late
ncy
ms)
Ban
dwid
th (M
bps)
Time (Secs)
Mice vs Elephants (Detection off)
Elephant
Mice
cumulusnetworks.com
![Page 23: Open vSwitch and the Intelligent Edge · An Intelligent Edge • We view the hypervisor as the edge of the network • An intelligent edge is in a unique position (the “Goldilocks](https://reader030.vdocuments.mx/reader030/viewer/2022041004/5ea7eb2cc73c4f3d7c1ccbd5/html5/thumbnails/23.jpg)
Results – flow statistic detection & alternate queue reaction
23
0
1
2
3
4
5
6
7
8
9
10
500
550
600
650
700
750
800
850
900
950
1000
1 11 21 31 41 51 61 71 81 91 101 111 121 131
Late
ncy
(ms)
Ban
dwid
th (M
bps)
Time (Secs)
Mice vs Elephants (Detection on)
Elephant
Mice
cumulusnetworks.com
![Page 24: Open vSwitch and the Intelligent Edge · An Intelligent Edge • We view the hypervisor as the edge of the network • An intelligent edge is in a unique position (the “Goldilocks](https://reader030.vdocuments.mx/reader030/viewer/2022041004/5ea7eb2cc73c4f3d7c1ccbd5/html5/thumbnails/24.jpg)
Results – flow statistic detection & alternate queue reaction
test case (120 sec period)
elephant mouse
Mbps drops Latency (ms) drops
elephant only 941 63 N/A N/A
mouse only N/A N/A 0.444 0
mouse vs elephant no detection 941 61 3.055 0
mouse vs elephant w/detection 937 1223 0.401 0
24
cumulusnetworks.com
![Page 25: Open vSwitch and the Intelligent Edge · An Intelligent Edge • We view the hypervisor as the edge of the network • An intelligent edge is in a unique position (the “Goldilocks](https://reader030.vdocuments.mx/reader030/viewer/2022041004/5ea7eb2cc73c4f3d7c1ccbd5/html5/thumbnails/25.jpg)
Open vSwitch Elephant POC Architecture • Implemented in kernel • Supports both threshold-based detection and TSO packet size
• Just proof of concept to try out different detection mechanisms and actions • Proof of concept code will be available on Github
25
![Page 26: Open vSwitch and the Intelligent Edge · An Intelligent Edge • We view the hypervisor as the edge of the network • An intelligent edge is in a unique position (the “Goldilocks](https://reader030.vdocuments.mx/reader030/viewer/2022041004/5ea7eb2cc73c4f3d7c1ccbd5/html5/thumbnails/26.jpg)
Elephant Flow References • Network Traffic Characteristics of Data Centers in the Wild
– http://pages.cs.wisc.edu/~akella/papers/dc-meas-imc10.pdf
• Of Mice and Elephants – http://networkheresy.com/2013/11/01/of-mice-and-elephants/
• Elephant Flow Mitigation via Virtual-Physical Communication – http://blogs.vmware.com/networkvirtualization/2014/02/elephant-flow-mitigation.html
26
![Page 27: Open vSwitch and the Intelligent Edge · An Intelligent Edge • We view the hypervisor as the edge of the network • An intelligent edge is in a unique position (the “Goldilocks](https://reader030.vdocuments.mx/reader030/viewer/2022041004/5ea7eb2cc73c4f3d7c1ccbd5/html5/thumbnails/27.jpg)
Monday VMware Demo 1:00-1:15 pm, Demo Theater
Enterprise Grade Scheduling 4:40-5:20 pm, B206
Bridging The Gap: OpenStack For VMware Administrators 5:30-6:10 pm, B206
Software Defined Networking Performance And Architecture Evaluation 5:30-6:10 pm, B103 Presented by Symantec & Mirantis
Learn more about VMware + OpenStack at the following sessions:
Tuesday
Scaling Neutron For Large Deployments 4:40-5:20 pm, B101 Presented by eBay & PayPal
Open vSwitch And The Intelligent Edge 5:30-6:10 pm, B206
Wednesday VMware + OpenStack: Accelerating OpenStack In The Enterprise 1:50-2:30 pm, B313
Deep-dive Demo for OpenStack On VMware 2:40-3:20 pm, B313
OpenStack Distribution Support For vSphere + NSX 3:30-4:10 pm, B313
Congress: A System For Declaring, Auditing, and Enforcing Policy In Heterogeneous Cloud Environments 4:30-5:10 pm, B313
VSAN and OpenStack 5:20-6:00 pm, B313
Thursday Recap: Nova-network Or Neutron For OpenStack Networking? 9:50-10:30 am, B309
Leveraging VMware Technology To Build An Enterprise Grade OpenStack Cloud - It's Not Always About KVM! 2:20-3:00 pm, B101 Presented by iLand
Session by VMware Customers / Partners
Session by VMware
Hands-on-Labs OpenStack on VMware vSphere and NSX Wed, May 14, 3:30-5:30 pm, B313
OpenStack Networking Wed, May 14, 4:30-6:00 pm, B314
The Enterprise-Grade Foundation For Your OpenStack Cloud