hyper-v network virtualization motivation & packet flows

Hyper-V Network VirtualizationMotivation & Packet Flows

Cost

Evolution of Clouds

Traditional Datacenters

with Dedicated Servers

Server Virtualization in

Datacenters

CloudPublicPrivateHybrid

Infrastructure Optimization

Flexibility

Servers

Any Service

Any Server

Any Cloud

Private Cloud Datacenter Consolidation

Sales

Finance

R&D

Sales

Finance

R&D

Hybrid Cloud: Seamless Datacenter Extension

Multi-Tenant Cloud Requirements

Contoso Bank Woodgrove Bank

Multiple customers on shared infrastructure

Finance Sales

Multiple business unitson shared infrastructure

Secure isolationDynamic service placementQoS & resource metering

Private Cloud

Public Cloud

Multi-Tenant Datacenter

Challenges in Building Clouds

• Physical location determines network address• IP address topology limits VM placement

Limited workload mobility

• Consolidate workloads to efficiently use CPU, storage, network• Limited VM placement leads to infrastructure

overprovisioning

Resource utilization

• Deploying VMs requires tight cooperation of server/network admins• Coordinating teams increases complexity and reduces

agility

Operational inefficiency

• VLANs not suited for dynamic cloud topologies• Reconfiguration of production switches increases risk

Scalable multi-tenancy

• VM IP addresses are entangled with security and access policies• Need to change IP addresses reduces cloud adoption

Onboarding

Datacenter Resource Utilization: Consolidation

Ideal: Consolidated

Typical: Fragmented

Resource Utilization: Flexibility and Growth

Ideal: Workloads placed anywhere and can dynamically grow and shrink without being constrained by the network

Dynamic VLAN Reconfiguration is Cumbersome

VLAN tags

ToR

AggregationSwitches

VMs

ToR

Topology limits VM placement and requires reconfiguration of production switches

To improve resource utilization on servers we virtualized them

Therefore…

Virtualize the Network!

Hyper-V Network Virtualization

Server VirtualizationRun multiple virtual serverson a physical serverEach VM has illusion it is running as a physical server

Hyper-V Network Virtualization

Run multiple virtual networks on a physical network Each virtual network has illusion it is running as a physical network

Blue VM Red VMVirtualization

PhysicalServer

Blue Network Red Network

PhysicalNetwork

Hyper-V Network Virtualization Benefits

To Workload Owners

• Seamless migration to the cloud

• Move n-tier topology to the cloud

• Preserve policies, VM settings, IP addresses

To Enterprises

• Private Cloud datacenter consolidation and efficiencies

• Extension of datacenter into hybrid cloud

• Incremental integration of acquired company network infrastructure

To Hosters

• Bring Your own IP• Bring Your

network topology• Scalable multi-

tenancy

To Private/Public Cloud Datacenter

Admins

• Flexible VM placement without reconfiguration

• Decoupling of server and network admin roles increases agility

Virtualization Policy

System Center

Virtualize Customer Addresses

Customer Address Space (CA)

Red2

Blue2

10.0.0.5

Red1

Blue1

10.0.0.5 10.0.0.7 10.0.0.7

Blue

10.0.0.5 192.168.4.11

10.0.0.7 192.168.4.22Red

10.0.0.5 192.168.4.11

10.0.0.7 192.168.4.22

Blue10.0.0.510.0.0.7

BlueCorp

RedCorp Red

10.0.0.510.0.0.7

Datacenter Network

Host 1 Host 2

Provider Address Space (PA)

192.168.4.22192.168.4.11

Blue

10.0.0.5192.168.4.

11

10.0.0.7192.168.4.

22Red

10.0.0.5192.168.4.

11

10.0.0.7192.168.4.

22

Blue

10.0.0.5192.168.4.

11

10.0.0.7192.168.4.

22Red

10.0.0.5192.168.4.

11

10.0.0.7192.168.4.

22

CA PA

Hyper-V Network Virtualization ConceptsCustomer VM Network

One or more virtual subnets forming an isolation boundary A customer may have multiple Customer VM Networks

e.g. Blue R&D and Blue Sales are isolated from each other

Virtual SubnetBroadcast boundary

Blue Corp Red Corp

Blue Subnet1

Blue Subnet3Blue Subnet2

Blue Subnet5

Blue Subnet4

Red Subnet2

Red Subnet1

Blue R&D Net Blue Sales Net Red HR Net

Hoster DatacenterCustomerVM Network

VirtualSubnet

Different subnets

Standards-Based Encapsulation - NVGREBetter network scalability by sharing PA among VMsExplicit Virtual Subnet ID for better multi-tenancy support

10.0.0.5 10.0.0.5 10.0.0.7 10.0.0.7

192.168.2.22 192.168.5.55

192.168.2.22192.168.5.55

10.0.0.5 10.0.0.7

GRE Key Blue Subnet

MAC

10.0.0.5 10.0.0.7

GRE Key Red Subnet

MAC192.168.2.22

192.168.5.55

10.0.0.510.0.0.7

10.0.0.510.0.0.7

10.0.0.5 10.0.0.7

10.0.0.510.0.0.7

Hyper-V Network Virtualization ArchitectureNetwork Virtualization is transparent to VMs

Management OS traffic is NOT virtualized; only VM traffic

Hyper-V Switch and Extensions operate in CA space

PA Y

CA Y

Datacenter

Host 1

VM2 VMY

Host 2

CA2

PA2

CA1

AA1

PA1

VM1

CAX

AAX

PAX

VMX

System Center

Blue• VM1: MAC1, CA1, PA1

• VM2: MAC2, CA2, PA3

• VM3: MAC3, CA3, PA5

• … Red• VM1: MACX, CA1, PA2

• VM2: MACY, CA2, PA4

• VM3: MACZ, CA3, PA6

• …

Data Center Policy

NIC

Management

Cluster Storage

Live Migration

NIC

Hyper-V Switch

VSID ACL IsolationSwitch Extensions

Host Network Stack

PA1

Network Virtualization

VM1 VM1

SystemCenterHost

Agent

Windows Server 2012 CA1 CA1

IP VirtualizationPolicy

EnforcementRouting

Packet Flow: Same Virtual Subnet Same Host

Packet Flow: Blue1 Sending to Blue2

192.168.4.11NIC

IP VirtualizationPolicy Enforcement

Routing

VSID ACL Enforcement


MACPA1

Blue1

10.0.0.5

VSID5001

Red1

10.0.0.5

VSID6001

where is 10.0.0.7 ?ARP for 10.0.0.7

Blue2

10.0.0.7

VSID5001

Red2

10.0.0.7

VSID6001

Hyper-V Switch broadcasts ARP to:1. All local VMs on VSID 50012. Network Virtualization filter

Hyper-V Switch

Same VSID :: Same Host

Use MACB2 for 10.0.0.7

Blue1 learns MAC of Blue2

Blue2 responds to ARP for IP 10.0.0.7 on VSID 5001

with Blue2 MAC

Packet Flow: Blue1 Sending to Blue2

192.168.4.11NIC


Routing



MACPA1

Blue1

10.0.0.5

VSID5001

Red1

10.0.0.5

VSID6001

Blue2

10.0.0.7

VSID5001

Red2

10.0.0.7

VSID6001

Hyper-V Switch


sent from Blue1

MACB1MACB2 10.0.0.5 10.0.0.7

OOB: VSID:5001

in Hyper-V switch

MACB1MACB2 10.0.0.5 10.0.0.7

Packet Flow: Blue2 Receiving

192.168.4.11NIC


Routing



MACPA1

Blue1

10.0.0.5

VSID5001

Red1

10.0.0.5

VSID6001

Blue2

10.0.0.7

VSID5001

Red2

10.0.0.7

VSID6001

Hyper-V Switch


OOB: VSID:5001

in Hyper-V switch

received by Blue2

MACB1MACB2 10.0.0.5 10.0.0.7

MACB1MACB2 10.0.0.5 10.0.0.7

Packet Flow: Same Virtual Subnet Different Hosts

Packet Flow: Blue1 Blue2where is 10.0.0.7 ?

ARP for 10.0.0.7


OOB: VSID:5001

Network Virtualization filter responds to ARP for IP 10.0.0.7 on VSID 5001

with Blue2 MAC

ARP for 10.0.0.7

ARP is NOT broadcast to the network

192.168.4.11NIC

Hyper-V Switch


Blue1 Red1


10.0.0.510.0.0.5

MACPA1

VSID5001

VSID6001


Routing

192.168.4.22NIC


MACPA2

Hyper-V Switch


Blue2 Red2

10.0.0.710.0.0.7

VSID5001

VSID6001


EnforcementRouting

Same VSID :: Different Host

Packet Flow: Blue1 Blue2

MACPA1


OOB: VSID:5001



Blue1 learns MAC of Blue2

192.168.4.11NIC

Hyper-V Switch


Blue1 Red1


10.0.0.510.0.0.5

MACPA1

VSID5001

VSID6001


Routing

192.168.4.22NIC


MACPA2

Hyper-V Switch


Blue2 Red2

10.0.0.710.0.0.7

VSID5001

VSID6001


EnforcementRouting



sent from Blue1

MACB1MACB2 10.0.0.5 10.0.0.7

OOB: VSID:5001

in Hyper-V switch

MACB1MACB2 10.0.0.5 10.0.0.7

in Network Virtualization filterOOB: VSID:5001

MACB1MACB2 10.0.0.5 10.0.0.7

NVGRE on the wireMACPA1 MACPA2 192.168.4.11 192.168.4.22 5001 MACB1MACB2 10.0.0.5 10.0.0.7

192.168.4.11NIC

Hyper-V Switch


Blue1 Red1


10.0.0.510.0.0.5

MACPA1

VSID5001

VSID6001


Routing

192.168.4.22NIC


MACPA2

Hyper-V Switch


Blue2 Red2

10.0.0.710.0.0.7

VSID5001

VSID6001


EnforcementRouting



received by Blue2

MACB1MACB2 10.0.0.5 10.0.0.7

OOB: VSID:5001

in Hyper-V switch

MACB1MACB2 10.0.0.5 10.0.0.7

NVGRE on the wire


MACB1MACB2 10.0.0.5 10.0.0.7

MACPA1 MACPA2 192.168.4.11 192.168.4.22 5001 MACB1MACB2 10.0.0.5 10.0.0.7

192.168.4.11NIC

Hyper-V Switch


Blue1 Red1


10.0.0.510.0.0.5

MACPA1

VSID5001

VSID6001


Routing

192.168.4.22NIC


MACPA2

Hyper-V Switch


Blue2 Red2

10.0.0.710.0.0.7

VSID5001

VSID6001


EnforcementRouting


Packet Flow: Different Virtual Subnet Same HostVSID 5001,5222 in same routing

domain


192.168.4.11NIC


Routing



MACPA1

Blue1

10.0.0.5

VSID5001

Red1

10.0.0.5

VSID6001

where is default gateway ?ARP for 10.0.0.1 (default gateway)

Blue2

10.0.1.7

VSID5222

Red2

10.0.0.7

VSID6001


Hyper-V Switch

Different VSID :: Same Host

OOB: VSID:5001

Network Virtualization filter responds to ARP with MACDGW

ARP for 10.0.0.1

MACDGW


192.168.4.11NIC


Routing



MACPA1

Blue1

10.0.0.5

VSID5001

Red1

10.0.0.5

VSID6001

Blue2

10.0.1.7

VSID5222

Red2

10.0.0.7

VSID6001

Hyper-V Switch


OOB: VSID:5001

Use MACDGW for 10.0.0.1

Default Gateway at MACDGW

Blue1 learns MAC of Default Gateway

MACDGW


192.168.4.11NIC


Routing



MACPA1

Blue1

10.0.0.5

VSID5001

Red1

10.0.0.5

VSID6001

Blue2

10.0.1.7

VSID5222

Red2

10.0.0.7

VSID6001

Hyper-V Switch


MACDGW

sent from Blue1

MACB1MACDGW 10.0.0.5 10.0.1.7

OOB: VSID:5001

in Hyper-V switch

MACB1MACDGW 10.0.0.5 10.0.1.7


MACB1MACDGW 10.0.0.5 10.0.1.7

Network Virtualization filter verifies Blue1 and Blue2 are in same routing domain,

otherwise packet is dropped


192.168.4.11NIC


Routing



MACPA1

Blue1

10.0.0.5

VSID5001

Red1

10.0.0.5

VSID6001

Blue2

10.0.1.7

VSID5222

Red2

10.0.0.7

VSID6001

Hyper-V Switch


MACDGW

received by Blue2

MACB1MACB2 10.0.0.5 10.0.1.7

OOB: VSID:5222

in Hyper-V switch

MACB1MACB2 10.0.0.5 10.0.1.7


MACB1MACB2 10.0.0.5 10.0.1.7

Network Virtualization filter usesVSID and dest MAC of Blue2

retains source MAC of Blue1

Packet Flow: Different Virtual Subnet Different HostsVSID 5001, 5222 in same routing

domain

Packet Flow: Blue1 Blue2where is default gateway ?

ARP for 10.0.0.1 (default gateway)


OOB: VSID:5001

Network Virtualization filter responds to ARP with MACDGW

ARP for 10.0.0.1


192.168.4.11NIC

Hyper-V Switch


Blue1 Red1


10.0.0.510.0.0.5

MACPA1

VSID5001

VSID6001


Routing

192.168.4.22NIC


MACPA2

Hyper-V Switch


Blue2 Red2

10.0.0.710.0.1.7

VSID5222

VSID6001


EnforcementRouting

Different VSID :: Different Host

MACDGW


MACPA1

OOB: VSID:5001

Use MACDGW for 10.0.0.1

Default Gateway at MACDGW

Blue1 learns MAC of Default Gateway

192.168.4.11NIC

Hyper-V Switch


Blue1 Red1


10.0.0.510.0.0.5

MACPA1

VSID5001

VSID6001


Routing

192.168.4.22NIC


MACPA2

Hyper-V Switch


Blue2 Red2

10.0.0.710.0.1.7

VSID5222

VSID6001


EnforcementRouting

MACDGW



sent from Blue1

MACB1MACDGW 10.0.0.5 10.0.1.7

OOB: VSID:5001

in Hyper-V switch

MACB1MACDGW 10.0.0.5 10.0.1.7


MACB1MACDGW 10.0.0.5 10.0.1.7

NVGRE on the wireMACPA1 MACPA2 192.168.4.11 192.168.4.22 5222 MACB1MACB2 10.0.0.5 10.0.1.7

192.168.4.11NIC

Hyper-V Switch


Blue1 Red1


10.0.0.510.0.0.5

MACPA1

VSID5001

VSID6001


Routing

192.168.4.22NIC


MACPA2

Hyper-V Switch


Blue2 Red2

10.0.0.710.0.1.7

VSID5222

VSID6001


EnforcementRouting

MACDGW


5222


received by Blue2

MACB1MACB2 10.0.0.5 10.0.1.7

OOB: VSID:5222

in Hyper-V switch

MACB1MACB2 10.0.0.5 10.0.1.7

NVGRE on the wire


MACB1MACB2 10.0.0.5 10.0.1.7

MACPA1 MACPA2 192.168.4.11 192.168.4.22 5222 MACB1MACB2 10.0.0.5 10.0.1.7

192.168.4.11NIC

Hyper-V Switch


Blue1 Red1


10.0.0.510.0.0.5

MACPA1

VSID5001

VSID6001


Routing

192.168.4.22NIC


MACPA2

Hyper-V Switch


Blue2 Red2

10.0.0.710.0.1.7

VSID5222

VSID6001


EnforcementRouting

MACDGW


Private Cloud

Private CloudIP addresses

VMs and CorpNet running 10.229.xDatacenter has 10.60.x PA addresses

Hyper-V Network Virtualization Gateway bridges network virtualized environment with non-network virtualized environment

Hyper-V Network

Virtualization

Gateway

DC SQL DNS

CorpNet

subnet 10.229.203.x

subnet 10.229.202.x

subnet 10.229.201.x

subnet 10.229.200.x

R1 R2B1 B2 B3 R3 R4Y1 Y2

10.60.x

Consolidated Datacenter Hyper-V Network Virtualization

Host1 Host2 Host3

Hybrid Cloud

Hybrid CloudWith Hyper-V Network Virtualization and on-premises Site-to-Site VPN on-premise resources seamlessly extended to the cloud

Blue Corp

S2S VPN

HostHost

Hoster DatacenterNetwork Virtualization Fabric

Web2R2

R1

Web3

Web1

Hyper-V Network

Virtualization

Gateway

DC

SQL

DNS

Red Corp

S2S VPN

Inte

rnet

Blue Private Cloud

Additional ResourcesHyper-V Network Virtualization Whitepaper

http://technet.microsoft.com/en-us/library/jj134230.aspx

Hyper-V Network Virtualization Blog Entryhttp://blogs.technet.com/b/windowsserver/archive/2012/04/16/introducing-windows-server-8-hyper-v-network-virtualization-enabling-rapid-migration-and-workload-isolation-in-the-cloud.aspx

Hyper-V Network Virtualization Survival Guidehttp://social.technet.microsoft.com/wiki/contents/articles/11524.windows-server-2012-hyper-v-network-virtualization-survival-guide.aspx

PowerShell ScriptsSimple deployment

http://gallery.technet.microsoft.com/scriptcenter/Simple-Hyper-V-Network-d3efb3b8

Simple gatewayhttp://gallery.technet.microsoft.com/scriptcenter/Simple-Hyper-V-Network-6928e91b

http://technet.microsoft.com/en-us/library/jj134230.aspx

http://blogs.technet.com/b/windowsserver/archive/2012/04/16/introducing-windows-server-8-hyper-v-network-virtualization-enabling-rapid-migration-and-workload-isolation-in-the-cloud.aspx



http://social.technet.microsoft.com/wiki/contents/articles/11524.windows-server-2012-hyper-v-network-virtualization-survival-guide.aspx





http://gallery.technet.microsoft.com/scriptcenter/Simple-Hyper-V-Network-6928e91b

http://gallery.technet.microsoft.com/scriptcenter/Simple-Hyper-V-Network-6928e91b

© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to

be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS

PRESENTATION.

hyper-v network virtualization motivation & packet flows

Documents