software defined networking in apache cloudstack

Software Defined Networking in Apache CloudStack

Chiradeep VittalCloudStack Committer

@chiradeep

Agenda

• Introduction to CloudStack and IAAS• What is SDN• Why SDN and IAAS?• CloudStack’s Network Model• Extensible Networking in CloudStack• SDN integrations in CloudStack• CloudStack’s native SDN approach• Future

•History• Incubating in the Apache

Software Foundation since April 2012

•Open Source since May 2010

• In production since 2009•Tons of deployments,

including large-scale commercial ones

Apache CloudStack

Build your cloud the way the world’s most

successful clouds are built

How did Amazon build its cloud?

Commodity Servers

Commodity StorageNetworking

Open Source Xen Hypervisor

Amazon Orchestration Software

AWS API (EC2, S3, …)

Amazon eCommerce Platform

How can YOU build a cloud?

Servers StorageNetworking

Open Source Xen Hypervisor

Amazon Orchestration Software

AWS API (EC2, S3, …)

Amazon eCommerce Platform

Hypervisor (Xen/KVM/VMW/)

CloudStack Orchestration Software

Optional Portal

CloudStack or AWS API

SDN Definition

• Separation of Control Plane from the hardware performing the forwarding function

• Control plane is logically centralized

SDN Advantages

• Centralized control makes it easier to configure, troubleshoot and maintain

• Eliminates ‘box’ mode of configuration

• Enables control at a high level

Related to SDN

• API layer over a collection of ‘boxes’– API layer communicates with boxes using box-

level APIs / ssh / telnet• OpenFlow

– Standard protocol for the centralized control plane to talk to the forwarding elements.

• Tunnels / overlays– SDN is valuable for virtual topologies– Initial target of SDN implementation

Centralized control plane

MySQL/NoSQL

Controller Cluster API

Boxes

Openflow/ssh/netconf/other

Defining Cloud Computing (IAAS)

• Agility– Re-provision complex infrastructure topologies

in minutes, not days• API

– Automate complex infrastructure tasks• Virtualization

– Enables workload mobility and load sharing• Multi-tenancy

– Share resources and costs

Defining Cloud Computing (IAAS)

• Scalability– Ability to consume resources limited by

budget, not by infrastructure• Elasticity

– Scale up and down on demand– Reduce need to engineer for peak load

• Self-service– No IT assistance

Cloud Networking Requirements

• Agile– Complex networking topologies created by

non-network engineers• API

– Language to talk with the network infrastructure layer (not CLI)

• Virtualization– Hypervisor-level switches work together with

physical infrastructure

Cloud Networking Requirements

• Scalability– Usually means L3 in the physical infrastructure

• Elasticity– Release resources when not in use– Introduce new resources on demand

• Self-service– Novices deploying, maintaining,

troubleshooting virtual networks

IAAS + SDN – made for each other

• SDN enables agility– API to controller enables easy changes to networks

• SDN works with virtualization / vSwitches– Typical of most SDN controllers

• SDN controllers are designed for large scale• SDN enables virtual networking

– The illusion of isolated networks on top of shared physical infrastructure

SDN issues

• Discovery of virtual address -> physical address mapping– VxLAN = multicast– GRE = programmed by control plane– L3 isolation = no mapping, no discovery

SDN issues

• State maintenance– Large number of endpoints + flows– High arrival rate of new flows– Needs fast and scalable storage and

processing– Differentiator between vendors

SDN issues

• L4-L7– Service insertion and orchestration– How do endpoints get services such as

• Firewall• Load balancers• IDS/IPS

– Service levels and performance– Service Chaining

Network Virtualization in IAAS

Tenant 1 VM

1

Tenant 1 VM

2

Tenant 1 VM

3

Tenant 1 VM

4

Tenant 1 Virtual Network 10.1.1.0/24

Gateway address 10.1.1.1

10.1.1.2

10.1.1.3

10.1.1.4

10.1.1.5

Internet

Network Virtualization in IAAS

Tenant 1 VM

1

Tenant 1 VM

2

Tenant 1 VM

3

Tenant 1 VM

4

Public Network

Tenant 1 Virtual Network 10.1.1.0/24

Gateway address 10.1.1.1

NATDHCPFW

Public IP address 65.37.141.1165.37.141.36

10.1.1.2

10.1.1.3

10.1.1.4

10.1.1.5

Tenant 1 Edge

Services Appliance(s)Interne

t

Network Virtualization in IAAS

Tenant 1 VM

1

Tenant 1 VM

2

Tenant 1 VM

3

Tenant 1 VM

4

Public Network

Tenant 1 Virtual Network 10.1.1.0/24

Gateway address 10.1.1.1

NATDHCPFW

Public IP address 65.37.141.1165.37.141.36

10.1.1.2

10.1.1.3

10.1.1.4

10.1.1.5

Tenant 1 Edge

Services Appliance(s)Interne

t

Tenant 1 Edge

Services Appliance(s)

Load BalancingVPN

Network Virtualization in IAAS

Internet

Tenant 1 VM

1

Tenant 1 VM

2

Tenant 1 VM

3

Tenant 1 VM

4

Public Network

Tenant 1 Virtual Network 10.1.1.0/24

Gateway address 10.1.1.1

NATDHCPFW

Public IP address 65.37.141.1165.37.141.36

10.1.1.2

10.1.1.3

10.1.1.4

10.1.1.5

Tenant 1 Edge

Services Appliance(s)

Tenant 2 VM

2

Tenant 2 VM

3

Tenant 2 VM

1

Tenant 2 Virtual Network 10.1.1.0/24

Gateway address 10.1.1.1

VPNNATDHCP

10.1.1.2

10.1.1.3

10.1.1.4

Tenant 2 Edge

Services Appliance

Public IP address 65.37.141.2465.37.141.80

Tenant 1 Edge

Services Appliance(s)

Load Balancing

Tenant 1

VM 1Tenan

t 1 VM 2Tenan

t 1 VM 3Tenan

t 1 VM 4

Public Network

Tenant 1 Virtual Network 10.1.1.0/24

Gateway address 10.1.1.1

NATDHCPFW

Public IP address 65.37.141.1165.37.141.36

10.1.1.2

10.1.1.3

10.1.1.4

10.1.1.5

Tenant 1 Edge

Services Appliance(s

)

Tenant 2

VM 2Tenan

t 2 VM 3

Tenant 2

VM 1

Tenant 2 Virtual Network 10.1.1.0/24

Gateway address 10.1.1.1

VPNNATDHCP

10.1.1.2

10.1.1.3

10.1.1.4

Tenant 2 Edge

Services Appliance

Public IP address 65.37.141.2465.37.141.80

Tenant 1 Edge

Services Appliance(s

)Load Balancing

CloudStack Network Model

• Map virtual networks to physical infrastructure

• Define and provision network services in virtual networks

• Manage elasticity and scale of network services

CloudStack Network Model: Network Services

Network Services

• L2 connectivity• IPAM• DNS• Routing• ACL• Firewall• NAT• VPN• LB• IDS• IPS

CloudStack Network Model: Network Services

Network Services

• L2 connectivity• IPAM• DNS• Routing• ACL• Firewall• NAT• VPN• LB• IDS• IPS

Service Providers

Virtual appliances

Hardware firewalls

LB appliances

SDN controllers

IDS /IPS appliances

VRF Hypervisor

CloudStack Network Model: Network Services

Network Services

• L2 connectivity• IPAM• DNS• Routing• ACL• Firewall• NAT• VPN• LB• IDS• IPS

Network Isolation

• No isolation• VLAN

isolation• Overlays• L3 isolation

Service Providers

Virtual appliances

Hardware firewalls

LB appliances

SDN controllers

IDS /IPS appliances

VRF Hypervisor

Service Catalog

• Cloud users are not exposed to the nature of the service provider

• Cloud operator designs a service catalog and offers them to end users.– Gold = {LB + FW, using virtual appliances}– Platinum = {LB + FW + VPN, using hardware

appliances}– Silver = {FW using virtual appliances, 10Mbps}

Service Catalog examples

10.1.1.0/24VLAN 100

10.1.1.1

DHCP, DNSNATLoad BalancingVPN

10.1.1.2

VM 1

10.1.1.3

VM 2

10.1.1.4

VM 3

10.1.1.5

VM 4

CSVirtual Router

L2 network with software appliances

65.37.141.11165.37.141.112

10.1.1.0/24VLAN 100

DHCP, DNS

CSVirtual Router

10.1.1.11265.37.141.112

10.1.1.2

VM 1

10.1.1.3

VM 2

10.1.1.4

VM 3

10.1.1.5

VM 4

Netscaler

Load Balancer

10.1.1.165.37.141.111 Juniper

SRXFirewall

L2 network with hardware appliances

NAT, VPN

Upgrade

Multi-tier virtual networking

Virtual appliance/Hardware Devices

Customer

Premises

IPSec or SSL site-to-site VPN

Internet

Network Services• IPAM• DNS• LB [intra]• S-2-S VPN• Static Routes• ACLs• NAT, PF• FW [ingress & egress]

Loadbalancer (virtual or HW)

MPLS VLAN

Web VM 1

Web VM 2

Web VM 3

Web VM 4

Web subnet 10.1.1.0/24VLAN 101

App subnet 10.1.2.0/24

App VM 1

App VM 2

VLAN 353

DB Subnet10.1.3.0/24

DB VM 1

VLAN 2724

Orchestration

• Orchestration describes the automated arrangement, coordination, and management of complex computer systems, middleware and services– Wikipedia

CloudStack Architecture

Orchestration Core

PluginFramework

Hypervisor Plugins

Hypervisor Plugins

Network PluginsNetwork Plugins

Allocator Plugins

Allocator Plugins

Storage Plugins

CloudStack Architecture

Orchestration Core

PluginFramework

Hypervisor Plugins

Hypervisor Plugins

Network PluginsNetwork Plugins

Allocator Plugins

Allocator Plugins

•XenServer•VMWare•KVM•OracleVM

•Random•User-concentrated•Intel TXT•Affinity

•Nicira•Netscaler•Brocade•MidoNet

CloudStack Orchestration

Orchestration Core

PluginFramework

Hypervisor PluginsHypervisor Plugins

Network PluginsNetwork Plugins

Allocator PluginsStorage Plugins

APIAPI

API

StorageResource

Physical Resources

StorageResource

NetworkResourceNetwork

Resource

HypervisorResourceHypervisor

Resource

Allocator PluginsAllocatorPlugins

1 2

3

45

6

7

8

9

Orchestration steps can be executed in parallel or in sequence

CloudStack and SDN

Orchestration core

PluginFramework

Hypervisor PluginsHypervisor Plugins

Network PluginsNetwork Plugins

Allocator PluginsStorage Plugins

APIAPI

API

StorageResource

Physical Resources

StorageResource

NetworkResourceSDN

controller

HypervisorResourceHypervisor

Resource

Allocator PluginsAllocatorPlugins

1 2

3

45

6

7

8

9

Network plugin is the glue that understands the SDN controller’s API

CloudStack SDN Integration• Nicira NVP

– L2 (STT) isolation in 4.0– Source NAT / Logical Router in 4.2

• BigSwitch– VLAN isolation in 4.1– VNS in 4.2

• Midokura– L2-L4 network virtualization– Coming in 4.2

• CloudStack Native– Tech preview (since 4.0)– Requires XenServer

Orchestration core

PluginFramework

Hypervisor PluginsHypervisor Plugins

Network PluginsNetwork Plugins

Allocator PluginsStorage Plugins

API AP

IAPI

StorageResourceStorage

Resource

NetworkResourceSDN controller

HypervisorResourceHypervisor

Resource

Allocator PluginsAllocatorPlugins

VM 1

VM3 VR

Host 1 Host 3

Host 4

VM2

Host 2

Start 3 VMs

Allocate hypervisors

VM Orchestration ExampleCall Hypervisor APIs

Built-in (native) controller

Host 1 (Pod 2)

Host 2 (Pod 4)

Host 3 (Pod 3)

Host 4 (Pod 2)

Create Full Mesh of GRE tunnels (if they don't already exist) between hosts on which VMs are deployed

CloudStack SDN controller programs the Open vSwitch (OVS) on XenServer to configure GRE tunnelsGRE Tunnel

GRE Tunnel GRE Tunnel

VM 1

VM2

VM3 VR

OVS

OVS OVS

CloudStack SDN

Controller

Built-in controller

Host 1

Host 2

Host 3

Host 4

Assign 'Tenant' key for isolation

New tenants can share the established GRE tunnels with separate tenant keys

GRE Tunnel

GRE Tunnel GRE Tunnel

VM 1

VM2

VM3 VR

VM 1

VM 2

VM3 VR

Tenant1Tenant2

What makes it different

• Purpose built for IAAS– Not general purpose SDN solution

• Proactive model– Deny all flows except the ones programmed by the end-

user API– Scaling problem is manageable

• Part of CloudStack– ASF project

• Uses Virtual Router to provide L3-L7 network services– Could change

Futures

• AWS VPC semantics– Support security groups, ACL

• Optimize ARP & DHCP responses• Cross-zone networks

– Optimize inter-subnet routing