openstack ops meetup

Romana Project

Network and Security Automation

romana.ioJune 2016

OpenStack

Operators Meetup

June 7, 2016

romana.io

New Networks, New Problems, New Solutions

• Legacy Apps/Enterprise Private

Cloud

• LAN Emulation to support vMotion

• Automated data center

infrastructure provisioning

• Cloud Native Apps

• Seamless public/private cloud

deployment and orchestration

• Docker and Container networking

• Endpoint explosion and compressed

lifecycle

• Whitebox and GIFEE Networks

• Enterprise SDN

• VMware/NSX

• Cisco ACI

• Others…

• Cloud Native Networks

• Network automation for

rapid provisioning

• Security automation

• Multi-cloud

romana.io

June 2016 Slide 1

Cloud Native vs. Enterprise Networks• Amazon AWS Style v. Enterprise Apps

• Service orientation (Cattle) v. Endpoint orientation (Pets)

• Network requirements

• Reachable IP addresses v. Auto discovered MAC (ARP on VLANs)

• Service orientation further decouples apps from infrastructure

• No VM migration

• No IP Failover

• Good News: Cloud Native apps don’t need layer 2 networks

• Avoiding Layer 2 networks eliminates a lot of SDN complexity

• Bad News: Layer 2 networks provided a convenient way to isolate apps

• Even a small number of VLANs were difficult to automate

Bottom Line: Need a new way to isolate networks

romana.ioJune 2016 Slide 2

Romana Network and Security Automation• Layer 3 based isolation and tenancy model

• Topology-aware addressing

• Embed tenant and segment IDs in IP addresses

• Requires nothing more than standard L3 routing

• Hierarchical design simplifies scalable deployment

• No virtual network required

• Native performance and visibility

• Eliminates overlays

• Routes map to services 1:1

• Simplifies composition, security and control

• Tightly integrated into Cloud Management/Orchestration IPAM


SDN Complexity melts away• No VLANs, VXLANs, VTEP/VNID, OpenFlow, OVS/OVN/OVSDB

• Route aggregation simplifies network

• Static routing eliminates need for route distribution (BGP, XMPP, KVS)

• Reduces the number of firewall rules (i.e. network v. endpoint)

• Simplifies Operations

• Existing tools, techniques and diagnostics all just work

• Existing security, policy and control systems all work

• Firewalls, IDS, LB, etc., etc., etc.

June 2016 romana.io Slide 4

North/South Traffic• Neutron Network node

routes traffic between

segments

• Network node

performs all

L3 functions

• East/West traffic

encapsulated, but is direct

to destination host

romana.io

VXLAN Decap

VXLAN Decap

VXLAN Encap

VXLAN Encap

2 Top of Rack

Round Trips

East/West

Traffic

Per Instance

Security

June 2016 Slide 5

North/South Traffic• Latency dramatically

reduced

• No Network node

• No encap

• Identical path for

East/West traffic

romana.io

Eliminated

Bypassed

Bypassed

Romana

Router

Romana

Router

1 Top of Rack

Round Trip

Per Network

Security

June 2016 Slide 6

Network Latency

• North/South Latency reduced 50%-85%

• 10% improvement for East/West traffic between hosts (no encap)

• No performance penalty for local on-host East/West traffic

romana.io

North/South

(Routed)

East/West

(Switched)

Time (ms) Local Remote Local Remote

Native OpenStack 1.51* 1.51 0.24 0.85

Romana Networks 0.24 0.77 0.24** 0.77**

Relative Performance Local Remote Local Remote

Native OpenStack 100% 100% 100% 100%

Romana Networks 16% 51% 100% 90%

* All N/S OpenStack traffic

goes off host

** All Romana traffic is

routed

June 2016 Slide 7

How does it work?• Assign CIDR length for host (node), tenant and segment

• Example: host 16, tenant 20, segment 24

• On every host, each tenant gets a real physical CIDR

• Tenant can further sub-net for their own private segments

• Assign IP addresses that maintain reachability

• Apply layer 3 firewall rules for network isolation

• Configure next hop gateway for service composition


Example


Bit location 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

Field

Capacity 0 0 0 0 1 0 1 0

Example: Bits Length Purpose

10/8 Network 8 10/8 Network

Hosts 8 Up to 255 Hosts

Tenants 4 Up to 255 Tenants

Segments 4 Up to 16 Segments per Tenant

Endpoints 8 Up to 16 Endpoints per Segment

Host 1 ID CIDR or IP Host 2 ID CIDR or IP Host 3 ID CIDR or IP

Physical Addr 192.168.0.10 Physical Addr 192.168.0.11 Physical Addr 192.168.0.12

Host 1 10.1/16 Host 2 10.2/16 Host 3 10.3/16

Tenant 0 10.1.0/20 Tenant 0 10.2.0/20 Tenant 0 10.3.0/20

Segment 1 10.1.1/24 Segment 1 10.2.1/24 Segment 2 10.3.2/24

VM 1 22 VM 1 22 VM 1 22

VM 2 33 VM 2 33 VM 2 33

Tenant 1 10.1.16/20 Tenant 1 10.2.16/20 Tenant 1 10.3.16/20

Segment 1 10.1.17/24 Segment 2 10.2.18/24 Segment 1 10.3.17/24

VM 3 44 VM 3 44 VM 3 44

VM 4 55 VM 4 55 VM 4 55

Endpoint ID

Up to 255 Hosts Up to 255 Tenants 255 Endpoints for each Tenant

20 17-20

10/8 Net Mask Host ID Bits (8) Tenant/Segment ID Bits (8)

Location

8 1-8

16 9-16

24 21-24

32 25-32

10.1.1.22

10.1.17.55 10.2.18.55 10.3.17.55

10.3.2.22

10.1.1.33 10.2.1.33 10.3.2.33

10.1.17.44 10.2.18.44 10.3.17.44

10.2.1.22

Physical Deployment


192.168.0.10 192.168.0.11 192.168.0.12

Host 1

VM 1: 10.1.1.22

G/W: 10.1.0.1/16

VM 2: 10.1.1.33

VM 3: 10.1.17.44

VM 4: 10.1.17.55

10.2/16 -> 192.168.0.11

10.3/16 -> 192.168.0.12

Host 2

VM 1: 10.2.1.22

G/W: 10.2.0.1/16

VM 2: 10.2.1.33

VM 3: 10.2.18.44

VM 4: 10.2.18.55

10.1/16 -> 192.168.0.10

10.3/16 -> 192.168.0.12

Host 3

VM 1: 10.3.2.22

G/W: 10.3.0.1/16

VM 2: 10.3.2.33

VM 3: 10.3.17.44

VM 4: 10.3.17.55

10.1/16 -> 192.168.0.10

10.2/16 -> 192.168.0.11

ECMP

BGP/OSPF Area

Leaf 1

Every host gets /16 network, announces to Leaf

Leaf aggregates 64 /16 networks, announces /10 to Spine

Spine contains only four /10 networks

0.0.0.0 via Spine 1

10.0.0.0/16 via Port 1

10.1.0.0/16 via Port 2

10.2.0.0/16 via Port 3

10.3.0.0/16 via Port 4

…

10.63.0.0/16 via Port 64

Spine 1

0.0.0.0 via Internet

10.0.0.0/10 via Leaf 1

10.64.0.0/10 via Leaf 2

10.128.0.0/10 via Leaf 3

10.192.0.0/10 via Leaf 4

Spine 2

Leaf 2 Leaf 3 Leaf 4

0.0.0.0 via Internet

10.0.0.0/10 via Leaf 1

10.64.0.0/10 via Leaf 2

10.128.0.0/10 via Leaf 3

10.192.0.0/10 via Leaf 4

10.2/16 RIP to Leaf for distribution0.0.0.0 via Leaf 1, Port 8

Port 8

Host 221

10.194.3.710.0.0.0 via Leaf 4, Port 3

Port 3

Host 8

10.2.16.34

0.0.0.0 via Spine 1

10.192.0.0/16 via Port 1

10.193.0.0/16 via Port 2

10.194.0.0/16 via Port 3

10.195.0.0/16 via Port 4

…

10.255.0.0/16 via Port 64


Endpoints on Host 8 must get address within 10.2.0.0/16

Endpoints on Host 221 must get address within 10.194.0.0/16

Announce route to ToR

Leaf 1

Spine 1 Spine 2

Leaf 2 Leaf 3 Leaf 4

10.2/16 RIP to Leaf for distribution172.16.1.25 host route0.0.0.0 via Leaf 1, Port 8

Host

10.194.3.710.0.0.0 via Leaf 4, Port 3Host 8

10.2.16.34

Edge/

NAT

Host routes to external service endpoints


SLB

VMSLB get FIP as VIP

FIP 172.16.1.25

Security

Policy

Neutron Node

OpenStack Deployment

May 2016 romana.io

IPAM

Routes

Tenant

DB

Topology

Policy

Slide 13

Neutron

ML2IPAM

Compute Node n

VM

iptables

VM

Nova

Agent

Network/Security PolicyNetPolicy.json

{

"Name": "policy2",

"PolicyID": "CF2D2BE2-4553-4C28-BD02-140CF83617A2", # unique identifier across tenants, auto generated for POST.

"AppliedTo": [ # can attach multiple tenants to which the policy can be applied to.

{

"Tenant":"tenant2",

"Segment": "Segment1",

“HostCIDR": “10.23.0.0/0", # Apply policy to entire host

},

],

"Tags": [], # meta data attached to policies for various external environments like openstack/kubernetes

"Direction" : "Ingress", # can be Egress or Ingress.

"Peers": [

{

"CidrBlock": "0.0.0.0/0", # IP from L3 header

},

],

"Rules": [{

"Protocol": "ICMP",

"IcmpTypeCode": [0,8],

"IsStateful": true,

},],

"Description": "hello there, security policies are fun!",

}


Scalable Deployments• Need more IP addresses

• Large OpenStack environments

• Container endpoint explosion

• Separate Romana deployment for each OpenStack cluster

• Clusters interact via service endpoints

• Explicitly manage overlapping IPs

• Use datacenter FIPs

• Support Overlapping in Romana IPAM

• Advantage of consistent policy across environment

• IPv6


Cluster 2Cluster 1

Romana 1: 10/8

Shared Block: 10.0.1/24

Local FIPs: 10.0.1.128/25

Remote FIPs: 10.0.1.0/25

Edge

Large Scale Deployments


Romana 2: 10/8

Shared Block: 10.0.1/24

Local FIPs: 10.0.1.0/25

Remote FIPs: 10.0.1.128/25Alternatively use FIPs from

DC addresses

Shared 172.16.1/24

FIPs

Security

Policy

k8s Master

Kubernetes Deployment

May 2016 romana.io

IPAM

Routes

Tenant

DB

Topology

Policy

Slide 17

Minion

Pod

iptables

Pod

Agent

Controllers

Scheduler

API

etcd

Pod/Service

Definition

CNI

Listener

Nested Container Networking


Bit location 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

Field

Capacity 0 0 0 0 1 0 1 0


10.0 Network 8 Full Network (10/8)





Bit location 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

Field Host ID Bits (4)

Capacity 1 0 1 0 1 1 0 0 0 0 0 1 Up to 16 Hosts


172.16 Network 12 Full Network (172.16/12)





Endpoint ID

Up to 255 Hosts Up to 255 Tenant/Segments 255 Endpoints

Tenant and Segment ID Bits (8) Endpoint ID

Up to 255 Tenant/Segments 255 Endpoints

Location

12 1-12

16

20 17-20

10/8 Net Mask Host ID Bits (8) Tenant and Segment ID Bits (8)

Location

8 1-8

16 9-16

24 21-24

32 25-32

13-16

20 17-20

24 21-24

32 25-32

172.16/12 Net Mask

Nested Containers

June 2016 romana.io

192.168.0.10 192.168.0.11 192.168.0.12

Slide 19

Host 1

VM 1: 10.1.1.22

G/W: 10.1.0.1/16

10.2/16 -> 192.168.0.11

10.3/16 -> 192.168.0.12

172.17/16-> 192.168.0.11

172.18/16 -> 192.168.0.12

Pod 172.16.1.8

Pod 172.16.2.9

GW 172.16.0.1/16

172.17/16 -> 10.2.0.1

172.18/16 -> 10.3.0.1

Host 2

VM 1: 10.2.1.22

G/W: 10.2.0.1/16

Pod 172.17.6.8

Pod 172.17.2.11

GW 172.17.0.1/16

172.18/16 -> 10.3.0.1

172.16.16 -> 10.1.0.1

Host 3

VM 1: 10.3.1.22

G/W: 10.3.0.1/16

Pod 172.18.3.8

Pod 172.18.4.9

GW 172.18.0.1/16

172.16/16 -> 10.1.0.1

172.17/16 -> 10.2.0.1

10.1/16 -> 192.168.0.10

10.3/16 -> 192.168.0.12

172.16/16 -> 192.168.0.10

172.18/16 -> 192.168.0.12

10.1/16 -> 192.168.0.10

10.2/16 -> 192.168.0.11

172.16/16 -> 192.168.0.10

172.17/16-> 192.168.0.11

Ubernetes

June 2016 romana.io

192.168.0.10 192.168.0.11 192.168.0.12

Slide 20

Host 1

VM 1: 10.1.1.22

G/W: 10.1.0.1/16

10.2/16 -> 192.168.0.11

10.3/16 -> 192.168.0.12

172.17/16-> 192.168.0.11

172.18/16 -> 192.168.0.12

Pod 172.16.1.8

Pod 172.16.2.9

GW 172.16.0.1/16

172.17/16 -> 10.2.0.1

172.18/16 -> 10.3.0.1

Host 2

VM 1: 10.2.1.22

G/W: 10.2.0.1/16

Pod 172.17.6.8

Pod 172.17.2.11

GW 172.17.0.1/16

172.18/16 -> 10.3.0.1

172.16.16 -> 10.1.0.1

Host 3

VM 1: 10.3.1.22

G/W: 10.3.0.1/16

Pod 172.18.3.8

Pod 172.18.4.9

GW 172.18.0.1/16

172.16/16 -> 10.1.0.1

172.17/16 -> 10.2.0.1

10.1/16 -> 192.168.0.10

10.3/16 -> 192.168.0.12

172.16/16 -> 192.168.0.10

172.18/16 -> 192.168.0.12

10.1/16 -> 192.168.0.10

10.2/16 -> 192.168.0.11

172.16/16 -> 192.168.0.10

172.17/16-> 192.168.0.11

WAN

Networks Define Services• Tenant ID + Segment ID become a Network ID

• Natural fit for micro- and shared platform

services

• Route control to/from micro services enable

transparent service insertion/chaining and policy

enforcement

• Local/remote/hybrid cloud deployments

romana.io

IP

Int

IP

Int

IP

Int

IP

Int

L/B

Microservice

Endpoint

F/W

Shared Services

June 2016 Slide 21

Romana Project• Cloud Native network and security automation

• All details available at romana.io

• Open source

• Apache 2.0

• Written in Go

• www.github.com/romana

• OpenStack and Kubernetes integration

• Release v0.9 available now


romana.io

Demo• OpenStack on four physical machines

• Launch VMs on private 10/8 network

• Kubernetes running on VMs

• Kubernetes Network 172.16/12

• Container Network Interface (CNI) configuration of pods

• Romana IPAM allocates IPs for VMs and pods

• Chosen specially to maintain static routes and CIDRs to each host

and VM

• All IPs reachable by construction
