Kubernetes v1.2

Multi-tenant Networking

Romana Cloud Native SDNChris Marino

Robert Starmer

romana.ioKubernetes Meetup 2/11/16

Multi-tenant Networking • Agenda

• Cloud Native Networks

• Romana Cloud Native SDN

• How it works

• Demo

• Q & A

Kubernetes Meetup 2/11/16 romana.io Slide 1

Cloud Native vs. Enterprise Networks• Amazon AWS Style v. Enterprise Apps

• Service orientation (Cattle) v. Endpoint orientation (Pets)

• Network requirements

• Reachable IP addresses v. Auto discovered MAC (ARP on VLANs)

• Service orientation further decouples apps from infrastructure

• No VM migration

• No IP Failover

• Good News: Cloud Native apps don’t need layer 2 networks

• Layer 2 networks introduce a lot of SDN complexity

• Bad News: Layer 2 networks provided a convenient way to isolate apps

romana.ioKubernetes Meetup 2/11/16 Slide 2

Romana Cloud Native SDN• Layer 3 based isolation and tenancy model

• Topology-aware addressing

• Embed tenant and segment IDs in IP addresses

• Requires nothing more than standard L3 routing

• Hierarchical design simplifies scalable deployment

• No virtual network required

• Native performance and visibility

• Eliminates overlays

romana.ioKubernetes Meetup 2/11/16 Slide 3

Complexity melts away• No VLANs, VXLANs, VTEP/VNID, OpenFlow, OVS/OVN/OVSDB

• Route aggregation simplifies operations

• Static routing eliminates need for route distribution (BGP, XMPP, KVS)

• Reduces the number of firewall rules (i.e. network v. endpoint)

• Simplifies Operations

• Existing tools, techniques and diagnostics all just work

• Existing security, policy and control systems all work

• Firewalls, IDS, LB, etc., etc., etc.

Kubernetes Meetup 2/11/16 romana.io Slide 4

How does it work?• Assign CIDR length for host (node), tenant and segment

• Example: host 16, tenant 24, segment 28

• On every host, each tenant gets a real physical CIDR

• Tenant can further sub-net for their own private segments

• Configure IP addresses that maintain reachability

• Apply layer 3 firewall rules for network isolation

Kubernetes Meetup 2/11/16 romana.io Slide 5

Example

Kubernetes Meetup 2/11/16 romana.io Slide 6

Bit location 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

Field

Capacity 0 0 0 0 1 0 1 0

Example: Bits Length Purpose

10/8 Network 8 10/8 Network

Hosts 8 Up to 255 Hosts

Tenants 8 Up to 255 Tenants

Segments 4 Up to 16 Segments per Tenant

Endpoints 4 Up to 16 Endpoints per Segment

Host 1 ID CIDR or IP Host 2 ID CIDR or IP Host 3 ID CIDR or IP

Physical Addr 192.168.0.10 Physical Addr 192.168.0.11 Physical Addr 192.168.0.12

Host 1 10.1/16 Host 2 10.2/16 Host 3 10.3/16

Tenant 1 10.1.1/24 Tenant 1 10.2.1/24 Tenant 1 10.3.1/24

Segment 1 10.1.1.16/28 Segment 1 10.2.1.16/28 Segment 1 10.3.1.16/28

Pod 1 11 Pod 1 4 Pod 1 4

Pod 2 14 Pod 2 5 Pod 2 5

Tenant 2 10.1.2/24 Tenant 1 10.2.1/24 Tenant 2 10.3.2/24

Segment 1 10.1.2.16/28 Segment 2 10.2.1.32/28 Segment 1 10.3.2.32/28

Pod 1 4 Pod 1 9 Pod 1 9

Pod 2 8 Pod 2 12 Pod 2 12

Location

10/8 Net Mask Host ID Bits (8) Tenant ID Bits (8) Segment ID and IID

Up to 255 Hosts Up to 255 Tenants 255 Endpoints for each Tenant

10.1.1.27

10.3.2.28

10.3.2.25

10.3.1.21

10.3.1.20

10.2.1.44

10.2.1.41

10.2.1.21

10.2.1.20

10.1.2.24

10.1.2.20

10.1.1.40

32

28

24

16

8

29-32

25-28

17-24

9-16

1-8

Host 1: 192.168.0.10 on Port 1

Host 2: 192.168.0.11 on Port 2

Host 3: 192.168.0.12 on Port 3

Router,

Switch

or VPC

Physical Deployment

Kubernetes Meetup 2/11/16 romana.io

192.168.0.10 192.168.0.11 192.168.0.12

Host 1

Pod 1

1.1.27

G/W: 10.1.0.1/16

Pod 2

1.1.40

Pod 1

1.2.20

Pod 2

1.2.24

Tap

Interfaces

Host 2

Pod 1

2.1.20

G/W: 10.2.0.1/16

Pod 2

2.1.21

Pod 1

2.1.41

Pod 2

2.1.44

Tap

Interfaces

Host 3

Pod 1

3.1.20

G/W: 10.3.0.1/16

Pod 2

3.1.21

Pod 1

3.2.25

Pod 2

3.2.28

Tap

Interfaces

Slide 7

Romana Project• Cloud Native SDN

• All details available at romana.io

• Open source

• Apache 2.0

• Written in Go

• www.github.com/romana

• Release v0.6.4 available now

• Integration with OpenStack

• Kubernetes integration very soon

romana.ioKubernetes Meetup 2/11/16 Slide 8

Node nNode nNode nNode nNode n

KubletAgentKube

Proxy

Docker

/rkt

Pod Pod

iptables

CNI

Romana

Romana Networks

Kubernetes Meetup 2/11/16 romana.io

K8S Master

IPAM

Routes

Tenant

DB

Topology

Controllers

Scheduler

API

etcd

ThirdParty Resource

Network Policy

Schema

Slide 9

Policy

/apis/romana.io/demo/v1

Pod/Service

Spec

Network Policy

Network Policy Resource

Kubernetes Meetup 2/11/16 romana.io Slide 10

name: network-policy.romana.io

apiVersion: extensions/v1beta1

kind: ThirdPartyResource

description: “Romana Network Policy Third Party Resource

Schema"

versions:

- name: demo/v1

Resulting API Endpoint/apis/romana.io/demo/v1/networkpolicy/

www.romana.io

Tenant t1 Pod Specifications

• FrontendapiVersion: v1

kind: Pod

metadata:

name: nginx-frontend

labels:

app: nginx

owner: t1

tier: frontend

spec:

containers:

- name: nginx

image: nginx

ports:

- containerPort: 80

• BackendapiVersion: v1

kind: Pod

metadata:

name: nginx-backend

labels:

app: nginx

owner: t1

tier: backend

spec:

containers:

- name: nginx

image: nginx

ports:

- containerPort: 80

Kubernetes Meetup 2/11/16

romana.io

Slide 11

www.romana.io

Replication Controller

• Tenant t2apiVersion: v1

kind: ReplicationController

metadata:

name: nginx-default

spec:

replicas: 3

template:

metadata:

labels:

app: guestbook

tier: default

owner: t2

spec:

containers:

- name: nginx-default

image: nginx

ports:

- containerPort: 80

Kubernetes Meetup 2/11/16

romana.io

Slide 12

www.romana.io

Pod Specifications

• FrontendapiVersion: v1

kind: Pod

metadata:

name: nginx-frontend

labels:

app: nginx

owner: t1

tier: frontend

spec:

containers:

- name: nginx

image: nginx

ports:

- containerPort: 80

• BackendapiVersion: v1

kind: Pod

metadata:

name: nginx-backend

labels:

app: nginx

owner: t1

tier: backend

spec:

containers:

- name: nginx

image: nginx

ports:

- containerPort: 80

Kubernetes Meetup 2/11/16

romana.io

Slide 13

Network Policy• Policy1

kind: NetworkPolicy

apiVersion: romana.io/demo/v1

metadata:

name: policy1

namespace: default

labels:

- owner: t1

spec:

podSelector: // Standard label selector - selects pods.

tier: backend

allowIncoming: // (Optional) List of allow rules.

- toPorts: // (Optional) List of dest ports to open.

- port: 80 // (Optional) Numeric or named port

protocol: TCP // [ TCP | UDP]

from: // (Optional) List of sources.

- pods: // (Optional) Standard label selector.

tier: frontend // (Optional) Standard label selector.

Kubernetes Meetup 2/11/16 romana.io Slide 14

Router,

Switch

or VPC

Demo

Kubernetes Meetup 2/11/16 romana.io

192.168.0.10 192.168.0.11

Host 1

T1

1.1.27

G/W: 10.1.0.1/16

T1

1.1.40

FE

1.2.20

BE

1.2.44

Tap

Interfaces

Host 2

T1

2.1.20

G/W: 10.2.0.1/16

Tap

Interfaces

Slide 15

Demo• Running Kubernetes on x EC2 instances

• Romana Services running on Kubernetes Master

• Demo Script

1. Apply NetworkPolicy ThirdParty Schema

2. Launch Pods as different isolated tenants

3. Within a single tenant, launch Pods on separate Tiers

4. Apply Network Policy to Tiers

5. Show Policy Enforcement

Kubernetes Meetup 2/11/16 romana.io Slide 16