kubecon eu 2016: cloud native sdn for kubernetes

+

A Cloud Native SDN for Kubernetes

Juergen Brendel, Stas KraevKubecon, London, March 2016

romana.io A cloud native SDN for Kubernetes @romanaproject

Agenda

● “Cloud native”, why does it matter?● A better network for cloud native architectures● New things in Kubernetes● Demos


About us

● Team background:– Data center networks

– Low-level traffic management

● Created L2 overlay network startup– Bought by Cisco

● OpenStack networking● There's got to be a better way

– Time is right

What is 'cloud native'?


The past: Enterprise networking

● Full control● Applications need L2 and L3

– May need hard-wired IP addresses

– Broadcasts

● Servers are pets, not cattle: “Careful!”– VM migration

● Complex!– Complexity in the applications

– Because apps may do anything, network needs to support everything!


Cloud native applications

● Automate all the things!– Infrastructure as code

– Cattle, not pets: “Meh... just kill it.”

– Workloads come and go quickly

– Build for resiliance

● IP is all you need– No hardcoded IP addresses, discovery

– No special network requirements

– Basic IP connectivity

● Restrictions– Accept them and get clarity and simplicity in return

The problem


We have a mismatch

● Building cloud native applications…● … on top of enterprise networking

– SDN controllers use overlay L2 domains

– VLAN, VXLAN, OVS, etc.

● Complexity and brittleness– Lose benefits of simplicity

– Lose performance (encap, blinded hardware)

– Difficult to maintain and trouble shoot


The price you pay: Complexity

VXLAN Decap

VXLAN Decap

VXLAN Encap

VXLAN Encap

2 Top of Rack Round Trips

East/West Traffic

Per Instance Security


The price you pay: Performance

Router

Endpoint A Endpoint B

Router

L2 overlay A

L2 overlay B

VRouter


Why do we do this to ourselves?

● We don't need any L2 features● Except traffic segmentation

– Multi tenancy

– Tiers and policies

The solution


Cloud native SDNs

● Use native L3 capabilities● No overlays● De-emphasize IP address ranges● Still provides segmentation, multi tenancy● Simple, clear and scalable network setup


A truly cloud native SDN: Romana

● Project Romana● Open source● Apache 2.0 license● Mostly written in Go● Kubernetes and OpenStack


A truly cloud native SDN: Romana

● Use only IP routing– No overlays

– All workload addresses are 'real'

– Simplicity!

● Use smart addressing– Encode tenant or segment in IP address

– Assign “virtual” addresses with host prefixes

– Massive (!) collapse of route table

● Routes are static– No route updates, no broadcasts for new endpoint


Routing and route aggregation

Host A

eth0:192.168.8.11

romana-gw:10.0.0.1/16

10.0.0.5

10.0.1.7

10.0.1.19

10.0.5.3

Host B

eth0:192.168.8.22

romana-gw:10.1.0.1/16

10.1.3.52

10.1.9.2

Host C

eth0:192.168.8.33

romana-gw:10.2.0.1/16

10.2.0.16

10.2.3.81

10.2.4.6

Routes:10.1/16 → 192.168.8.2210.2/16 → 192.168.8.33

Routes:10.0/16 → 192.168.8.1110.2/16 → 192.168.8.33

Routes:10.0/16 → 192.168.8.1110.1/16 → 192.168.8.22


Architecture

Host A Host B Host C

Agent Agent Agent

Tenant

Topology

IPAM

Root

Kubernetes


Architecture

Host A Host B Host C

Agent Agent Agent

Tenant

Topology

IPAM

Root

OpenStack

Romana / Kubernetes integration


Integration points

● CNI (Container Network Interface)– Developed last year by CoreOS

– Supported by Kubernetes since version 1.1

● Third party resources– Develop Kubernetes extensions via external

processes

● Network Policies– Still under development by networking SIG

– Different proposals under discussion


CNI_COMMAND (ADD | DEL)CNI_CONTAINERIDCNI_NETNSCNI_IFNAMECNI_ARGS...

CNI: Interface creation workflow

Host A

eth0:192.168.8.11

RomanaCNI plugin

Kubelet Create interface



Host A

eth0:192.168.8.11

RomanaCNI plugin

Kubelet

RomanaIPAM

RomanaTenant

RomanaTopology

HostTenant

Segment



Host A

eth0:192.168.8.11

RomanaCNI plugin

Kubelet

RomanaAgent

10.0.0.5

connectivity

policies

RomanaIPAM

RomanaTenant

RomanaTopology

IP address


Third party resources

● Tell Kubernetes about your new resource

$ kubectl create f thirdpartyresourcedefinition.yml

● Start listening for events on new URLs

/apis/romana.io/demo/v1/namespaces/default/networkpolicys/

metadata: name: networkpolicy.romana.ioapiVersion: extensions/v1beta1kind: ThirdPartyResourcedescription: "Network policy"versions: name: demo/v1


Kubernetes network polices

● Recognized need for policies– Grant / deny access, isolate tiers and tenants

– Basically: ACLs

– Different proposals exist

– Implementations use Kubernetes 3rd party resources

● Namespaces– Use namespace as 'tenant'

– Add 'isolation' flag to namespace


Example network policy

POST /apis/romana.io/demo/v1/namespaces/tenanta/networkpolicys/

{ "kind": "NetworkPolicy", "metadata": { "name": "pol1" }, "spec": { "allowIncoming": { "from": [ { "pods": { "segment": "frontend" } } ], "toPorts": [ { "port": 80, "protocol": "TCP" } ] }, "podSelector": { "segment": "backend" } }}

Gets applied to

namespace

“segments”: Natural fit

for Romana


Network policy workflow

Kubernetes master

Kubernetes API 3rd party resource type definitionkubectl



Kubernetes master

Kubernetes APIURLs

New URLs for this resource type, per

namespace


Host

RomanaAgent

iptables

Host

RomanaAgent

iptables


Kubernetes master

RomanaK8S listener

Kubernetes API

Host

RomanaAgent

New Romana policy definition

URLs

Events streamed

through GET request

Some client

POST /…..{ new policy }

iptables


Conclusion

● Cloud native architectures simplify things● Need a cloud native SDN to enjoy benefits● Romana:

– Cloud native without compromises

– Native network performance

– Mostly static config: Solid network

– Very easy to work with and understand

● Easy to try:– Simple installers for Kubernetes and OpenStack


Thank you!

● Romana Links– http://romana.io - Project home

– http://romana.io/blog - Blog

– https://github.com/romana/romana - Sources

● Contact– @romanaproject - Twitter

– [email protected] - Email

– https://romana.slack.com/ - Slack channel

● Kubernetes links– http://bit.ly/1RMVkrr - CNI spec

http://romana.io/

http://romana.io/blog

https://github.com/romana/romana

mailto:[email protected]

https://romana.slack.com/

http://bit.ly/1RMVkrr

Appendix: Romana technical notes


Semantic and topological addressing

31

30

29

28

27

26

25

24

23

22

21

20

19

18

17

16

15

14

13

12

11

10

9 8 7 6 5 4 3 2 1 0

0 0 0 0 1 0 1 0 0 0 0 0 0 1 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 1 1

10

Network prefix bitsThe network prefix. In this example, we are using the 10/8

address space.

6

Host ID Segment IDWe currently

store tenant ID in upper bits of segment ID.

4 67

Endpoint ID

Widths are configurable, don't have to use byte boundaries.


Segment and tenant bits

31

30

29

28

27

26

25

24

23

22

21

20

19

18

17

16

15

14

13

12

11

10

9 8 7 6 5 4 3 2 1 0

0 0 0 0 1 0 1 0 0 0 0 0 0 1 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 1 1

10

Network prefix bits

6

Host ID Segment ID

4 67

Endpoint ID

Encode thetenant ID


Romana: Traffic segmentation

● Tenant traffic separated:– Tenants don't get whole CIDR prefix or L2 domain

– But fully isolated from other tenants' traffic

● Tenants can define segments:– Like tiers, provide isolation and policies

● Use segment and tenant bits in IP addresses:– Apply policies (iptables) based on that

– Segments can stretch across hosts


Host BHost A

Allowing traffic within tenant

10.0.0.5 10.1.0.12

iptables:check src/dst addrs“tenant/segment bits

must match”

Src: 10.0.0.5Dst: 10.1.0.12

Same tenant/segment bits


Host BHost A

Isolating tenant traffic: Default

10.0.0.5 10.1.128.9

iptables:check src/dst addrs“tenant/segment bits

must match”

Src: 10.0.0.5Dst: 10.1.128.9

Different tenant/segment bits

Differenttenant


Host BHost A

Apply network policy between segments (full isolation as default)

10.0.0.5 10.1.1.9

iptables:Does policy chain

exist?Otherwise: DROP

Src: 10.0.0.5Dst: 10.1.1.9

Same tenant, different segment

policy-chain:From segment 0?Protocol TCP?To port 80?

kubecon eu 2016: cloud native sdn for kubernetes

Technology