docker meetup - melbourne 2015 - kubernetes deep dive

Networking & Storage Deep Dive

Ken ThompsonSenior Cloud Solution ArchitectMelbourne Docker Meetup - October 2015

KUBERNETES: THE HELMSMEN

Melbourne Docker Meetup Oct 2015 - Kubernetes Networking & Storage Deep Dive2

AgendaKubernetes Deep Dive

● Kubernetes Networking● Kubernetes Persistent Volumes● Live Demos:

– Persistent Storage– Something Fun!...


Container orchestration at scale

KUBERNETES

● Scale workloads across a cluster ofcontainer hosts

● Declarative state of environmentthat gets managed

● Intelligent policy-based scheduling● Multi-host networking support (with

OpenShift-SDN/OvSwitch etc.)● Multi-host shared storage for

containers● Image update mechanisms (with

OpenShift Deployment Configs -rolling update, a-b, etc.)

Melbourne Docker Meetup Oct 2015 - Kubernetes Networking & Storage Deep Dive

4

KUBERNETES DEEP DIVE:NETWORKING


DOCKER IN-HOST NETWORKINGSuitable for single-host, not for multi-host

● Docker assigns an IP to eachcontainer

● Its local and private only by default● Complex for multi-host● Plugins starting to become available● Now we have a container..hold on!

– What happens If we want tohave containers sharingfilesystem

– Or IPC communications

– Or localhost communications(mysql & phpmyadmin for eg.)

eth0

Linux bridge: docker0: 172.17.42.1/16

vethXXX

172.17.x.x/16

container

eth0

vethXXX

172.17.x.y/16

container

eth0


KUBERNETES IN-HOST NETWORKINGKubernetes Pods

● Pods (aka gears in OSE v2)– Pod typically runs only one

container for independent scaling,with exceptions such as “sidecars”.Ie. MySQL & phpMyAdmin

– Same IP / network namespace

– Same IPC

– Ability to share disk

– Scheduled on same machine

– Atomic unit

– Can listen (to other containers)on127.0.0.1 or 0.0.0.0


INTER-HOST NETWORKINGSolutions Available

PROGRAMMABLEINFRA

● GCE – each host gets asubnet of 256 Ips

● AWS 40-100 Ips perhost

● Custom enterprisenetworking

OVERLAY NETWORKS

● Flannel (ie. AtomicHost) – subnet withsimple overlay.

● Others solutionsavailable like Weave,etc.

Open vSwitch

● OpenStack

● Very largedeployments

● Programmable

● OpenShift-SDN usesthis


● Kubernetes assigns 10.1.x.0/24 subnet to pods in each node● Gateway IP 10.1.x.1 is assigned to lbr0● Out of the box with OpenShift, Open vSwitch VXLAN overlay is used to connect nodes/pods● Flat network, allow subdivision via flows

OPENSHIFT-SDN NETWORKING

OVS: br0

vxlan0

eth0

vovsbr

Linux bridge: lbr0: 10.1.0.1/24

vlinuxbr

veth pair

vethXXX

Pod10.1.0.2

eth0

vethXXX

Pod10.1.0.3

eth0VXLAN

VXLAN overlay

OVS: br0

vxlan0

eth0

vovsbr

Linux bridge: lbr0: 10.1.1.1/24

vlinuxbr

veth pair

vethXXX

Pod10.1.1.2

eth0

vethXXX

Pod10.1.1.3

eth010.1.1.0/24

VXLAN


● OpenShift-SDN or similar overlays allow communications within a cluster, but how doaccess from outside; ie. WWW?

● Router Pod – Runs HAproxy, proxies incoming traffic through SDN – HTTP/S Only

OPENSHIFT-SDN ROUTING


10

KUBERNETES DEEP DIVE:STORAGE


DOCKER STORAGE CONSTRAINTS KUBERNETES GOALS

CONTAINER STORAGE

1) Containers are ephemeral(stateless), once they disappear, sodoes the data

2) You can mount the host filesystem,but

1) What about loss of host?

2) How do you scale across1000s of hosts?

3) How is this data shared?

1) Allow administrators to describeavailable storage

2) Allow application developers todiscover and request persistentstorage

3) Persistent storage should beavailable without being closelybound to a particular disk, server,network or storage device

4) Support iSCSI, NFS, EBS, Gluster,Ceph… and many more!

Problem Statement


PERSISTENT VOLUME CLAIMS

● Admins add PVs to cluster [1]● Like Pods, PVClaims live in a

namespace● Pods [3] and Pvclaims [2] are

requests for resources by users● Users request resources with

limits, like cpu usage by acontainer or storage capacity of avolume [2]

● OpenShift binds requests toavailable resources

● Multiple access modes(RWO,ROM,RWM)

Requests for storage

kind: PersistentVolumeClaimapiVersion: v1metadata: name: myclaimspec: accessModes: - ReadWriteOnce resources: requests: storage: 8Gi

kind: PodapiVersion: v1metadata: name: mypodspec: containers: - name: myfrontend image: dockerfile/nginx volumeMounts: - mountPath: "/var/www/html" name: mypd volumes: - name: mypd persistentVolumeClaim: claimName: myclaim

POST:kind: PersistentVolumeapiVersion: v1metadata: name: pv0001spec: capacity: storage: 10 nfs: Server: srv.com path: /data/path

1 2

3


DB

Host 1 Host 2

Storage Pool

Kubernetes Persistent Volumes

CONTAINER STORAGE

mysql-claim

pv0001

Network Storage NFSiSCSI EBS RBD

2. Dev “Claims” PV

1. Admin creates PV


DB

Host 1 Host 2

DB

What happens to a container & its storage when a node is lost?

CONTAINER STORAGE

Storage Pool

mysql-claim

pv0001



What happens to a container & its storage when a node is lost?

CONTAINER STORAGE

DB

Host 1 Host 2

DB

Storage Pool

mysql-claim

pv0001



16

Storage Demo(https://github.com/openshift/origin/tree/master/examples/wordpress)

& HexBoard Demo

(https://www.youtube.com/watch?v=wWNVpFibayA&feature=youtu.be&t=24m25s)

https://github.com/openshift/origin/tree/master/examples/wordpress

https://www.youtube.com/watch?v=wWNVpFibayA&feature=youtu.be&t=24m25s


Free Kubernetes eBookOpenShift Enterprise Test-Drive

(Free 8 hour Environment on AWS)

Questions?

http://red.ht/1NbW2wi http://red.ht/1MQVgqb

Plus some great ways to get started with Kubernetes...

http://red.ht/1NbW2wi

http://red.ht/1MQVgqb


18

APPENDIX


WHAT ARE LINUX CONTAINERS?Software packaging concept that typically includes an application and all ofits runtime dependencies

● Easy to deploy and portable across host systems

● Isolates applications on a host operating system

● In RHEL, this is done through:● Control Groups (cgroups)● Kernel namespaces● SELinux, sVirt, iptables● Docker

HOST OS

SERVER

CONTAINER

LIBS

APP


TRADITIONAL OS VS. CONTAINERS

Traditional OS Containers

HARDWARE

HOST OS

HARDWARE

HOST OS

CONTAINER

LIBS

APP A

LIBS A LIBS B LIBS LIBS

APP A APP BCONTAINER

LIBS

APP B

Packaged dependencies = faster boot times + greater portability


Developers can access OpenShift via Web, CLI or IDE

OPENSHIFT 3Turn-key solution for Developer Productivity + Container Orchestration