coreos + kubernetes @ all things open 2015

Modern Container Orchestration

Kubernetes, CoreOS, and more

@coreoslinux@brandonphilips

Brandon PhilipsCTO, CoreOS

github.com/philips

Easy CoreOS+Kubernetes Setupvagrant, aws, bare metal, etc

coreos.com/kubernetes/docs/latest/

https://coreos.com/kubernetes/docs/latest/


Demo Instructionsgithub.com/philips/hacks

2015-all-things-open

http://github.com/philips/hacks

http://github.com/philips/hacks

What is CoreOS?

The smartest way to run your container infrastructure.

tectonic.com @tectonic

QUAYSecure hosting for private Docker repositories

quay.io @quayio

Why build CoreOS?

you as a sw engineer

your

with Ada.Text_IO;

procedure Hello_World is

use Ada.Text_IO;

begin

Put_Line("Hello, world!");

end;

#include <stdio.h>

int main()

{

printf("Hello, world!\n");

}

package main

import "fmt"

func main() {

fmt.Println("Hello, world!")

}

your containerimage

your /bin/java/opt/app.jar/lib/libc

your /bin/python/opt/app.py/lib/libc

your com.example.app

d474e8c57737625c

your d474e8c57737625c

Signed By: Alice

you as an ops engineer

your

com.example.webappx3

your

???

com.example.webappx3

How do we do it?

reduce API contracts

minimal

kernelsystemdrktsshdocker

pythonjavanginxmysqlopenssl

app

dis

tro

dist

ro d

istr

o di

stro

dis

tro

dist

ro

pythonjavanginxmysqlopenssl

app d

istr

o di

stro

dis

tro

dist

ro d

istr

o di

stro


pythonopenssl-A app1

dis

tro

dist

ro d

istr

o di

stro

dis

tro

dist

ro

javaopenssl-B app2

javaopenssl-B app3


CoreOS

container

dis

tro

dist

ro d

istr

o di

stro

dis

tro

dist

ro

container

container

OS operations

updates

OS operations

manual updates

automatic updates

atomic update with rollback

CoreOS Updates

machine configuration

OS operations

get into the cluster

machine config

[Service]

ExecStart=/usr/bin/kubelet --

api_servers=https://172.17.4.101 --

register-node=true --hostname-

override=172.17.4.201 --cluster_dns=10.

3.0.10 --cluster_domain=cluster.local

--tls-cert-file=worker.pem --tls-

private-key-file=worker-key.pem

distributed configuration

cluster operations

/etcdistributed

AvailableLeader

Follower

UnavailableLeader

Follower

AvailableLeader

Follower

Temporarily Unavailable

Leader

Follower

Available

Leader

Follower

Unavailable

Leader

Follower

what should run

cluster operations

k8s/mesos/etc scheduler

scheduling

getting work to servers

scheduling

$ scp app host:/opt$ ssh host systemd-run /opt/app

$ fab deploy:app

$ fab deploy:collector-app

$ fab deploy deploy:collector-app

$ fab lowest-loadaverage

$ fab lowest-loadaveragehost1

$ fab lowest-loadaveragehost1$ fab -H host1 deploy:job

You

Scheduler API

Scheduler

Machine(s)

while true { todo = diff(desState, curState) schedule(todo)}

$ kubectl run host-info--image=quay.io/philips/host-info--replicas=1

$ kubectl get podsPOD IPhost-info-97wt8 10.2.29.4

$ kubectl scale rc host-info--replicas=2

$ kubectl get podsPOD IPhost-info-97wt8 10.2.29.4host-info-f839d 10.2.29.8

podenv=prodapp=web

podenv=prodapp=web

podenv=prodapp=web

rc web-prodselect(env=prod,app=web)count=1

podenv=prodapp=web


podenv=prodapp=web

podenv=prodapp=web

podenv=prodapp=web

podenv=prodapp=web

podenv=prodapp=web


where is it running

cluster operations

dns, LBs, k8s labels

services

flexible service discovery

k8s labels

podenv=devapp=web

podenv=testapp=web

podenv=prodapp=web

podenv=devapp=web

podenv=testapp=web

podenv=prodapp=web

service test.example.comselect(env=dev,app=web)

service beta.example.comselect(env=test,app=web)

ORselect(env=prod,app=web)

service example.comselect(env=prod,app=web)

podenv=testapp=web

podenv=prodapp=web

podenv=prodapp=web

podenv=devapp=web

podenv=testapp=web

podenv=prodapp=web

service test.example.comselect(env=dev,app=web)

service beta.example.comselect(env=test,app=web)

ORselect(env=prod,app=web)

service example.comselect(env=prod,app=web)

podapp=foo,version=1

service foo.cluster.localselect(app=foo)



service foo.cluster.localselect(app=foo)

$ kubectl expose rc host-info --port=80 --target-port=5483 --type=NodePort

$ curl http://172.17.4.202:32430/

architecture in practice

cluster operations

workerkubelet

workerkubelet

workerkubelet

scheduler& API

workerkubelet

workerkubelet

workerkubelet

workerkubelet

workerkubelet

workerkubelet

scheduler& API

worker &API

works on 1 node too

Easy CoreOS+Kubernetes Setupvagrant, aws, bare metal, etc

coreos.com/kubernetes/docs/latest/



coreos.com/careers

work with us

@coreoslinux@tectonicstack

@brandonphilips

thank you

continuous delivery of the OS

- Linux Kernel API promise

- Containers are required

- Consistency of configuration

$ date -d "-674 days"Mon Jul 1 2013

Alpha

α

Alpha Beta

α β

Alpha Beta Stable

α β S

$ cd coreos/manifest

$ cd coreos/manifest$ git tag | wc -l

$ cd coreos/manifest$ git tag | wc -l329

$ uname -r3.8.0

$ uname -r3.8.0

$ uname -r4.0.0

$ init --versionsystemd 207

github.com/coreos/etcd

ok?

ok?

no

yes

github.com/coreos/etcd

containernetworking

github.com/appc/cni

- Defining external plugins e.g. ipvlan, bridge, etc

- Used in rkt today for setting up network namespaces

- Collaborating with folks from Red Hat, Cisco, and others

192.168.1.10192.168.1.40

10.0.0.3

10.0.0.8 10.0.1.10

10.0.1.20

192.168.1.10192.168.1.40

192.168.1.10192.168.1.40

10.0.0.0/24 10.0.1.0/24

routes to192.168.1.40

192.168.1.10192.168.1.40

10.0.0.0/24 10.0.1.0/24

192.168.1.40

10.0.1.0/24

192.168.1.10

routes to192.168.1.10

coreos + kubernetes @ all things open 2015

Technology