Copyright © 2017 Mirantis, Inc. All rights reserved
Contributions to Kubernetes Ecosystem
Moscow Kubernetes Meetup15.02.2017
2017 | www.mirantis.com
● Why we’re getting involved in Kubernetes community?○ Containers and Kubernetes are the future○ Need to improve the state of Kubernetes on Bare Metal○ Our customers want Kubernetes○ Open contribution process
■ Much more meritocratic approach then in case of Docker Swarm or Mesos
● It started with Kubernetes on OpenStack via Murano
The beginning of the story
2017 | www.mirantis.com
● Fixing bugs, writing tests● Participating in SIG and Community meetings● Helping with project management● Trying to add features we need
Getting started with participating in k8s community
2017 | www.mirantis.com
● Lack of reviewers● Complicated discussions about new functionality
○ DaemonSet upgrades and anti-affinity○ Multiple runtimes in Kubelet○ Node evacuation / maintenance (partially in)○ ConfigMap templates (rejected)
● It’s hard to land XXL patches in general● The upstream wants to keep the core small
○ They want Kubernetes to be “the Linux kernel of distributed systems”
Problems with working upstream
2017 | www.mirantis.com
● Third Party Resources● Container Runtime Interface● External controllers that connect to apiserver● Provide external tooling around kubernetes,
including one used for cluster setup
Extending Kubernetes without changing the core
2017 | www.mirantis.com
● k8s-AppController: managing complex deployments● k8s-externalipcontroller: external IP support for bare
metal k8s clusters● Kargo: setting up a Kubernetes cluster
○ It was not Mirantis project in the beginning, but as of now it’s mostly ours
● Virtlet: running VM workloads on Kubernetes clusters
Mirantis projects belonging to k8s ecosystem
2017 | www.mirantis.com
● kubeadm-dind-cluster: running multinode development clusters locally using Docker-in-Docker
● We didn’t stop being active in k8s core○ we continue fixing bugs○ also adding new features, e.g. for example, we’re working on
DaemonSet upgrades● We lead SIG On-Prem, SIG-OpenStack and SIG-PM● Mirantis is a member of CNCF
Mirantis projects belonging to k8s ecosystem
2017 | www.mirantis.com
● The very initial purpose of k8s was mostly running “cattle” workloads
● Support for stateful workloads is improving over time○ For instance, there’s support for PVs, StatefulSets, init
containers etc.● Defining dependencies between Kubernetes objects
is hard○ E.g. a web app pod may need to wait for its database to
become ready. This may be a problem for legacy apps
AppController: managing complex deployments
2017 | www.mirantis.com
AppController: managing complex deployments
Kubernetes cluster
AppController pod
ThirdPartyResources
Resource Definitions Dependencies
Kubernetes Objects
Kubectl (operator)
Creates
Extends API
CreatesCreates
Creates when
dependencies are met
Reads
Starts
Retrieves status
2017 | www.mirantis.com
● AppController represents k8s objects and their dependencies as TPRs
● k8s objects are created when their dependencies are satisfied○ E.g. pod can depend on a service or a job that needs to be
complete before the pod is created○ Objects can depend on objects that are created by
AppController or pre-existing k8s objects● Helm integration is WiP● Application Lifecycle Management (planned)
AppController: managing complex deployments
2017 | www.mirantis.com
● Ansible-based Kubernetes installer● Supports AWS, GCE, Azure, OpenStack and BM● Supports HA● Flexible deployment options
○ A possibility to choose network plugin, load balancer, rkt support for core services etc.
● Support most popular Linux distributions● The project has extensive CI setup● Battle-tested on Scale Lab (up to 1000 nodes)● There are kubeadm integration plans
Kargo: setup a Kubernetes cluster
2017 | www.mirantis.com
● Some legacy applications can’t be easily containerized○ Substantial effort may be necessary for the transition, there
can be licensing problems, there’s need for extra isolation and so on
● Virtlet runs VMs as Pods, supports QCOW2 images● VMs can communicate with other Pods and access
cluster services
Virtlet: running VM workloads on k8s clusters
2017 | www.mirantis.com
● The implementation is based on Container Runtime Interface○ CRI is also being used by cri-o, hyper, rkt○ It will be also be used for Docker in Kubernetes soon
● We’re making it easy to install Virtlet on k8s clusters○ It can run as a DaemonSet and still avoid chicken-and-egg
problem thanks to multiple runtime support provided by CRI Proxy
● “Futuristic” use case: running Unikernel applications on Kubernetes cluster
Virtlet: running VM workloads on k8s clusters