coreos battle stories

Download CoreOS Battle Stories

Post on 12-Jan-2017




0 download

Embed Size (px)


  • CoreOS Battle StoriesJari Kolehmainen, Founder

  • Background

  • 2015 Kontena, Inc.

    What is Kontena?

    Open Source container platform built to maximize developer happiness. Works on

    any cloud, easy to setup, simple to use.

  • How does it work?

    Kontena Grid A number of physical or virtual machines Kontena Nodes create a Kontena Grid. The nodes may be located anywhere; in single data center, different AZs or different cloud providers.

    Overlay Network Kontena will automatically create an overlay network powered by Weave and connect all nodes of a Grid. Overlay network enable services to communicate with each other in multi-host, multi-AZ environment.

    Service Discovery Kontena has a built-in service discovery powered by etcd. It is used to automatically assign DNS addresses for any services running in Kontena. It is also used by Kontenas load balancer for zero-downtime operation.

    Orchestration Kontenas orchestrator is distributing, running and monitoring all Kontena Services in a Grid. Services may be stateless or stateful, and they are automatically distributed across Nodes in a Grid.

    Containerized Workloads With Kontena, all containerized workloads are described as Services. Kontena Service is composed of containers based on the same image file. Services may be scaled and linked together to create complex elastic apps.



    Kontena Nodes & Agent Kontena Agent may be installed to any machine capable of running Docker. It

    is running as a privileged container in a machine.

    Kontena Master Kontena Master is orchestrating the

    entire Kontena system. It provides APIs used by Kontena CLI, Web UI and third

    party integrations.

    Kontena Master may be installed as high-availability setup if needed.

  • What about OS?

  • 2016 Kontena, Inc.

    Perfect OS: Requirements

    Minimal footprint Container native Zero maintenance Stable Secure

  • 2016 Kontena, Inc.

    Why Container OS?

    Our expectation from an OS has changed Pets vs Cattle

    Maintaining the system should be easy It needs to be more secure than traditional OS

  • Finding the Right OS

  • 2016 Kontena, Inc.

    Choices (Back Then)

    Boot2Docker CoreOS Project Atomic DIY (not an option, really)

  • 2016 Kontena, Inc.


    Based on Tiny Core Linux Small (24MB download, 5s to boot) No automatic updates Not recommended for production use

  • 2016 Kontena, Inc.


    Based on Gentoo Minimal (~100MB) Designed for containers Focus on security and stability Automatic updates

  • 2016 Kontena, Inc.

    Project Atomic

    Not a new Linux distribution Framework to create on OS from RHEL, CentOS

    and Fedora Designed for containers Focus on security and stability

  • CoreOS

  • Is not just an OS

  • 2016 Kontena, Inc.

    CoreOS Project

    etcd rkt fleet locksmith flannel many more

  • 2016 Kontena, Inc.

    CoreOS Host

  • Automation

  • 2016 Kontena, Inc.

    Kontena Provisioning Goals

    single command that just works register host to etcd cluster register host to Kontena Grid should work on any infrastructure

  • 2016 Kontena, Inc.

    CoreOS Problems

    configuration management etcd cluster / discovery etcd security coordinated auto-updates

  • 2016 Kontena, Inc.

    CoreOS & Configuration Management

    Chef Puppet Ansible Saltstack ??

  • CloudInit

  • 2016 Kontena, Inc.

    Bootstrapping with CloudInit

    de-facto way to initialize cloud instances integrated to CoreOS only sane way to bootstrap

  • etcd

  • 2016 Kontena, Inc.

    Etcd: The Hard Parts

    discovery security (tls certificates) central services vs workers maintenance

  • 2016 Kontena, Inc.

    Initial Implementation

    run etcd inside a container bind etcd only to localhost & overlay network use public discovery service

  • 2016 Kontena, Inc.

    Etcd: Current Implementation

    run etcd inside a container bind etcd only to localhost & overlay network master coordicates etcd discovery

    static ips

  • 2016 Kontena, Inc.

    Etcd: Future Improvements

    automatic failover with magic support for external etcd cluster

  • Automatic Updates

  • 2016 Kontena, Inc.

    Automatic Updates

    several update strategies best-effort, etcd-lock, reboot, off

    our pick: best-effort if etcd is running, locksmith coordinates the

    reboots otherwise just reboot once update is available

  • 2016 Kontena, Inc.

    Automatic Updates

    chaos monkey for free! updates also kontena-agent works like a charm

  • Overlay Network

  • 2016 Kontena, Inc.

    Overlay Network Options

    Flannel by CoreOS Weave Net by Weaveworks Calico Docker Overlay Network Most likely you need one of these

  • 2016 Kontena, Inc.


    bundled with CoreOS depends on working etcd has multiple backends to choose from

  • 2016 Kontena, Inc.

    Weave Net

    simple setup optional encryption multicast, multi-hop, fast datapath dns requires some external coordination

    needs information about other peers

  • 2016 Kontena, Inc.

    Our Pick: Weave Net

    can start before etcd makes it possible to expose etcd only to overlay net secure communication between nodes

    not dependant on infrastructure features easy-ish to orchestrate just works

  • Demo!

  • Summary

  • 2016 Kontena, Inc.

    CoreOS Summary

    the OS part is currently best option for containers etcd is a must, but a little hard to handle pick orchestrator that hides all the complexities automate everything

  • Thank You!