lxc docker and the future of software delivery
TRANSCRIPT
![Page 1: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/1.jpg)
LXC, Docker,and the future of software delivery
Linuxcon – New Orleans, 2013
Jérôme Petazzoni, dotCloud Inc.
![Page 2: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/2.jpg)
Outline
● Why Linux Containers?● What are Linux Containers exactly?● What do we need on top of LXC?● Why Docker?● What is Docker exactly?● Where is it going?
![Page 3: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/3.jpg)
Why Linux Containers?
What arewe tryingto solve?
![Page 4: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/4.jpg)
The Matrix From Hell
![Page 5: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/5.jpg)
The Matrix From Hell
![Page 6: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/6.jpg)
Many payloads
● backend services (API)● databases● distributed stores● webapps
![Page 7: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/7.jpg)
Many payloads
● Go● Java● Node.js● PHP● Python● Ruby● …
![Page 8: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/8.jpg)
Many payloads
● CherryPy● Django● Flask● Plone● ...
![Page 9: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/9.jpg)
Many payloads
● Apache● Gunicorn● uWSGI● ...
![Page 10: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/10.jpg)
Many payloads
+ your code
![Page 11: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/11.jpg)
Many targets
● your local development environment● your coworkers' developement environment● your Q&A team's test environment● some random demo/test server● the staging server(s)● the production server(s)● bare metal● virtual machines● shared hosting
+ your dog's Raspberry Pi
![Page 12: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/12.jpg)
Many targets
● BSD● Linux● OS X● Windows
![Page 13: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/13.jpg)
Many targets
● BSD● Linux● OS X● Windows
![Page 14: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/14.jpg)
The Matrix From Hell
Static website ? ? ? ? ? ? ?
Web frontend ? ? ? ? ? ? ?
background workers ? ? ? ? ? ? ?
User DB ? ? ? ? ? ? ?
Analytics DB ? ? ? ? ? ? ?
Queue ? ? ? ? ? ? ?
Development VM
QA ServerSingle Prod
ServerOnsite Cluster Public Cloud
Contributor’s laptop
Customer Servers
![Page 15: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/15.jpg)
Real-world analogy:containers
![Page 16: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/16.jpg)
Many products
● clothes● electronics● raw materials● wine● …
![Page 17: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/17.jpg)
Many transportation methods
● ships● trains● trucks● ...
![Page 18: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/18.jpg)
Another matrix from hell
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
![Page 19: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/19.jpg)
Solution to the transport problem: the intermodal shipping container
![Page 20: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/20.jpg)
Solution to the transport problem: the intermodal shipping container
● 90% of all cargo now shipped in a standard container● faster and cheaper to load and unload on ships
(by an order of magnitude)● less theft, less damage● freight cost used to be >25% of final goods cost,
now <3%● 5000 ships deliver 200M containers per year
![Page 21: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/21.jpg)
Solution to the deployment problem: the Linux container
![Page 22: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/22.jpg)
Linux containers...
● run everywhere– regardless of kernel version
– regardless of host distro
– (but container and host architecture must match)
● run anything– if it can run on the host, it can run in the container
– i.e., if it can run on a Linux kernel, it can run
![Page 23: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/23.jpg)
What are Linux Containers exactly?
![Page 24: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/24.jpg)
High level approach:it's a lightweight VM
● own process space● own network interface● can run stuff as root● can have its own /sbin/init
(different from the host)
![Page 25: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/25.jpg)
Low level approach:it's chroot on steroids
● can also not have its own /sbin/init● container = isolated process(es)● share kernel with host● no device emulation (neither HVM nor PV)
![Page 26: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/26.jpg)
Separation of concerns:Dave the Developer
● inside my container:– my code
– my libraries
– my package manager
– my app
– my data
![Page 27: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/27.jpg)
Separation of concerns:Oscar the Ops guy
● outside the container:– logging
– remote access
– network configuration
– monitoring
![Page 28: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/28.jpg)
How does it work?Isolation with namespaces
● pid● mnt● net● uts● ipc● user
![Page 29: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/29.jpg)
How does it work?Isolation with cgroups
● memory● cpu● blkio● devices
![Page 30: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/30.jpg)
Efficiency: almost no overhead
● processes are isolated,but run straight on the host
● CPU performance = native performance● memory performance = a few % shaved off for
(optional) accounting● network performance = small overhead; can
be optimized to zero overhead
![Page 31: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/31.jpg)
Efficiency: storage-friendly
● unioning filesystems(AUFS, overlayfs)
● snapshotting filesystems(BTRFS, ZFS)
● copy-on-write(thin snapshots with LVM or device-mapper)
This wasn't part of LXC at first; but you definitely want it!
![Page 32: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/32.jpg)
Efficiency: storage-friendly
● provisioning now takes a few milliseconds● … and a few kilobytes● creating a new base/image/whateveryoucallit
takes a few seconds
![Page 33: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/33.jpg)
Docker
![Page 34: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/34.jpg)
What's Docker?
● Open Source engine to commoditize LXC● using copy-on-write for quick provisioning● allowing to create and share images● propose a standard format for containers
![Page 35: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/35.jpg)
Yes, but...
● « I don't need Docker; I can do all that stuff with LXC tools, rsync, some scripts! »
● correct on all accounts; but it's also true for apt, dpkg, rpm, yum, etc.
● the whole point is to commoditize,i.e. make it ridiculously easy to use
![Page 36: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/36.jpg)
Docker: authoring images
● you can author « images »– either with « run+commit » cycles, taking
snapshots
– or with a Dockerfile (=source code for a container)
– both ways, it's ridiculously easy
● you can run them– anywhere
– multiple times
![Page 37: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/37.jpg)
Dockerfile example
FROM ubuntu
RUN apt-get -y updateRUN apt-get install -y g++RUN apt-get install -y erlang-dev erlang-manpages erlang-base-hipe ...RUN apt-get install -y libmozjs185-dev libicu-dev libtool ...RUN apt-get install -y make wget
RUN wget http://.../apache-couchdb-1.3.1.tar.gz | tar -C /tmp -zxf-RUN cd /tmp/apache-couchdb-* && ./configure && make install
RUN printf "[httpd]\nport = 8101\nbind_address = 0.0.0.0" > /usr/local/etc/couchdb/local.d/docker.ini
EXPOSE 8101CMD ["/usr/local/bin/couchdb"]
![Page 38: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/38.jpg)
Docker: sharing images
● you can push/pull images to/from a registry(public or private)
● you can search images through a public index● dotCloud maintains a collection of base
images(Ubuntu, Fedora...)
● satisfaction guaranteed or your money back
![Page 39: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/39.jpg)
Docker: not sharing images
● private registry– for proprietary code
– or security credentials
– or fast local access
![Page 40: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/40.jpg)
Typical workflow
● code in local environment(« dockerized » or not)
● each push to the git repo triggers a hook● the hook tells a build server to clone the code and run
« docker build » (using the Dockerfile)● the containers are tested (nosetests, Jenkins...),
and if the tests pass, pushed to the registry● production servers pull the containers and run them● for network services, load balancers are updated
![Page 41: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/41.jpg)
Hybrid clouds
● Docker is part of OpenStack « Havana »,as a Nova driver + Glance translator
● typical workflow:– code on local environment
– push container to Glance-backed registry
– run and manage containers using OpenStack APIs
● Docker confirmed to work with:Digital Ocean, EC2, Joyent, Linode, and many more(not praising a specific vendor, just pointing that it « just works »)
![Page 42: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/42.jpg)
Docker: the community
● Docker: >160 contributors● latest milestone (0.6): 40 contributors● GitHub repository: >600 forks
![Page 43: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/43.jpg)
Docker: the ecosystem
● CoreOS (full distro based on Docker)● Deis (PAAS; available)● Dokku (mini-Heroku in 100 lines of bash)● Flynn (PAAS; in development)● Maestro (orchestration from a simple YAML file)● OpenStack integration● Shipper (fabric-like orchestration)
And many more
![Page 44: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/44.jpg)
Docker roadmap
● Today: Docker 0.6– LXC
– AUFS
● Tomorrow: Docker 0.7– LXC
– device-mapper thin snapshots (target: RHEL)
● The day after: Docker 1.0– LXC, libvirt, qemu, KVM, OpenVZ, chroot…
– multiple storage back-ends
– plugins
![Page 45: LXC Docker and the Future of Software Delivery](https://reader034.vdocuments.mx/reader034/viewer/2022042607/557d8f89d8b42ac8528b5294/html5/thumbnails/45.jpg)
Thank you! Questions?
http://docker.io/
https://github.com/dotcloud/docker
@docker
@jpetazzo