there's more to docker than the container · there's more to docker than the container...
TRANSCRIPT
There's More to Docker than the Container The Docker Platform
Fabio Chiodini
dotnext Team – Dell EMC @FabioChiodini
github.com/kacole2 github.com/FabioChiodini
Kendrick Coleman
@KendrickColeman {code} by Dell EMC
© Copyright 2017 Dell Inc. 2
Open source at Dell EMC
– Contribute to meaningful OSS projects – Create new thought leading OSS applications – Drive awareness of OSS opportunities with Dell EMC
product teams – Participate in relevant community engagement projects – Act in the interest of building a community
{code} by Dell EMC is a group of passionate open source engineers and advocates working to build a community around software-based infrastructure.
Platinum Sponsor
© Copyright 2017 Dell Inc. 3
© Copyright 2017 Dell Inc. 4
© Copyright 2017 Dell Inc. 5
Monolithic vs. Microservices
© Copyright 2017 Dell Inc. 6
Applications Are Changing
Loosely Coupled Services
Many Small Servers
~2000 Today
Monolithic
Big Servers
Slow changing Rapidly
updated
© Copyright 2017 Dell Inc. 7
Challenge: The Matrix From Hell
Virtual machines
Server Public Cloud
Disaster Recovery
Developer Laptop
Server Cluster
Data Center
Static Website
Web Front End Background Workers
User DB Analytics DB
Queue API Endpoint
Development Test & QA Production Scale Out
© Copyright 2017 Dell Inc. 8
2013: Enter The Docker Container
• Packages up software binaries and dependencies
• Isolates software from each other
• Container is a standard format
• Easily portable across environment
• Allows ecosystem to develop around its standard
© Copyright 2017 Dell Inc. 9
Eliminating The Matrix
Static Website
Web Front End
Background Workers
User DB Analytics DB
Queue API Endpoint
Any App Anywhere
Composable Dynamic Portable
© Copyright 2017 Dell Inc. 10
Docker Image The basis of a Docker container
Docker Container The standard unit in which the application service resides
Docker Engine Creates, ships and runs Docker containers deployable on physical or virtual host locally, in a datacenter or cloud service provider
Docker Registry/Hub On-premises registry or Docker Hub for image storing and collaboration
Docker Basics
DEMO Docker basics
© Copyright 2017 Dell Inc. 12
What Just Happened? docker run –d –-name webserver –p 5000:80 nginx
Docker Hub
Dockerfile
############################################################ FROM debian:jessie MAINTAINER NGINX Docker Maintainers "[email protected]" ENV NGINX_VERSION 1.9.15-1~jessie RUN apt-key adv --keyserver hkp://pgp.mit.edu:80 --recv-keys 573BFD6B3D8FBC641079A6ABABF5BD827BD9BF62 \ && echo "deb http://nginx.org/packages/mainline/debian/ jessie nginx" >> /etc/apt/sources.list \ && apt-get update \ && apt-get install --no-install-recommends --no-install-suggests -y \ ca-certificates \ nginx=${NGINX_VERSION} \ nginx-module-xslt \ nginx-module-geoip \ nginx-module-image-filter \ nginx-module-perl \ nginx-module-njs \ gettext-base \ && rm -rf /var/lib/apt/lists/* # forward request and error logs to docker log collector RUN ln -sf /dev/stdout /var/log/nginx/access.log \ && ln -sf /dev/stderr /var/log/nginx/error.log EXPOSE 80 443 CMD ["nginx", "-g", "daemon off;"]
Github
© Copyright 2017 Dell Inc. 13
Cui Prodest?
• Clean and portable runtime environment • No missing dependencies during
deployments • Run each app in its own isolated container • Improves speed and reliability of CI/CD
systems • Eliminates overhead issues normally
associated with VMs
DEV - Build once, run anywhere
• Universal Packaging • Built-in orchestration • Secure by default • App centric networking and service
discovery • Extensible architecture to work with 3rd
party integrations like storage and networking with no code changes
OPS - Configure once, run anything
© Copyright 2017 Dell Inc. 14
Containers “vs” VMs App
A App
A App A’
Guest OS
Guest OS
Guest OS
Bins/Libs
Bins/Libs
App
A
Bins/Libs
App A
Original app
Copy of app
App A’
V M s
C O N T A I N E R S
Modified app
Hypervisor
Infrastructure
Operating System
Infrastructure
Container Runtime
© Copyright 2017 Dell Inc. 15
DOCKER != CONTAINERS
© Copyright 2017 Dell Inc. 16
Docker =! Containers
Docker Engine
Linux Container
Registry
docker build ……. docker push ……. docker pull ……… docker run ……...
Docker (Engine) provides
application life cycle capabilities
Containers provide a mechanism to instantiate the code (shipped
as a docker image)
*Borrowed from@mreferre
© Copyright 2017 Dell Inc. 17
DOCKER != DOCKER
© Copyright 2017 Dell Inc. 18
3/2/17 https://news.ycombinator.com/item?id=13774295
© Copyright 2017 Dell Inc. 19
CONTAINERD != DOCKER
© Copyright 2017 Dell Inc. 20
CONTAINERD SUPPORTS THE DOCKER PLATOFRM
© Copyright 2017 Dell Inc. 21
DOCKER != CLOUD NATIVE
© Copyright 2017 Dell Inc. 22
THE CLOUD NATIVE LANDSCAPE
*From Cloud Native Computing Foundation (CNCF)
© Copyright 2017 Dell Inc. 23
Docker in 2017
Open Source Project* • 3300 contributors • 43K+ GitHub stars • 900K+ Dockerized apps • 240 Meetups in 70 countries • 95K Meetup members • 12B+ Docker Image Downloads • 14M Docker hosts
Containers as a Service provider • Integrated platform for dev and IT • Commercial technical support Docker project sponsor • Primary sponsor of Docker project • Supports project maintainers
Dockercon 2017 • Closed with 5500 attendees (4k in 2016)
The docker Project Docker Inc
*The original docker project is now under the name of Moby: https://github.com/moby/moby
© Copyright 2017 Dell Inc. 24
CE vs EE DOCKER COMMUNITY EDITION FEATURES
• The latest Docker version with integrated tooling to build, test and run container apps
• Available for free with software maintenance for the latest shipping version
• Integrated and optimized for developer desktops, Linux servers and clouds
• Monthly Edge and quarterly Stable release channels available
• Native desktop or cloud provider experience for easy onboarding
• Unlimited public and one free private repo storage as a service *
• Automated builds as a service *
• Image scanning and continuous vulnerability monitoring as a service *
© Copyright 2017 Dell Inc. 25
CE vs EE
DOCKER ENTERPRISE EDITION FEATURES
• Certified Infrastructure provides an integrated environment for enterprise Linux (CentOS, Oracle Linux, RHEL, SLES, Ubuntu) Windows Server 2016 and Cloud providers like AWS and Azure
• Certified Containers provide trusted ISV products packaged and distributed as Docker containers - built with secure best practices cooperative support
• Certified Plugins provide networking and volume plugins and easy to download and install containers to the Docker EE environment.
• REX-Ray in the Docker Store!
© Copyright 2017 Dell Inc. 26
Why Does Dell EMC Care?
• Databases – Postgres, MongoDB, MySQL, MariaDB, Redis, Cassandra
• Search, Analytics, Messaging – ElasticSearch, LogStash, Kafka, RabbitMQ
• Content Management – Wordpress, Joomla, Drupal, SugarCRM
• Service Discovery – Consul, Zookeeper, etcd
• Continuous Integration and Delivery – Jenkins, GitLab, SonarQube, Selenium, Nexus
• Custom Applications – That Java app your company built
Stateful and persistent applications
© Copyright 2017 Dell Inc. 27
What's the problem?
• When I run a persistent application in a container, where does my data get stored?
– The container holds the data directory and structure of the entire application
– Optionally use local volumes
• Stateless applications work well – nginx, httpd, kibana, haproxy,
memcached, solr, celery
$ docker r un - v r edi sDat a: / dat a r edi s
redisData
/etc /var /bin /opt /data
© Copyright 2017 Dell Inc. 28
What's the problem?
• Lose a container – Lose the data
• Lose a server – Lose the data
• Local data storage – Failed hard drives or failed RAID – Can not scale beyond the physical
limit of the server
/etc /var /bin /opt /data
© Copyright 2017 Dell Inc. 29
Introducing REX-Ray
REX-Ray
The leading container storage orchestration engine enabling persistence for cloud native workloads
rexray.codedellemc.com
• Cloud Native Interoperability
• Open Source
• Enterprise Ready – High Availability – CLI Intuitiveness – Effortless Deployment – Architectural Choices
• Multi-Platform Storage Management – Storage agnostic (block/file/object)
© Copyright 2017 Dell Inc. 30
Solving the problem
• REX-Ray is installed and configured on all hosts in the cluster as a stateless service
• REX-Ray acts as storage plug-in and container engines re-direct storage operations to REX-Ray
– Create/Mount/Unmount/Delete/Snapshot
$ docker r un - - vol ume- dr i ver =r exr ay - v r edi sDat a: / dat a r edi s
/redisData
/etc /var /bin /opt /data
© Copyright 2017 Dell Inc. 31
Solving the problem
• Lose the container or lose the server
– Data persists and remains intact on the remote storage platform
/etc /var /bin /opt
/redisData
© Copyright 2017 Dell Inc. 32
Solving the problem
• Attach the volume to a new container on a different host
– Equivalent of a hard reset. Application starts and resumes from the last write to disk
– Container schedulers can perform automated failover
• Scalability – Application data can scale to the
maximum supported by the storage platform
/etc /var /bin /opt /data /redisData
DEMO REX-Ray = Container Persistence
© Copyright 2017 Dell Inc. 34
Docker For…
© Copyright 2017 Dell Inc. 35
Introducing Docker Infrakit
Declarative, Self-Healing Configuration
Common Infrastructure Experience
Declarative & Self-Healing Infrastructure
Groups
Cluster Configuration Comprised of Instances & Flavor Combination
Instances
Logical Cluster Instance. i.e. EC2 Image, Vagrant VM, Physical Host
Flavors
Role & Healthcheck. i.e. Zookeeper, Swarm Node, etc.
Consistent User Experience for Docker Everywhere
RackHD
What Makes RackHD Different?
DISCOVERY & CATALOGING
TELEMETRY & GENEALOGY
DEVICE MANAGEMENT
CONFIGURATION PROVISIONING
FIRMWARE MANAGEMENT
LOGGING ENVIRON-MENTALS
FAULT DETECTION
ANALYTICS DATA
RackHD Capabilities
© Copyright 2017 Dell Inc. 40
Docker Infrakit + RackHD Process Flow
WORKFLOW API
MONGODB
RAB
BIT
MQ
DH
CP
PXE
/ TFT
P
SYSL
OG
IPM
I / O
BM
RACKHD
INFRAKIT
GR
OU
P
DOCKER INFRAKIT
INST
ANC
E
FLAV
OR
https://github.com/codedellemc/infrakit.rackhd
© Copyright 2017 Dell Inc. 41
How to get Started
1. PXE Boot Bare Metal for Passive Discovery
2. Define a server SKU
3. Create a custom workflow, if needed
Discover Hardware
1. Configure an Infrakit Cluster Configuration
2. Run Infrakit Provisioning
Provision Clusters
https://github.com/codedellemc/infrakit.rackhd
© Copyright 2017 Dell Inc. 42
DOCKER SWARM (swarmkit)
• Provides native clustering capabilities to turn a group of Docker engines into a single, virtual Docker Engine.
• Scale out your application as if it were running on a single, huge computer
DEMO Docker Swarm
© Copyright 2017 Dell Inc. 44
Learn More
• Cloud Native Training – https://github.com/dotnext/training
• {code} Labs – https://github.com/codedellemc/labs
• Join {code} Community – community.codedellemc.com/
© Copyright 2017 Dell Inc. 45
codedellemc.com
community.codedellemc.com
@codeDellEMC
blog.codedellemc.com
{code} by Dell EMC is a group of passionate open source engineers and
advocates working to build a community around software-based infrastructure.
rexray.codedellemc.com
github.com/codedellemc/labs
HOL01 Use REX-Ray & ScaleIO w/ Docker, Mesos and Kubernetes
Kendrick Coleman
@KendrickColeman github.com/kacole2
Fabio Chiodini
@FabioChiodini github.com/FabioChiodini