ricon 2014 running riak in a docker cloud using apache brooklyn
DESCRIPTION
RICON 2014 presentation introducing Clocker - The Docker Cloud Maker featuring demonstration of Riak being deployed.TRANSCRIPT
![Page 1: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/1.jpg)
Running Riak in a Docker cloud using Apache Brooklyn
Alex Heneveld @ahtwee<n Duncan Johnston-‐Wa@ @duncanjw
RICON 2014
![Page 2: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/2.jpg)
Agenda • Clocker – Docker – Apache Brooklyn – Apache jclouds – Weave
• Deep Dive • Demonstra<on
![Page 3: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/3.jpg)
Meet Clocker The Docker Cloud Maker
![Page 4: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/4.jpg)
Clocker • What does it do? – Spin up and manage Docker clouds – Serve up containers on-‐demand
• What is it? – Apache Brooklyn Applica<on – Apache Brooklyn Loca<on
• Who started it? – Andrew Kennedy @grkvlt
![Page 5: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/5.jpg)
Docker
![Page 6: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/6.jpg)
Orchestra<on – The New Fron<er “The way in which Docker has enabled workloads to migrate across any opera<ng system and any environment has really kind of freed up development and opera<ons. We want to keep them going at the next layer of the stack, which is orchestra2on. Docker containers have been around for 15 months, so the developers' tools around them have an even shorter lifecycle. We're in the very early days of the category.”
Sco$ Johnston, SVP Product, Docker
![Page 7: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/7.jpg)
Clocker Project • Open Source • Apache Licensed • S<ll in Beta – 0.7.0-‐SNAPSHOT
– h@p://brooklyncentral.github.io/clocker
![Page 8: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/8.jpg)
Apache Brooklyn • Open Source • Donated to the Apache So^ware Founda<on • h@p://brooklyn.incubator.apache.org • Started by @ahtwee<n
![Page 9: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/9.jpg)
Apache Brooklyn • Applica<on Management Pla`orm • Model, Deploy and Manage Blueprints – Services (En<<es) – Metrics (Sensors) – Ac<ons (Effectors)
![Page 10: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/10.jpg)
Brooklyn Blueprint name: Riak Cluster with Webapp services: -‐ type: brooklyn.en<ty.nosql.riak.RiakCluster ini<alSize: 2 id: cluster brooklyn.config: install.version: 2.0.0 -‐ type: brooklyn.en<ty.webapp.jboss.JBoss7Server name: Hello Riak Web App brooklyn.config: wars.root: "h@ps://s3-‐eu-‐west-‐1.amazonaws.com/brooklyn-‐clocker/brooklyn-‐example-‐hello-‐world-‐sql-‐webapp.war" java.sysprops: brooklyn.example.riak.nodes: $brooklyn:component("cluster").a@ributeWhenReady("riak.cluster.nodeList") loca<on: jclouds:so^layer:sjc01
![Page 11: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/11.jpg)
Apache Brooklyn • Deployment – Provisioning – Loca<ons – Installa<on and Customiza<on
• Packages, Scripts, Chef, SaltStack • Management – Policies
• AutoScaling, Resilience, Performance, Access
![Page 12: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/12.jpg)
Apache jclouds • Java Cloud Library – h@p://jclouds.apache.org
• API Agnos<c – CloudStack, OpenStack, AWS EC2, GCE, So^Layer…
• Create Virtual Machines – Return SSH Endpoint – Manage Proper<es
![Page 13: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/13.jpg)
Apache jclouds • Drivers for REST APIs • Docker Driver – Created by Andrea Turli @turlinux
• Virtual Container – Using SSH Daemon – Same Endpoint Type as VM – Composi<on on any Image or Dockerfile
![Page 14: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/14.jpg)
Weave • So^ware Defined Networking – Ethernet Switch – User Space – Docker Container
• Sniffs Traffic on Host • Forwards over TCP • h@p://github.com/zepo/weave/
![Page 15: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/15.jpg)
Weave
![Page 16: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/16.jpg)
Clocker Deep Dive
![Page 17: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/17.jpg)
The Docker Cloud Maker • Apache Brooklyn Applica<on – Docker Infrastructure
• Docker Engine • Docker Containers
– Weave Infrastructure • Weave Container
![Page 18: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/18.jpg)
The Docker Cloud Maker • Apache Brooklyn Loca<on – Des<na<on for Blueprints
• Added Features – Create Containers – Provision Docker Hosts – A@ach to Weave Network – Manage Applica<on
![Page 19: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/19.jpg)
Clocker Architecture
![Page 20: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/20.jpg)
Clocker Architecture
![Page 21: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/21.jpg)
Clocker Features • Applica<on Deployment – Oasis CAMP Blueprint – Same as Core Brooklyn
• Mixed Des<na<ons – Some Virtual Machines – Some Bare Metal – Some Containers
![Page 22: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/22.jpg)
Clocker Features • Applica<on Deployment – Oasis CAMP Blueprint – Same as Core Brooklyn
• Docker Extensions – Container or Image – Placement Strategy – Dockerfile URL
![Page 23: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/23.jpg)
Clocker Placement • Demand Side – New Container
• Supply Side – Where? – Placement Strategy – Provisioning Strategy
![Page 24: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/24.jpg)
Clocker Placement • Placement Strategies – Depth First – Breadth First – CPU Usage – Affinity or An< Affinity – Memory or CPU Core Availability
![Page 25: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/25.jpg)
Clocker Placement • Provisioning Strategy – New Docker Host Loca<on
• Constraints – Docker Infrastructure Constraints – En<ty or Applica<on Constraints
• User Defined Strategies • Intelligent Container Orchestra<on
![Page 26: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/26.jpg)
Container Management • Sources – Docker Image Defini<on – Docker Hub – Dockerfile – Brooklyn En<ty Defini<on
• Create Image Automa<cally
![Page 27: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/27.jpg)
Container Management id: dockerfile-‐mysql name: Docker Hub MySQL Application origin: "https://registry.hub.docker.com/_/mysql/" locations: -‐ my-‐docker-‐cloud services: -‐ type: brooklyn.entity.container.docker.application.DockerfileApplication id: mysql name: MySQL brooklyn.config: docker.dockerfile.url: file://Users/grkvlt/Git/docker-‐library/mysql/5.6/ env: MYSQL_ROOT_PASSWORD: "s3cr3t"
![Page 28: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/28.jpg)
Container Management • Installa<on of Services – Defined by Brooklyn or Dockerfile – Common to all En<ty Instances
• Commit Image – Available for next En<ty
• Push Image – Available for all Hosts
![Page 29: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/29.jpg)
Networking • Shared Weave LAN – Common to All Containers – Private (Link Local) Addresses
• Clocker Controls IP Alloca<on – Applica<ons Segmented by CIDR
• Docker Port Forwarding Access
![Page 30: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/30.jpg)
Networking • S<ll First Steps… • Name Resolu<on – BIND and DNSmasq – Needed for JMX et al
• Enables Many More En<<es • But Needs Tested!
![Page 31: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/31.jpg)
Demonstra2on
![Page 32: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/32.jpg)
Riak Blueprint name: Hello Riak Web App services: -‐ type: brooklyn.en<ty.nosql.riak.RiakCluster ini<alSize: 2 id: cluster brooklyn.config: install.version: 2.0.0 -‐ type: brooklyn.en<ty.webapp.jboss.JBoss7Server name: Web App brooklyn.config: wars.root: "h@ps://s3-‐eu-‐west-‐1.amazonaws.com/brooklyn-‐clocker/brooklyn-‐example-‐hello-‐world-‐sql-‐webapp.war" java.sysprops: brooklyn.example.riak.nodes: $brooklyn:component("cluster").a@ributeWhenReady("riak.cluster.nodeList") loca<on: my-‐docker-‐cloud
![Page 33: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/33.jpg)
Summary • Clocker – Brooklyn + Docker + Jclouds + Weave
• Solves – Docker Networking – Container Placement – Applica<on Defini<on
![Page 34: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/34.jpg)
Web Resources
http://clocker.io/
http://brooklyn.io/
http://docker.io/
http://github.com/zettio/weave/
http://abstractvisitorpattern.co.uk/
![Page 35: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/35.jpg)
Thanks! Ques<ons?
![Page 36: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/36.jpg)
Roadmap Now • Improvements To Networking – DNS and DNSmasq Integra<on – Work in Progress
• Be@er Gepng Started – Self Hos<ng on Localhost – Brooklyn Dockerfile
![Page 37: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/37.jpg)
Roadmap Soon • Be@er Integra<on with Repositories – Docker Hub, Ar<factory, Quay.io – Private Repositories
• Easier Applica<on Defini<on – Open Standard? – Kubernetes Pods?
![Page 38: RICON 2014 Running Riak in a Docker Cloud using Apache Brooklyn](https://reader034.vdocuments.mx/reader034/viewer/2022042700/557d5f28d8b42ae1438b4ed7/html5/thumbnails/38.jpg)
Roadmap Next • Integra<on – Google Kubernetes – ClusterHQ Flocker – Ar<factory
• Improvements – Bootstrapping