deploying a dockerized distributed application in mesos

Download DEPLOYING A DOCKERIZED DISTRIBUTED APPLICATION IN MESOS

Post on 19-Feb-2017

297 views

Category:

Software

1 download

Embed Size (px)

TRANSCRIPT

  • DEPLOYING A DOCKERIZED DISTRIBUTED APPLICATION TO MESOS

    Julia Mateo

    DEPLOYING A DOCKERIZED DISTRIBUTED APPLICATION IN MESOS

    Julia Mateo

    1

  • Who am I

    Java developer and tech lead at Hortis GRC (Geneva, Switzerland)

    Team jDuchess Swiss

    @juliamateodc@duchessswiss http://jduchess.ch/

    2

    http://jduchess.ch/

  • Summary

    Mesos what is it? Docker A Mesos framework : Marathon Deploying a distributed app in Mesos Service Discovery Load balancing

    3

  • Mesos... what is it?

    Deployment system

    Execution platform

    4

  • Mesos... what is it?

    Deployment system : Mesos : smart deployment system Ansible, Chef, Puppet static configuration Mesos dynamically allocates resources depending on

    cluster load (or other external factors)

    5

  • Mesos... what is it?

    Execution platform Hosting high level applications More flexibility :

    If one slave crashes, app is deployed in new instance (Marathon)

    HAProxy for load balancing Alternative to paying third party Paas solutions

    (Heroku, Azure, Openshift)

    6

  • Mesos... what is it?

    Abstraction of cluster resources

    Share resources across multiple frameworks (versions of the same framework)

    Resource fair sharing : alternative to static partitioning

    Data locality

    7

  • ZKMaster Master

    Executor

    ExecutorSlaves

    Executor Executor

    Executor Executor

    Executor Executor Executor

    Mesos

    8

  • http://mesos.apache.org/documentation/latest/mesos-architecture/

    Mesos

    9

    http://mesos.apache.org/documentation/latest/mesos-architecture/

  • From https://www.docker.com/whatisdocker/

    Docker is an open platform for developers and sysadmins to build, ship, and run distributed applications.

    DOCKER

    10

    https://www.docker.com/whatisdocker/

  • ServerHost OS

    Hypervisor

    GuestOS

    Guest OS

    Guest OS

    Guest OS

    Libs/ Libs/ Libs/

    App 1 App 2 App 3 App 4

    Libs/

    ServerHost OS

    Docker Engine

    Libs/Bins Libs/

    App 1 App 2 App 3 App 4c1 c2 c3 c4

    VM1VM2VM3VM4

    Docker

    11

  • Docker Images Docker Images

    libcontainer, Union Filesystem

    Centos Ubuntu

    Jetty

    add App.war

    Container 1

    OracleDB

    Container 2

    Read only

    WritableWritable

    Read only

    Containers and images

    12

  • Marathon is a Mesos framework written in Scala

    Provides easy deployment of Docker containers

    Management of long running apps

    Rest API for developers

    Marathon

    13

  • ZKMaster Master

    Executor

    ExecutorSlaves

    Executor Executor

    Executor Executor

    Executor Executor Executor

    Mesos

    14

  • ZKMaster Master

    Executor

    ExecutorSlaves

    Executor Executor

    Executor Executor

    Executor Executor Executor

    Mesos

    Marathon

    15

  • Demo

    cm-voting

    Infrastructure

    Mesos UI

    Marathon UI

    16

  • The infrastructure

    Io

    Europa

    Ganymede

    Callisto

    JupiterMesos master

    Mesos slave

    Mesos slave

    Mesos slave

    Mesos slave

    17

  • Service discovery

    cm-voting

    18

    Mesos slave 1

    Mesos slave 2

  • Service discovery

    cm-voting

    19

    Mesos slave 2

    Mesos slave 1

  • Service discovery

    cm-votingIP ?

    PORT ?

    CREDENTIALS ??

    Mesos slave 2

    Mesos slave 1

    20

  • cm-voting

    Mesos slave 1

    Mesos slave 2

    REPLICATION, LOAD BALANCING...

    Service discovery

    21

  • Service discovery

    cm-voting

    ... AND FAILOVER

    Mesos slave 1

    Mesos slave 2

    22

  • Service discovery with MesosDNS

    https://github.com/mesosphere/mesos-dns

    23

    https://github.com/mesosphere/mesos-dns

  • Demo

    Deploy MesosDNS

    Deploy Mongo

    24

  • lb-marathon

    Tool for managing HAProxy It subscribes to Marathons event bus Updates the HAProxy configuration in

    real time

    Health checking SSL support

    25

  • Deploy 2 instances of cm-voting

    Add external proxy and load balancer : lb-marathon

    Demo

    26

  • MesosDNS : Service discoveryDOES NOT IDENTIFY SERVICE PORTS :

    NEED SRV LOOKUP

    QUERY MIGHT BE CACHED

    AND NOT CORRECTLY LOADED

    NO HEALTH CHECKING

    DOES NOT HAVE FAST FAILOVER

    27

  • Alternative : lb-marathon for service discovery

    cm-voting

    cm-voting

    lb-marathon

    lb-marathon

    Marathon

    Mongolb-marathon

    28

  • Other Mesos frameworks

    Singularity

    Chronos

    29

  • Feedback from developer perspective

    Quite young tools : lots of changes in a short time

    Service discovery and load balancing troubleshooting

    30

  • Feedback from developer perspective

    Developer-oriented APIs Big community

    31

  • Future

    Multitenant workloads : not enough isolation -> app performance affected (multiple apps running in the same machine)

    Support for databases (Cassandra, HDFS, Kafka) Until now mesos -> computed oriented tasks. Disk management.

    32

  • Thank you ! Questions ?

    33

View more >