MESOS AT SCHIBSTEDLEASONS LEARNED

Created by Alan Bover / alan.bover@schibsted.com

MESOS 0.27

Fault-tolerant - Multi-tenant cluster management system

Mostly Resource management, Container execution andAuthorization

Two level-scheduling system

HOW DO WE USE MESOS?

Marathon: Container orchestration platform Chronos: Fault tolerant job scheduler In-house developed framework for fraud detection

ZEUS PROJECT

STABILITY"With Mesos, your experience will be AS GOOD as the

framework you use"

DON'T TRUST ME, TRUST NETFLIX

CHRONOS FRAMEWORKUNREGISTERED

...sched.cpp:736] Got error 'Completed framework attempted to re-register'

...sched.cpp:1320] Asked to abort the driver INFO Error: Completed framework attempted to re-register (com.airbnb.scheduler.mesos.MesosJobFramework:216)INFO Shutting down job scheduler (com.airbnb.scheduler.jobs.JobScheduler:661)...sched.cpp:777] Aborting framework '20150116-150711-4084312256-5050-20769-0001'...sched.cpp:1286] Asked to stop the driver ...sched.cpp:752] Stopping framework '20150116-150711-4084312256-5050-20769-0001'

MODULESExperimental support for Mesos 0.21.0

MASTER CONTENDER ANDDETECTOR

LOGGER MODULES (FROM 0.27.0)

Default logger writes to disk Writing a Custom ContainerLogger is not easy https://github.com/cloudflare/complainer

DEPLOYMENT"Lasagna infrastructure"

DEPLOYMENT

Inmutability: Bake big AMI's (need to contain all services)

Too many boot steps (AWS -> Mesos -> Marathon ->Chronos)

MESOSPHERE + DCOS

KUBERNETES VS MESOS"Is like compare apples to oranges"

OUR OWN MESOS OFFERdef jobInSchibsted(offer: Offer, people: *Person, executorInfo: ExecutorInfo) = { val peopleResource = Resource.newBuilder .setType(SCALAR) .setName("people") .setScalar(Scalar.newBuilder.setValue(people)) .setRole("Systems engineer") .build

Link to zeus speak