sbg infrastructure tools

The DevOps MeetUp

Seven Bridges InfrastructureTools

Ivan Vandot

SevenBridges.com

The way backWe had one production environment with multiple serversOne beta server with all components with versions matchingcurrent production as a hot�x environmentOne beta server clone for next release, we can call it “staging”One integrations server that never worked

The way backTen production componentsCon�gurations for all services were kept in Git repo in separatedirectories for di�erent environments and componentsCon�gurations were changed manually and merged from onerelease to another

The way backBunraku was on worker instances and Skynet system utilizedBunraku on static server infrastructureNobody knew how to manually con�gure and bring up newenvironment, whether it is a production, development or betaWe knew all build dependencies for all components, but not theruntime

Vayu -> Context -> Minion

ContextWritten in DjangoIt has almost complete APICreated with security in mind

ContextCan clone and merge con�gurations from one environment toanotherCon�gurations are versionedGlobals feature

Context

MinionFirst it was written in PythonMinion was created on the principles of SkynetLater rewritten in Go

MinionGo minion has CLI client called mnn also written in GoIt has JSON based con�guration �les with possibility ofcon�guration inheritance

Minion JSON configuration

{ "=": [ "_component", "_config", "_host" ], "check-url": "https://{{fqdn}}:{{port}}/status", "port": "8889", "start": [ { "#": "start the service", "=": "cmd", "args": [ "start" ], "dir": "/data/app/{{name}}", "env": [ "JAVA_HOME=/usr/lib/jvm/java-8-oracle/jre", "LANG=en_US.UTF-8" ], "exec": "bin/{{name}}-server.sh"

MinionMinion provided us uniformity in managing our servicesRuler and goruler

VayuWritten in DjangoVayu service is keeping the data of all development environmentsand provides web-interface for creating and updating themMinion and Context emerged as a need for Vayu, not as servicesthat were planned

VayuMany things were left to be part of Amazon AMIVayu web interface has not changed much, neither thefunctionality

Vayu

Vayu StateVayu state was added to Vayu

Current State

Multiple production environments, eachwith its own staging environment, more

than 100 vayus

How can something ofinfrastructure move into

production and contribute tothe product

Bunraku muerto

Emerging from infrastructure -Minion

Minit - Minion Initialization

Clockwork

Houston we have a problem!

Clockwork

Houston

Q&A