storageos, storage for containers shouldn't be annoying at container camp uk
TRANSCRIPT
Storage for containers shouldn’t
be annoying
Karolis Rusenas@KRusenas
https://github.com/[email protected]
Agile Storage. On-Demand, Anywhere, Made Easy.
What we care about
Developer experience Use existing tools (create and use Docker volumes without ever interacting with StorageOS directly
(UI/CLI/API). Reasonable defaults - compression, encryption, replication…
Operating experience Run StorageOS as a container, separate image or whatever way you like, it’s just a binary! :) API first - UI and CLI reuses same single HTTP API, easy to automate stuff. Pluggable configuration backends for whatever you have in your stack - Consul, Zookeeper, BoltDB,
Etcd (easy to add custom ones). Security Perfomance
Challenges• Persistent storage
• Networking
• Security
• Infrastructure
• Orchestration
Our targets• Persistent storage
• Networking
• Security
• Infrastructure
• Orchestration
Persistent storage
Jim was almost ready to implement his first app with persistent storage running in Docker
container
How easy is it to get persistent storage with
StorageOS?
That’s the line
• sudo docker run -d --name test-redis01 -v test-dev-redis01:/data --volume-driver=storageos redis redis-server --appendonly yes
Usual stuff
Let’s give our new friend a name
Telling Docker to use StorageOS
driverMore of the usual
stuff
Infrastructure
Infrastructure• Cloud providers such as GCE or AWS provide
EBS volumes for persistent storage.
You can always build something on your own…
On-premises infrastructure &
StorageOS• Benefits of containers without moving to cloud by providing EBS volume
alternative via Docker volume plugin.
• Hyper-converged mode - run your workloads on the same nodes as StorageOS controllers for maximum performance.
• Client mode - present virtual volumes to Docker containers, easy access to remote volumes.
• QoS
• Compression
• Data deduplication
• E2E encryption
Agile Storage. On-Demand, Anywhere, Made Easy.
StorageOS Use Cases
Stateful Containers for Databases and Fast DB recovery
Continuous Integration/Delivery
Secure Cloud Mobility and Cost Reduction Performance Acceleration and Volume Management
API
StorageOS insides
Controller Node
control plane
HTTP API NatsUI
(Angular)
Docker
pluginSchedule
rDataplane drivers
Supervisor 5 secret binaries
Configfs
data plane
Controller Node
control plane
HTTP API NatsUI
(Angular)
Docker
pluginSchedule
rDataplane drivers
Supervisor 5 secret binaries
Configfs
Client Node
control plane
Docker
pluginNats HA
Client
Supervisor 5 secret binaries
data plane
controplane
High level overview
controplane controplane
dataplane dataplane dataplane
nats
Consul Consul Consul
http
clienthttp/nats
Docker dataplane
server
Our stack• Consul/BoltDB - store configuration. BoltDB is
useful when running a single node or during development/testing.
• Nats - messaging system
• Go - control plane is written in this awesome language
• C - data plane, mostly for speed and available libraries
Consul/BoltDB
KV store, easy to use, backup. Service discovery (when using BoltDB
it’s not important since we assume that you are running a single node)
Leadership election - some components of the system should be running only on one node so they are all fighting for leadership (i.e. scheduler, retry logic).
Split brain detection
Leadership election.. why?
Sometimes (and quite often) you need only one node in your distributed system performing specific actions, i.e. scheduling, retrying some actions..
Most of the distributed KV stores implement locking mechanism on keys, that could be used to elect leaders and detect leader failures.
Check out https://github.com/docker/leadership - probably not enough code there to include it as a library though.
Node 1 Node 2 Node 3 Node 4 Node 5
Leader’s key
All nodes try to acquire a lock on specific key
Node 1(leader) Node 2 Node 3 Node 4 Node 5
Leader’s key
Only one will succeed
P.S. Don’t forget to use locks with TTL!
Nats (https://nats.io)
Lightweight Server is just a goroutine in your main
process Instant messaging between
components Simple pub/sub or request/reply
syntax Mesh networking
Nats (https://nats.io) auto discovery:https://github.com/nats-io/gnatsd/issues/126
Safer messaging
Messaging wrapper
Dispatch event
Persist event in KV store
Publish event to
Nats
Send event:
Receive event:
Receive event
Update event to “receive”
stateDo stuff
Then we can have a process that constantly looks for forgotten events :)
Statik (https://github.com/rakyll/statik)
Compile your static assets into Go binary!
Suture (https://github.com/thejerf/suture)
Erlang-ish supervisor trees for Go Stable Combined with statik and go-bindata
you can drastically simplify your deployments
Using Docker’s healthcheck functionality (added in 1.12)
https://docs.docker.com/engine/reference/builder/#/healthcheck
Dockerfile:
$docker ps
Useful when you have several dependencies, like KV store.
Thanks!@Storage_OS
We are hiring!http://storageos.com/careers/