storage for containers and cloud-native deployments - rancher online meetup - march 2017

© 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc .

Storage for Containers and Cloud-Native Deployments March 28, 2017

#ranchermeetup

© 2017 Rancher Labs, Inc.2 © 2017 Rancher Labs, Inc .

Shannon WilliamsCo-Founder/VP Sales @smw355

Will ChanCo-Founder/VP Engineering @willchan00

#ranchermeetup


Bjorn KolbeckCo-FounderQuobyte @quobyte

#ranchermeetup

Gou RaoCo-Founder and CTOPortworx @raogou


First things first…

This is a not a !

#ranchermeetup


There are rules for a meetup!• We won’t be done on time• Questions are always welcome• Demo, then demo some more• Things will break, be patient

#ranchermeetup

© 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc .6

Join the conversation on Twitter #ranchermeetup


Agenda• Rancher Overview – Shannon • Storage Concepts and Data Persistence in Rancher – Will• Quobyte Overview – Bjorn• Portworx Overview – Gou• Latest Rancher Release Details – Will

#ranchermeetup


Rancher Labs

#ranchermeetup

An open-source software platform for managing containers

A minimalist OS built explicitly to run Docker

© 2017 Rancher Labs, Inc.9

A complete container management platform that makes it easy to…

INNOVATE WITH CONTAINERSwithout compromising flexibility by empowering developers with fast access

to the latest tools

MANAGE APPLICATIONSby simplifying day to day application lifecycle management

RUN CONTAINERSwith the most complete set of container and infrastructure management capabilities

Production ready✔ 20 million+ downloads

✔ Open platform for innovating

✔ Easy to use interface

✔ Multi-tenant

✔ Role based access

✔ 24X7 support

✔ And more….

© 2017 Rancher Labs, Inc.10

Complete Container Management Platform

Application Catalog

Container Orchestration and SchedulingUser MgmtRBAC

AD/LDAPSAML

Ops MgmtCI/CD

RegistriesMonitoring

Networking

Multi-tenant Environments

Environment 1 Environment N

Infrastructure Services

Storage

……. ..Environment 2

Security DNS/LB


#ranchermeetup

Storage for Containers and Cloud-Native Deployments

© 2017 Rancher Labs, Inc.

Docker Volume Plugins• Released as part of Docker 1.8 in Aug 2015.• Since then, there has been many volume

plugins created including storage plugins to:• cloud providers (AWS, GCE, Azure, DO) • storage vendors (NetApp, EMC, Quobyte, Portworx)• more frameworks (rex-ray, openstorage)

• While is this great, you still have to download it, install it, and manage it…


Volume Plugin Manager• Rancher is your Docker volume plugin manager

and provides the following:• Distribution – automatically deploys volume plugins to desired hosts managed by

Rancher so they are made available immediately• Version Management and Upgrades – provides version management and allows

users to automatically update plugins on all hosts with a single click (or API call)• Lifecycle Management – Provides health checks and automatic redeployment of

plugins if things go wrong• Plugin-aware Container Scheduling – provides container scheduling based on

deployment of plugins so you never schedule containers on hosts that have no access to storage


How is this done?• Leverage Rancher’s catalog to for version

management, upgrades, and distribution.• Leverage Rancher for lifecycle management of

the volume plugin which needs to be packaged as a Docker container.

• Leverage Rancher for container scheduling.


Supported Plugins


Supported Plugins• Rancher already supports NFS with EBS and

EFS made GA in 1.6 (end of April)• Open source contributions include plugins for

Rex-ray• Vendor contributions include Netapp, Portworx,

and Quobyte


Docker PluginsStorage Vendors Rancher Supported Cloud Providers

QUOBYTE - THE DATA CENTER FILE SYSTEM

Fast and Reliable Software Storage

Introduction

• Björn KolbeckCo-Founder at Quobyte

2006

2010-2013

201319

Google-Style Infrastructure

Data Center“Warehouse-scale”

Container InfrastructureLinux + Containers + Borg

Google File SystemReplication + Erasure Coding

Applications

20

GIFEE… but I’m not Google?!

21

• Scalability• Agility / Flexibility• Fault-tolerance

• Commodity hardware• Containers (Kubernetes)• Storage?

– Google has custom stack– Everyone else: POSIX & legacy

apps

QuobyteData Center File System

Scale-out shared POSIX file system

Linux Linux Linux

ApplicationsMySQL, LAMP, Cassandra, Elastic

Search…

22

Commodity Serversno RAID, NVRAM, journaling

device…

Data Center File SystemReliability in Software: Storage must be cattle!• Split-brain safe replication• Erasure Coding• End-to-end checksums

• Never trust the hardware!

• Ignore: Broken/slow drives, servers, network cards, packet loss, network splits, broken switches…

23

Data Center File SystemLinear Scalabiliy without Bottlenecks• 2x4x6x Servers = 2x4x6x IOPS, throughput,

capacity• No bottleneck: Big data, video, webserver, hpc…

• Grow compute & storage together

24

Data Center File SystemHigh Performance for All Workloads• High performance for all workload

types– Throughput– IOPS– Metadata intensive

(small file workloads)

– Consistent low latency < 1ms• Share data via interfaces– File system (Linux, Mac,

Win)– S3– Hadoop sta

tread(4k) write(4k)

420µs

25 150µs100µs

Data Center File SystemSmart Data Placement: Optimize your Apps

➡ Tiering➡ Isolation➡ Application-aware

data placement

• File name• Extension• Application• Age /

Access

26

HDDSSD

Database

Data Center File SystemSmart Data Placement: Cassandra fast and cheap

Commit log (*.log)• Append workload• Replication: x3• NVMe• Transactions <

500usec

SSTables (*.db)• Read-only• EC: x1.375• Read optimized

SSDs• Lowest $/IOPS

10

Quobyte + Containers Integrated

• Persistent Volume Plugin (>= 1.4)– Global, replicas, container…– Automatic volume provisioning

• UserId mapping (Quobyte blog)

• Volume plugin• https://github.com/quobyte/docker-volume

28

Get Quobyte Today!• www.quobyte.com/get-i

t• Soon: Rancher Catalog

• [email protected]

• Meet us at

29

http://www.quobyte.com/get-it

http://www.quobyte.com/get-it

mailto:[email protected]


Data Services for Cloud Native Applications


Portworx + Rancher: The Transformation to Cloud Native DevOps Infrastructure

X86 Commodity Servers or Cloud

Monitoring

Orchestration/Scheduling

Minimal OS

Portworx

Cloud Native Infrastructure

Fixed Infrastructure

Security

Kubernetes w/ Rancher

Containers

HP

EMC

NetApp

IBM

Cisco

VMware

Dell

Oracle

RedHat

Microsoft


The Problem: Static, Out of Band Storage Provisioning

SSD

HDDnginx python mysql

SAN

html5 rest sql

Apps hit a wall going to production Traditional Storage forced into a container, Out-of-Band

Applications App Unaware Storage Cloud

Connectors like RexRay

SSDEBS


Persistence

IOPs

Connectors SSD

HDDSAN EBS

S3

Admins Realize that Apps Don’t Want Volumes

1. Don’t make data protection a science project2. Stop trying to connect legacy SAN/NAS/SDS to containers – They are not container or application

aware3. Your applications want on-demand services delivered programmatically via the scheduler4. Storage services need to be “app-aware”, for example Cassandra-aware – Traditional storage is

not.


The Solution: Data Services built for DevOps for ANY Infrastructure

nginx python mysql

html5 rest sqlSSD

HDD

SAN

Cloud

Portworx

SSDEBS

• App-aware • Any infrastructure• Programmatic

No more volumes or storage per application to manage

Rancher with Kubernetes


Programmatic Data Services for DevOps

WordPress WordPress

PX

MySQL

MySQL

persistence

encryption

CoS

namespace

Auth/ACLs

HA Operational ExperienceDelivered as a lightweight container with support for

volumes and graph. Runs on any server, any cloud.

Converged PerformanceData collocated with container

compute

Container Granular ControlsEncryption, CoS, replication,

snapshots are done at container granularity

Hybrid Cloud OpsCloud agnostic deployment and

data accessibility across multiple clouds

S3 Object


Portworx is Topology and Application Aware

Application

…x86 Servers x86 Servers

rack

x86 Servers x86 Servers

rack


rack

vol volvol vol vol

Scheduler

Portworx places an application’s data ON THE NODE that it executes on

Container granular volumes on ANY infrastructure


Portworx vs ScaleIO or CEPH


rack


rack

Portworx Places an Application’ss Data Local to a Few Topology Constrained Servers. Cassandra runs

converged.


rack


rack

x86 Servers

ScaleIO and CEPH shard an application’s data across many servers to get capacity. This breaks convergence. You also don’t have application granular settings on the

volumes.


Portworx is Enterprise Grade Software


Demo: Portworx in the Rancher Catalog

Demo


Why Rancher: from the Portworx Perspective

▪ Start to finish • Launch Containers and Infrastructure services in seconds• Rich Community + Enterprise Catalog of services

▪Deploy anywhere• Support for top cloud providers like Amazon, Digital Ocean,

Packet … • Broad community of users moving to cloud-native microservices


Portworx is Open Core

OpenStorage.org Scheduler Data Layer: OpenStorage is a massively scalable data layer for Containers and Schedulers like Kubernetes and Mesosphere

LCFS Data Path: LCFS provides core data path functionality for Docker images. Check it out on https://github.com/portworx/lcfs

PORX Bundled Data Services: Try out the Portworx community edition at https://github.com/portworx/px-dev


Demo

#ranchermeetup


Latest Release

Rancher 1.5 – March 3, 2017

Key Features:- Templating support for catalog

items- API Interceptor- Enhanced Network Policies- Webhooks – Host Scaling,

DockerHub upgrade- Metadata Improvements- Portainer.io for Swarm UI


Next ReleasesRancher 1.6 – Late April

Key features:- Catalog Enhancements – Ability to add catalogs per environment- HAProxy Enhancements - - Certifying Rancher-EBS - - Scheduling Enhancements - schedule containers evenly across

pools of hosts - K8S 1.6 Support with Docker 1.13/Docker 17.03 and etcdv3


Getting StartedRancher and RancherOS are in GitHub – Get Involved!

#ranchermeetup

http://github.com/rancher

http://github.com/rancher


Even better - try.rancher.com…


Then join a free training class…

http://rancher.com/training

http://rancher.com/training


Thank yourancher.com

#ranchermeetup

storage for containers and cloud-native deployments - rancher online meetup - march 2017

Technology