minimum viable infrastructure

32
Minimum Viable Infrastructure Noah Zoschke [email protected] @nzoschke LA DevOps 09-26-2016

Upload: noah-zoschke

Post on 15-Apr-2017

203 views

Category:

Internet


0 download

TRANSCRIPT

Minimum Viable Infrastructure

Noah Zoschke [email protected]

@nzoschke

LA DevOps 09-26-2016

Goal

• Take a web app

• Run it in the cloud forever

• Sleep well every night

10000 Year Clock from http://www.mawer.com/insights/articles/the-10000-year-clock/

Strategy• Developer provides app code

and manifest

• Community develops strategy and architecture to run it

• Providers operate cloud services to support it

• Software automates setup, monitoring and maintenance

Leroy Buffington skyscraper (1888) from http://patentroom.com/architecture

Single Node Architecture Well Understood

Secure Compute

• CPU

• Memory

• Network

• Crypto

┌────────────────────────┐ ┌──┤ HAProxy ├─┐ │┌─┤ Load Balancer ├┐│ ││ └────────────────────────┘││ ││ ┌──────┐┌──────┐┌──────┐ ││ ││ │apache││ php ││python│ ││ ││ └──────┘└──────┘└──────┘ ││ ││ iptables ││ ││ Network Security ││ │└───────────────────────────┘│ │┌───────┐┌────────┐┌────────┐│ ││syslogd││collectd││Postgres││ ││ Log ││ Metric ││Database││ │└───────┘└────────┘└────────┘│ │┌───────────────────────────┐│ ││ ┌──────────┐ ┌──────────┐ ││ ││ │ key.pem │ │ /var/ │ ││ ││ │ Crypto │ │ Data │ ││ ││ └──────────┘ └──────────┘ ││ ││ ext4 ││ ││ File System ││ │└───────────────────────────┘│ │ Ubuntu OS │ └─────────────────────────────┘ ┌───┐┌───┐┌───┐┌───┐ │CPU││RAM││ETH││SSD│ └───┘└───┘└───┘└───┘

App Workload

• Package

• Config

• Data

• Proxy

Visibility

• Logs

• Metrics

• Events

Single Node Challenges Well Understood

• OS and Software Versions

• Single Point of Failure

• Coarse Scaling

• Constant Pricing

┌────────────────────────┐ ┌──┤ HAProxy ├─┐ │┌─┤ Load Balancer ├┐│ ││ └────────────────────────┘││ ││ ┌──────┐┌──────┐┌──────┐ ││ ││ │apache││ php ││python│ ││ ││ └──────┘└──────┘└──────┘ ││ ││ iptables ││ ││ Network Security ││ │└───────────────────────────┘│ │┌───────┐┌────────┐┌────────┐│ ││syslogd││collectd││Postgres││ ││ Log ││ Metric ││Database││ │└───────┘└────────┘└────────┘│ │┌───────────────────────────┐│ ││ ┌──────────┐ ┌──────────┐ ││ ││ │ key.pem │ │ /var/ │ ││ ││ │ Crypto │ │ Data │ ││ ││ └──────────┘ └──────────┘ ││ ││ ext4 ││ ││ File System ││ │└───────────────────────────┘│ │ Ubuntu OS │ └─────────────────────────────┘ ┌───┐┌───┐┌───┐┌───┐ │CPU││RAM││ETH││SSD│ └───┘└───┘└───┘└───┘

Service Oriented Architecture Evolving Best Practices

Secure Compute

• CPU/Memory → VM

• Network → VPC

• Crypto → Crypto

App Workload

• Package → Image, Container

• Config → Crypto, Blob

• Data → Database

• Proxy → Load Balancer

Visibility

• Logs → Logs

• Metrics → Metrics

• Events → KV, Blob

Minimum Viable Infrastructure Simple Monolith → Simple SOA

┌────────────────────────┐ ┌──┤ HAProxy ├─┐ │┌─┤ Load Balancer ├┐│ ││ └────────────────────────┘││ ││ ┌──────┐┌──────┐┌──────┐ ││ ││ │apache││apache││python│ ││ ││ └──────┘└──────┘└──────┘ ││ ││ iptables ││ ││ Network Security ││ │└───────────────────────────┘│ │┌───────┐┌────────┐┌────────┐│ ││syslogd││collectd││Postgres││ ││ Log ││ Metric ││Database││ │└───────┘└────────┘└────────┘│ │┌───────────────────────────┐│ ││ ┌──────────┐ ┌──────────┐ ││ ││ │ key.pem │ │ /var/ │ ││ ││ │ Crypto │ │ Data │ ││ ││ └──────────┘ └──────────┘ ││ ││ ext4 ││ ││ File System ││ │└───────────────────────────┘│ │ Linux OS │ └─────────────────────────────┘ ┌───┐┌───┐┌───┐┌───┐ │CPU││RAM││ETH││SSD│ └───┘└───┘└───┘└───┘

┌────────────────────────────────────┐ ┌┤ Load Balancer ├┐ │└────────────────────────────────────┘│ │┌─────────────────┐┌─────────────────┐│ ││┌─────┐┌────────┐││ ┌─────┐ ││ │││web 1││worker 1│││ │web 2│ ││ ││└─────┘└────────┘││ └─────┘ ││ ││ VM 1 ││ VM 2 ││ │└─────────────────┘└─────────────────┘│ │ ┌────────┐ │ │ │Database│ │ │ └────────┘ │ │ VPC │ └──────────────────────────────────────┘ ┌──────┐┌─────┐┌───┐┌──────┐┌──┐┌────┐ │Crypto││Image││Log││Metric││KV││Blob│ └──────┘└─────┘└───┘└──────┘└──┘└────┘

SOA Advantages Somewhat Understood

• Service Level Agreements

• Versioned APIs

• Independent Scaling

• Utility Pricing

┌────────────────────────────────────┐ ┌┤ Load Balancer ├┐ │└────────────────────────────────────┘│ │┌─────────────────┐┌─────────────────┐│ ││┌─────┐┌────────┐││ ┌─────┐ ││ │││web 1││worker 1│││ │web 2│ ││ ││└─────┘└────────┘││ └─────┘ ││ ││ VM 1 ││ VM 2 ││ │└─────────────────┘└─────────────────┘│ │ ┌────────┐ │ │ │Database│ │ │ └────────┘ │ │ VPC │ └──────────────────────────────────────┘ ┌──────┐┌─────┐┌───┐┌──────┐┌──┐┌────┐ │Crypto││Image││Log││Metric││KV││Blob│ └──────┘└─────┘└───┘└──────┘└──┘└────┘

SOA Challenges Somewhat Understood

• Competing providers, services and software solutions

• Configuration

• Customization

• Impedance Mismatch

┌────────────────────────────────────┐ ┌┤ Load Balancer ├┐ │└────────────────────────────────────┘│ │┌─────────────────┐┌─────────────────┐│ ││┌─────┐┌────────┐││ ┌─────┐ ││ │││web 1││worker 1│││ │web 2│ ││ ││└─────┘└────────┘││ └─────┘ ││ ││ VM 1 ││ VM 2 ││ │└─────────────────┘└─────────────────┘│ │ ┌────────┐ │ │ │Database│ │ │ └────────┘ │ │ VPC │ └──────────────────────────────────────┘ ┌──────┐┌─────┐┌───┐┌──────┐┌──┐┌────┐ │Crypto││Image││Log││Metric││KV││Blob│ └──────┘└─────┘└───┘└──────┘└──┘└────┘

Virtual Machine (VM) Secure Compute

A Virtual Machine (VM) service provides CPU, Memory, Networking and a server Operating System.

┏━━━━━━━━━━━━━━━━━┓ ┃ ┃ ┃ ┃ ┃ ┃ ┃ VM ┃ ┗━━━━━━━━━━━━━━━━━┛

Virtual Private Cloud (VPC) Secure Compute

A Virtual Private Cloud (VPC) service provides private networking.

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ ┃ ┃ ┃ ┃ ┌─────────────────┐ ┃ ┃ │ │ ┃ ┃ │ │ ┃ ┃ │ │ ┃ ┃ │ VM │ ┃ ┃ └─────────────────┘ ┃ ┃ ┃ ┃ ┃ ┃ VPC ┃ ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛

Cryptography Secure Compute

A Cryptography (Crypto) service provides a way to create, import and rotate an unguessable key for encrypting data, and provides a way to get and audit access to the key for decrypting data.

┌──────────────────────────────────────┐ │ │ │ │ │ ┌─────────────────┐ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ VM │ │ │ └─────────────────┘ │ │ │ │ │ │ VPC │ └──────────────────────────────────────┘ ┏━━━━━━┓ ┃Crypto┃ ┗━━━━━━┛

Image App Workload

An Image service provides a private place to push, store and pull binary application and dependency data.

┌──────────────────────────────────────┐ │ │ │ │ │ ┌─────────────────┐ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ VM │ │ │ └─────────────────┘ │ │ │ │ │ │ VPC │ └──────────────────────────────────────┘ ┌──────┐┏━━━━━┓ │Crypto│┃Image┃ └──────┘┗━━━━━┛

Container App Workload

A Container service provides a way to run many specialized process types, defined by Images and commands, on fewer homogeneous VMs.

┌──────────────────────────────────────┐ │ │ │ │ │┌─────────────────┐┌─────────────────┐│ ││┏━━━━━┓┏━━━━━━━━┓││ ┏━━━━━┓ ││ ││┃web 1┃┃worker 1┃││ ┃web 2┃ ││ ││┗━━━━━┛┗━━━━━━━━┛││ ┗━━━━━┛ ││ ││ VM 1 ││ VM 2 ││ │└─────────────────┘└─────────────────┘│ │ │ │ │ │ VPC │ └──────────────────────────────────────┘ ┌──────┐┌─────┐ │Crypto││Image│ └──────┘└─────┘

Load Balancer App Workload

A Load Balancer service provides a single, stable hostname that accepts network requests and proxies them to one or more healthy Containers.

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┌┫ Load Balancer ┣┐ │┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛│ │ │ │┌─────────────────┐┌─────────────────┐│ ││┌─────┐┌────────┐││ ┌─────┐ ││ │││web 1││worker 1│││ │web 2│ ││ ││└─────┘└────────┘││ └─────┘ ││ ││ VM 1 ││ VM 2 ││ │└─────────────────┘└─────────────────┘│ │ │ │ │ │ VPC │ └──────────────────────────────────────┘ ┌──────┐┌─────┐ │Crypto││Image│ └──────┘└─────┘

Database App Workload

A Database service provides a single network hostname that is used to save, update and delete application data records concurrently by one or more Containers.

┌────────────────────────────────────┐ ┌┤ Load Balancer ├┐ │└────────────────────────────────────┘│ │┌─────────────────┐┌─────────────────┐│ ││┌─────┐┌────────┐││ ┌─────┐ ││ │││web 1││worker 1│││ │web 2│ ││ ││└─────┘└────────┘││ └─────┘ ││ ││ VM 1 ││ VM 2 ││ │└─────────────────┘└─────────────────┘│ │ ┏━━━━━━━━┓ │ │ ┃Database┃ │ │ ┗━━━━━━━━┛ │ │ VPC │ └──────────────────────────────────────┘ ┌──────┐┌─────┐ │Crypto││Image│ └──────┘└─────┘

Log Visibility

A Log service provides a place to send ordered text from app Containers’ stdout and stderr streams, so all the application events can be tailed in real-time and searched later.

┌────────────────────────────────────┐ ┌┤ Load Balancer ├┐ │└────────────────────────────────────┘│ │┌─────────────────┐┌─────────────────┐│ ││┌─────┐┌────────┐││ ┌─────┐ ││ │││web 1││worker 1│││ │web 2│ ││ ││└─────┘└────────┘││ └─────┘ ││ ││ VM 1 ││ VM 2 ││ │└─────────────────┘└─────────────────┘│ │ ┌────────┐ │ │ │Database│ │ │ └────────┘ │ │ VPC │ └──────────────────────────────────────┘ ┌──────┐┌─────┐┏━━━┓ │Crypto││Image│┃Log┃ └──────┘└─────┘┗━━━┛

Metric Visibility

A Metric service provides a place to save numerical data from apps and underlying cloud services so operational properties of the entire system can be aggregated, analyzed and graphed in real-time and reviewed later.

┌────────────────────────────────────┐ ┌┤ Load Balancer ├┐ │└────────────────────────────────────┘│ │┌─────────────────┐┌─────────────────┐│ ││┌─────┐┌────────┐││ ┌─────┐ ││ │││web 1││worker 1│││ │web 2│ ││ ││└─────┘└────────┘││ └─────┘ ││ ││ VM 1 ││ VM 2 ││ │└─────────────────┘└─────────────────┘│ │ ┌────────┐ │ │ │Database│ │ │ └────────┘ │ │ VPC │ └──────────────────────────────────────┘ ┌──────┐┌─────┐┌───┐┏━━━━━━┓ │Crypto││Image││Log│┃Metric┃ └──────┘└─────┘└───┘┗━━━━━━┛

Key-Value (KV) Visibility

A Key-Value (KV) service provides a way to save small amounts of structured data in a highly-available (HA) fashion while still being easy to query for reporting purposes.

┌────────────────────────────────────┐ ┌┤ Load Balancer ├┐ │└────────────────────────────────────┘│ │┌─────────────────┐┌─────────────────┐│ ││┌─────┐┌────────┐││ ┌─────┐ ││ │││web 1││worker 1│││ │web 2│ ││ ││└─────┘└────────┘││ └─────┘ ││ ││ VM 1 ││ VM 2 ││ │└─────────────────┘└─────────────────┘│ │ ┌────────┐ │ │ │Database│ │ │ └────────┘ │ │ VPC │ └──────────────────────────────────────┘ ┌──────┐┌─────┐┌───┐┌──────┐┏━━┓ │Crypto││Image││Log││Metric│┃KV┃ └──────┘└─────┘└───┘└──────┘┗━━┛

Blob Visibility

A Blob service provides a way to save and retrieve large amounts unstructured data in a HA fashion.

┌────────────────────────────────────┐ ┌┤ Load Balancer ├┐ │└────────────────────────────────────┘│ │┌─────────────────┐┌─────────────────┐│ ││┌─────┐┌────────┐││ ┌─────┐ ││ │││web 1││worker 1│││ │web 2│ ││ ││└─────┘└────────┘││ └─────┘ ││ ││ VM 1 ││ VM 2 ││ │└─────────────────┘└─────────────────┘│ │ ┌────────┐ │ │ │Database│ │ │ └────────┘ │ │ VPC │ └──────────────────────────────────────┘ ┌──────┐┌─────┐┌───┐┌──────┐┌──┐┏━━━━┓ │Crypto││Image││Log││Metric││KV│┃Blob┃ └──────┘└─────┘└───┘└──────┘└──┘┗━━━━┛

MVI Apps

Rails App Production Grade┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐

web: │ build: . │ labels: │ - convox.port.443.protocol=tls │ - convox.port.443.proxy=true │ links: │ - db ┌───────────────────┐ │ - redis │ │ TLS Load Balancer │ ports: ┌┤https + websockets ├┐ ┌─────────┐ ┌─────────┐ │ - 80:4000 │ │└────────┬─┬────────┘│ │┌───────┐│ │┌───────┐│ - 443:4001 │ ┌─────┐ │ │ ┌─────┐ │ ││ rake ││ ││ rake ││ │ │ ┌ ─ ─ ─ ─ ─ ─ ─ ─ │ │nginx│ │ │ │nginx│ │ ││resque ││ ││resque ││ worker: Rails Image │ │ └─────┘ │ │ └─────┘ │ │└───────┘│ │└───────┘│ │ build: . │ │ │ ┌─────┐ │ │ ┌─────┐ │ │ worker │ │ worker │ command: rake resque work ──────▶ Ubuntu 16.04 OS │──────▶│ │ruby │ │ │ │ruby │ │ │Container│ │Container│ │ │ │ pg, redis gems │ │puma │ │ │ │puma │ │ └─────────┘ └─────────┘ db: + code │ │ └─────┘ │ │ └─────┘ │ ┌─────────┐ ┌─────────┐ │ image: convox/postgres │ └ ─ ─ ─ ─ ─ ─ ─ ─ │ web │ │ web │ │┌───────┐│ │┌───────┐│ labels: │Container│ │Container│ ││ rake ││ ││ rake ││ │ - convox.health.timeout=60 │ └─────────┘ └─────────┘ ││resque ││ ││resque ││ ports: ┌─────────┐ ┌─────────┐ │└───────┘│ │└───────┘│ │ - 5432 │ │Postgres │ │ Redis │ │ worker │ │ worker │ volumes: │Database │ │Database │ │Container│ │Container│ │ - /var/lib/postgresql/data │ └─────────┘ └─────────┘ └─────────┘ └─────────┘ │redis: │ image: convox/redis │ ports: │ - 6433 └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┘

Challenges• Containers, Images, Load balancers and data services only

• Just like Heroku / Twelve-Factor, but more ports and protocols

• Constraints are good

• Orchestration

• Rolling deploys

• Health Checks

• Resilience is good

MVI Systems

Build System API docker build Load Balancer Blob KV Container daemon Container Image ┌┐ ┌┐ ┌┐ ┌┐ ┌┐ ┌┐ ┌┐ POST code ││ ││ ││ ││ ││ ││ ││ ──────────────▶││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ PUT obj ││ ││ ││ ││ ││ ││ ││──────────────▶││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ PUT item ││ ││ ││ ││ ││ ││ ││───────────────┼┼──────────────▶││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ RUN container ││ ││ ││ RUN container ││ ││ ││ ││───────────────┼┼───────────────┼┼──────────────▶││─ ─ ─ ─ ─ ─ ─ ▶││ ││ ││ ││ ││ ││ ││ ││ ││ ││ GET logs ││ ││ ││ ││ ││ ││ ││ ──────────────▶││─ ─ ─ ─ ─ ─ ─ ─│┼ ─ ─ ─ ─ ─ ─ ─ ┼│─ ─ ─ ─ ─ ─ ─ ─│┼ ─ ─ ─ ─ ─ ─ ─▶││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ GET obj ││ ││ ││ ││◀──────────────┼┼───────────────┼┼───────────────┼┼───────────────││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ BUILD ││ ││ ││ ││ ││ ││ ││◀──────────────││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ TAG ││ ││ ││ ││ ││ ││ ││◀──────────────││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ PUSH ││ ││ ││ ││ ││ ││ ││◀──────────────││ ││ ││ ││ ││ ││ ││─ ─ ─ ─ ─ ─ ─ ─│┼ ─ ─ ─ ─ ─ ─ ─▶││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ ││ UPDATE item ││ ││ ││ ││ ││◀──────────────┼┼───────────────┼┼───────────────││ ││ ││ ││ ││ ││ ││ ││ ││ GET result ││ ││ ││ ││ ││ ││ ││ ──────────────▶││─ ─ ─ ─ ─ ─ ─ ─│┼ ─ ─ ─ ─ ─ ─ ─▶││ ││ ││ ││ ││ └┘ └┘ └┘ └┘ └┘ └┘ └┘

MVI Landscape

IaaS ProvidersAmazon Google Microsoft

VM EC2 Google Compute Engine Azure Virtual Machines

VPC VPC GCE Firewall Rules Azure Virtual Network

Crypto KMS Azure Key Vault

Image ECR GCE Registry

Container ECS Google Container Engine (Kubernetes)

Azure Container Service (DC/OS or Swarm)

Load Balancer ELB Google Cloud Load Balancing Azure Load Balancer

Database RDS Google Cloud SQL Azure SQL

Log CloudWatch Logs Stackdriver Logging Azure Log Analytics

Metric CloudWatch Metrics Stackdriver Monitoring Azure Monitor

KV DynamoDB Google Cloud Bigtable Azure Tables Storage

Blob S3 Google Cloud Storage Azure Blob Storage

https://cloud.google.com/docs/compare/aws/ https://azure.microsoft.com/en-us/services/

Service ProvidersVM VPC Crypto Image Container LB DB Log Metric KV Blob

Digital Ocean Heroku CloudFlare Heroku Postgres Sumo logic MongoLab Rackspace

Cloud Files

Linode CircleCI Compose Loggly DataDog Compose

Rackspace Docker Hub Papertrail Newrelic

Quay Librato

Software ProjectsVM VPC Crypto Image Container LB DB Log Metric KV Blob

KVM OpenStack Vault Docker Distribution

Docker Swarm HAProxy Postgres Elastic

Search RRD Redis minio

Xen Knox GitLab Kubernetes nginx MySQL Splunk Graphite MongoDB

VMWare Jenkins Mesos Prometheus Cassandra

VirtualBox Nomad CouchDB

MVI Open Questions• Future additions or simplifications?

• Three database types?

• Shared File System

• Lambda

• Anti-patterns?

• Service discovery

• Overlay networking

• Naming Things

• What knobs and gauges?

MVI Key Takeaways• We can support any app or system with 11 component SOA

• Building with MVI services has advantages:

• Off-the-shelf components

• Privacy

• Reliability

• Scale

• Time to first deploy

• Total cost of ownership

Thanks!

• Feedback and questions to @nzoschke

• See MVI in action

• https://github.com/convox/rack

• https://github.com/nzoschke/mvi