-defined datacenter

metadata

mike dvorkin aspiring super-villain

twitter: @dvorkinistaemail: mike@dvorkinista.com

Death by Micromanagement

Automation has been an attempt at industrialization of micromanagement practices

do do do do do do do do do dodo do do do

Do this sequence of things

linearly orchestrated workflows how to do, not what to achieve

Automated micromanagement with highly quantized choices. Linear operational

convergence. Fragile, little fault recovery, self-healing

AI will help us!

AI will solve everything. AI will tell us what’s

wrong. AI will reverse-

engineer and tell us what the original

intent was. You, know…

It works for Google

In the datacenter galaxy…

infrastructure

virtualization

infrastructure cloud

up-the-stack tools

custom scripts, point tools, process oriented orchestration per instance of compute.

Boxes are a unit of control

VM is the unit of control

custom scripts, process oriented orchestration, fragile integration.

custom scripts, process oriented orchestration, fragile integration.

Allows allocation of one or several VMs at a time on a wall of gear.

VM is the unit of controlScripting frameworks of varying complexity. Linearly orchestrated.

Single Instance is still the the unit of control

It’s all about application… But the application knowledge is not preserved or tracked anywhere in the control stack.

Original intent is lost!

application

Introducing Metadata…

infrastructure

virtualization

infrastructure cloud

up-the-stack tools

Metadata: description of requirements on infrastructure, platforms and other components

app metadata

automatically enforcedself convergentfault-tolerant

Define application structure, its behaviors, dependencies on other apps, infra, platforms…. instance-independentapplication-centricarchitecture-agnostic

application

fully automate entire application, its lifecycle and all of infrastructure underneath…

What is App Metadata?

AP

P

comp

comp

vm

vm

vm

phys

phys

privateoutsid

e

APP

shared service

s

APP

comp

vm

vm

vm

vm

*

shared service interface

*

publicoutsid

e

storage space

app owner

compute

scaling/ha & redundancy

boot

placement rules

component requirements

networking dependency

privacy propagation

interactivity requirements

quality privacy

redundancy capacity

storage requirements

networking

shared services

dependency

security

external entity interactivity

schedules

ordering

staging scripts

img config scripts

bring up rules

Description of app structure, its infrastructure dependencies, and behaviors

Recursive App Relationships

Applications can have dependencies on other applications

AP

P

comp

comp

vm

vm

vmphy

s

phys

private

outside

APP

shared

services

APP

comp

vm

vm

vm

vm

*

shared service interface

*

publicoutsid

e

storage space

Explicit relationships with externally defined apps Shared services offered by tenants or infrastructure

Easily described by metadata specifying such structural relationships

app owner

A building block of service centric architectures

…

app owner

app owner

Define Intent: How do apps talk to each other

Automate instrumentation of intent

The Logical Container

app metadata

meta data

logical overlay

Physical or Virtualnetwork

stora

ge

com

pute

virtualization

security

container

vm phys

vm vm

self enforced logical container

Policy Surfacesintent

capabilities and state

op

s constra

intsg

overn

ance

Enforcement Objectivesintent

capabilities and state

op

s constra

intsg

overn

ance

desire/intent

realitydis

appoin

tment

disappointment := desire - reality

objective: disappointment 0

On decentralization of enforcement

Fully distributed implementation of “promise theory”

policy authority

scheduler

graphobserv

erboot

pe

pe

pe

pe

pe

pe

pe

pecon

trol hiv

e

…..

comp switch storage

… … …

pe

pe

pe

All enforcement is performed as close to the “metal” as possible, but not closer Multi-level enforcement loop Scale Fast reaction time

Thank you.twitter: @dvorkinista