distributed design and architecture of cloud foundry
DESCRIPTION
In this session we will dig deep into Cloud Foundry's core architecture and design principles. We will discuss the challenges around scaling and operating a large scale service as we combined the PaaS and traditional IaaS layers, and how we achieve multiple updates per week to the system with no perceived downtime. Allowing user to download a single virtual machine that is a complete replica of the service presented some challenges as well, and we will discuss our approach to offering up the downloadable private cloud.TRANSCRIPT
![Page 1: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/1.jpg)
Derek Collison
Design and Architecture
![Page 2: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/2.jpg)
What isCloud Foundry?
2
![Page 3: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/3.jpg)
The OpenPlatform as a Service
3
![Page 4: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/4.jpg)
What is PaaS?
4
![Page 5: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/5.jpg)
Or more specifically, aPaaS?
5
![Page 6: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/6.jpg)
aPaaS
• Application Platform as a Service
• Applications and Services
6
![Page 7: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/7.jpg)
aPaaS
• Application Platform as a Service
• Applications and Services
• Not • VMs
• Memory
• Storage
• Networks
• CPU
7
![Page 8: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/8.jpg)
What isOpenPaaS?
8
![Page 9: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/9.jpg)
OpenPaaS
• Multi-Language
• Multi-Framework
• Multi-Services
• Multi-Cloud, Multi-IaaS
• Hybrid - Public or Private or Both
• OpenSource
9
![Page 10: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/10.jpg)
OpenPaaS
• Multi-Language• Ruby, Java, Scala, Node.js, Erlang, Python, PHP..
• Multi-Framework• Rails, Sinatra, Spring, Grails, Express, Lift
• Multi-Services• MySQL, Postgres, MongoDB, Redis, RabbitMQ
• Multi-Cloud, Multi-IaaS• vSphere, MicroCloud, OpenStack, AWS
10
![Page 11: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/11.jpg)
The Open PaaS
Clou
d Pr
ovide
r Int
erfa
ce
Application Service Interface
Private Clouds
PublicClouds
MicroClouds
11
Data Services
Other Services
Msg Services
vFabric Postgres
vFabric RabbitMQTM
![Page 12: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/12.jpg)
What isour Goal?
12
![Page 13: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/13.jpg)
What was our Goal?
13
Raise the unit of currency to be the application and its associated services,
not the infrastructure
![Page 14: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/14.jpg)
What was our Goal?
14
Best of breed delivery platform for all modern
applications and frameworks
![Page 15: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/15.jpg)
What was our Goal?
15
Favor Choice
and
Openness
![Page 16: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/16.jpg)
How was it Built?
16
![Page 17: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/17.jpg)
How was it Built?
• Kernel (CloudFoundry OSS)• Core PaaS System
• Kernel and Orchestrator Shells• Layered on top of IaaS
• Orchestrator• IaaS creation, management and
orchestration
17
![Page 18: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/18.jpg)
High Level
18
IaaS
Orchestrator
CF Kernel
Hardware - CPU/Memory/Disk/Network
Clients (VMC, STS, Browser)
![Page 19: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/19.jpg)
Basic Premises
• Fail Fast
• Self Healing
• Horizontally Scalable Components
• Distributed State
• No Single Point of Failure
• Should be as simple as possible
19
![Page 20: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/20.jpg)
Basic Patterns
• Event-Driven
• Asynchronous
• Non-blocking
• Independent, Idempotent
• Message Passing
• Eventually Consistent
20
![Page 21: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/21.jpg)
Basic Design
• All components loosely coupled• Few “Classes”, many “Instances”
• Messaging as foundation• Addressing and Component Discovery
• Command and Control
• JSON payloads
• HTTP or File/Blob for data transport
21
![Page 22: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/22.jpg)
Kernel Components
• All dynamically discoverable
• Launch and scale in any order
• Can come and go as needed
• Monitor via HTTP and JSON
• Location independent
22
![Page 23: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/23.jpg)
Kernel Components
• Router
• CloudController
• DEA
• HealthManager
• Service Provisioning Agent
• Messaging System
23
![Page 24: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/24.jpg)
Logical View
24
VMC client STS plugin Browser(user app access)
Routers
CloudControllers App
Services
App
HealthManager
DEA Pool
Messaging
![Page 25: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/25.jpg)
25
Arc
hit
ec
ture
![Page 26: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/26.jpg)
Messaging
26
![Page 27: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/27.jpg)
Messaging
27
“The Nervous System”
![Page 28: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/28.jpg)
Messaging
28
VMC client STS plugin Browser(user app access)
Routers
CloudControllers App
Services
App
HealthManager
DEA Pool
Messaging
![Page 29: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/29.jpg)
Messaging
• Addressing and Discovery• No static IPs or DNS lookups req’d
• Just Layer 4
• Command and Control
• Central communication system
• Dial tone, fire and forget
• Protects *itself* at all costs
• Idempotent semantics
29
![Page 30: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/30.jpg)
Router
30
![Page 31: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/31.jpg)
Router
31
“Traffic Cop”
![Page 32: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/32.jpg)
Router
32
VMC client STS plugin Browser(user app access)
Routers
CloudControllers App
Services
App
HealthManager
DEA Pool
Messaging
![Page 33: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/33.jpg)
Router
• Handles all HTTP traffic
• Maintains distributed routing state
• Routes URLs to applications
• Distributes load among instances
• Realtime distributed updates to routing tables from DEAs
33
![Page 34: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/34.jpg)
CloudController
34
![Page 35: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/35.jpg)
CloudController
35
“The King”
![Page 36: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/36.jpg)
CloudController
36
VMC client STS plugin Browser(user app access)
Routers
CloudControllers App
Services
App
HealthManager
DEA Pool
Messaging
![Page 37: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/37.jpg)
CloudController
• Handles all state transitions
• Deals with users, apps, and services
• Packages and Stages applications
• Binds Services to Applications
• Presents external REST API
37
![Page 38: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/38.jpg)
HealthManager
38
![Page 39: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/39.jpg)
HealthManager
39
“Court Jester”
![Page 40: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/40.jpg)
HealthManager
40
VMC client STS plugin Browser(user app access)
Routers
CloudControllers App
Services
App
HealthManager
DEA Pool
Messaging
![Page 41: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/41.jpg)
HealthManager
• Monitors the state of the world
• Initial value with realtime delta updates to “intended” vs “real”
• Determines drift
• Complains to the CloudControllers when something is not correct
• No power to change state itself
41
![Page 42: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/42.jpg)
DEA
42
![Page 43: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/43.jpg)
DEA
43
“Droplet Execution Agent”
![Page 44: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/44.jpg)
DEA
44
VMC client STS plugin Browser(user app access)
Routers
CloudControllers App
Services
App
HealthManager
DEA Pool
Messaging
![Page 45: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/45.jpg)
DEA (Droplet Execution Agent)
• Responsible for running all applications
• Monitors all applications
• CPU, Mem, IO, Threads, Disk, FDs, etc
• All apps look same to DEA• start and stop
• Express ability and desire to run an application• runtimes, options, cluster avoidance, memory/cpu
• Alerts on any change in state of applications
• Provides secure/constrained OS runtime
• Hypervisor, Unix File and User, Linux Containers*
• Single or Multi-Tenant
45
![Page 46: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/46.jpg)
How does it allWork?
46
![Page 47: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/47.jpg)
Pushing an App
• Client (VMC/STS) pushes meta-data to CC
• Client optionally pushes resource signatures (diff analysis, sys wide)
• Client pushes app resources to CC
• CC puts app together
• CC stages app asynchronously
• CC binds and stages services
• Droplet ready
47
![Page 48: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/48.jpg)
48
Arc
hit
ec
ture
![Page 49: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/49.jpg)
Running an App
• CC asks DEAs for “help”
• First DEA back wins! Simple
• CC sends start request to selected DEA
• DEA pushes the “green” button
• DEA waits and monitors pid and ephemeral port for app to bind
• When app is healthy, sends “register” message
• Register message is seen by HM and Routers
• Routers bind URL to host:port
49
![Page 50: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/50.jpg)
DEAs answer?
• DEAs first determine YES or NO• correct runtime, options, memory, etc
• Then calculate a Delay Taint• SHA hash of application
• memory
• cpu
• Taint allows balancing and selection
50
![Page 51: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/51.jpg)
Scale up & down?
• Exact steps as running the app the first time
• SHA1 taint helps avoid clustering
• memory/cpu taint helps distribute as evenly as possible
• Nothing pre-computed
• Nothing assumed
51
![Page 52: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/52.jpg)
Crashes?
• If your app stops and we did not tell it to, that is a crash
• Crashed apps are immediately detected by DEA and messaged
• Routers disconnect route instantly
• HM will signal CC• something is wrong
• CC will issue run sequence again52
![Page 53: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/53.jpg)
53
Arc
hit
ec
ture
![Page 54: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/54.jpg)
Access to my App?
• All routers understand where all instances of your application are running
• Will randomly pick backend, not semantically aware.
• Will remove routes that are stale or unhealthy
• Session stickiness and replication available, but best to avoid if possible
54
![Page 55: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/55.jpg)
What aboutServices?
55
![Page 56: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/56.jpg)
Services
56
VMC client STS plugin Browser(user app access)
Routers
CloudControllers App
Services
App
HealthManager
DEA Pool
Messaging
![Page 57: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/57.jpg)
Services
• Service Advertisement
• Service Provisioning
• Gateway fronts multi-backends
• Service Nodes scale independent
• App and service talk directly
• API to register into system
• Closure for additional value
57
![Page 58: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/58.jpg)
Provisioning
58
VMC/STS
Routers
CloudControllers Services Gateway
Service NodeMySQL
Service NodeRedis
Service NodeRedis
Messaging
Application
1
2
3
45
6
![Page 59: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/59.jpg)
Access (Direct)
59
Routers
CloudControllers Services Gateway
Service NodeMySQL
Service NodeRedis
Service NodeRedis
Messaging
Application
1
2
Browser(user app access)
![Page 60: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/60.jpg)
Services
60
Cloud Foundry
vSphere
core services
Enterprise Services
SQLFire
apps
service controller service broker
provision/bind
consume consume
bind
VMware Dev Tools Partner Dev Tools
Data Director
Relational DB
![Page 61: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/61.jpg)
Learn more:
www.cloudfoundry.org
blog.cloudfoundry.com
support.cloudfoundry.com
61
![Page 62: Distributed Design and Architecture of Cloud Foundry](https://reader035.vdocuments.mx/reader035/viewer/2022081403/554a0f65b4c905825d8b4910/html5/thumbnails/62.jpg)
62
Thank You