orchestration tool roundup
TRANSCRIPT
![Page 1: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/1.jpg)
Orchestra)on Tool Roundup -‐ Docker Swarm vs. Kubernetes, TerraForm vs.
TOSCA/Cloudify vs. Heat
![Page 2: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/2.jpg)
Agenda
• Orchestra)on 101.. • Different approaches for orchestra)on • Method of comparison • Comparison • Synergies • Summary -‐ which tool to choose?
![Page 3: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/3.jpg)
Orchestra)on 101
Orchestra)on is a means to Automate Manual Processes
![Page 4: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/4.jpg)
Orchestra)on 101 • Common Characteris)cs
– Use DSL to define a model – Execute processes based on the model – Pass context informa)on between the deployed en))es
– Plugin oriented architecture
• Different assump)ons lead to different approaches – Applica)on Architecture – PlaRorm bias – Infrastructure – Scope of automa)on
![Page 5: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/5.jpg)
Goals of this Exercise
Explore the different
approaches to orchestra)on
Infrastructure Centric
Pure Play Container Centric
![Page 6: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/6.jpg)
Method of Comparison • Same Applica)on Requirements • Full Produc)on Deployment • Broken into three main groups
– Container Centric – Kubernetes, Docker/Swarm – Pure Play –Cloudify/TOSCA, Terraform,
– Infrastructure Centric -‐ Heat
• Out of scope* – PaaS, Configura)on Management (e.g Chef, Puppet, Ansible,..) – Covering all orchestra)ons solu)ons – Deep Dive into each orchestra)on technology
![Page 7: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/7.jpg)
The Test Applica)on
VM
mongod
VM
NodeJS
mongos
VM
NodeJS
mongos
VM
Mongo-‐cfg
VM
Mongo-‐cfg
VM
Mongo-‐cfg
Load Balancer
VM
NodeJS
mongos
VM
mongod
VM
mongod VM
mongod
VM
mongod
VM
mongod VM
mongod
VM
mongod
VM
mongod
![Page 8: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/8.jpg)
Orchestra)on Process -‐ Setup
VM
VM VM
VM
VM
VM
Load Balancer
VM
VM VM
VM VM
VM
VM VM
VM
Create network and compute resources: VMs, security group, network, subnet, routers, LB pool
1
![Page 9: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/9.jpg)
Orchestra)on Process -‐ Setup
VM
mongod
VM
NodeJS
mongos
VM
NodeJS
mongos
VM
Mongo-‐cfg
VM
Mongo-‐cfg
VM
Mongo-‐cfg
Load Balancer
VM
NodeJS
mongos
VM
mongod
VM
mongod VM
mongod
VM
mongod
VM
mongod VM
mongod
VM
mongod
VM
mongod
Install Mongo and Node Binaries
2
![Page 10: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/10.jpg)
Orchestra)on Process -‐ Setup
VM
mongod
VM
NodeJS
mongos
VM
NodeJS
mongos
VM
Mongo-‐cfg
VM
Mongo-‐cfg
VM
Mongo-‐cfg
Load Balancer
VM
NodeJS
mongos
VM
mongod
VM
mongod VM
mongod
VM
mongod
VM
mongod VM
mongod
VM
mongod
VM
mongod
Start mongod processes
Start mongod processes
Start mongod processes
3
![Page 11: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/11.jpg)
Orchestra)on Process -‐ Setup
VM
mongod
VM
NodeJS
mongos
VM
NodeJS
mongos
VM
Mongo-‐cfg
VM
Mongo-‐cfg
VM
Mongo-‐cfg
Load Balancer
VM
NodeJS
mongos
VM
mongod
VM
mongod VM
mongod
VM
mongod
VM
mongod VM
mongod
VM
mongod
VM
mongod
Start mongo-‐cfg proecesses
Start mongo-‐cfg proecesses
Start mongo-‐cfg processes
4
![Page 12: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/12.jpg)
Orchestra)on Process -‐ Setup
VM
mongod
VM
NodeJS
mongos
VM
NodeJS
mongos
VM
Mongo-‐cfg
VM
Mongo-‐cfg
VM
Mongo-‐cfg
Load Balancer
VM
NodeJS
mongos
VM
mongod
VM
mongod VM
mongod
VM
mongod
VM
mongod VM
mongod
VM
mongod
VM
mongod
Start mongos processes,
poin)ng to config servers
Start mongos processes,
poin)ng to config servers
Start mongos processes, poin)ng
to mongo-‐cfg servers
5
![Page 13: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/13.jpg)
Orchestra)on Process -‐ Setup
VM
mongod
VM
NodeJS
mongos
VM
NodeJS
mongos
VM
Mongo-‐cfg
VM
Mongo-‐cfg
VM
Mongo-‐cfg
Load Balancer
VM
NodeJS
mongos
VM
mongod
VM
mongod VM
mongod
VM
mongod
VM
mongod VM
mongod
VM
mongod
VM
mongod
Pick one mongos and ini)alize replica set Pick one mongos and ini)alize replica set
Pick one VM per shard and ini)alize replica set
6
![Page 14: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/14.jpg)
Orchestra)on Process -‐ Setup
VM
mongod
VM
NodeJS
mongos
VM
NodeJS
mongos
VM
Mongo-‐cfg
VM
Mongo-‐cfg
VM
Mongo-‐cfg
Load Balancer
VM
NodeJS
mongos
VM
mongod
VM
mongod VM
mongod
VM
mongod
VM
mongod VM
mongod
VM
mongod
VM
mongod
Pick one mongos and add shards, one at a )me
7
![Page 15: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/15.jpg)
Orchestra)on Process -‐ Setup
VM
mongod
VM
NodeJS
mongos
VM
NodeJS
mongos
VM
Mongo-‐cfg
VM
Mongo-‐cfg
VM
Mongo-‐cfg
Load Balancer
VM
NodeJS
mongos
VM
mongod
VM
mongod VM
mongod
VM
mongod
VM
mongod VM
mongod
VM
mongod
VM
mongod
Pick one mongos and ini)alize data in mongodb
8
![Page 16: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/16.jpg)
Orchestra)on Process -‐ Setup
VM
mongod
VM
NodeJS
mongos
VM
NodeJS
mongos
VM
Mongo-‐cfg
VM
Mongo-‐cfg
VM
Mongo-‐cfg
Load Balancer
VM
NodeJS
mongos
VM
mongod
VM
mongod VM
mongod
VM
mongod
VM
mongod VM
mongod
VM
mongod
VM
mongod
Start nodejs processes Start nodejs processes Start nodejs processes
9
![Page 17: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/17.jpg)
Orchestra)on Process -‐ Setup
VM
mongod
VM
NodeJS
mongos
VM
NodeJS
mongos
VM
Mongo-‐cfg
VM
Mongo-‐cfg
VM
Mongo-‐cfg
Load Balancer
VM
NodeJS
mongos
VM
mongod
VM
mongod VM
mongod
VM
mongod
VM
mongod VM
mongod
VM
mongod
VM
mongod
Add nodejs VMs to LB pool
10
![Page 18: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/18.jpg)
Orchestra)ng in Produc)on • Monitoring and log collec)on • Manual/Auto healing • Manual/Auto scaling • Maintenance:
– Backup and restore – Con)nuous deployment – Infrastructure upgrades and patches
![Page 19: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/19.jpg)
Common Requirements
• Dependency management • Reproducible • Cloneable • Recoverable
![Page 20: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/20.jpg)
Series 1: Container Centric
![Page 21: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/21.jpg)
The Container Difference • Containers viewed as pre-‐baked images, not
“machines” • Why: performance mainly • Also: it’s logical (one to many) • Consequences for orchestra)on:
• CM node prep via Dockerfile • Images poten)ally need non-‐trivial startup to
configure run)me
• Thought: how does one scale the cluster itself?
![Page 22: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/22.jpg)
Quick Overview of Docker Swarm A Docker-‐na)ve clustering system • Use a pool of docker engine hosts • Expose containers as scalable services • Agach storage volumes • Scaling rules (replicas) • Integral support for rolling upgrade • Placement constraints, affinity/an)-‐affinity • Overlay networking
![Page 23: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/23.jpg)
Swarm Architecture
![Page 24: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/24.jpg)
Quick Overview of Docker Compose
A Docker-‐na,ve command line tool for defining mul,-‐container applica,ons
• Single descriptor for complex applica)on • Define services/containers • Define volumes • Define Dockerfile builds • Define start order • Container/service links • Networking configura)on (Docker) • Resource limits (cpu, memory, etc…)
![Page 25: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/25.jpg)
Solu4on Overview -‐ Deploy -‐ Create Cfg Servers
version: "2" services: mongocfg: build: ./mongodb command: mongod -‐-‐dbpath /data/db -‐-‐configsvr -‐-‐port 27019 labels: node-‐type: config ports: -‐ "27019:27019“ environment: “affinity:node-‐type!=config” MONGODB_PORT: 27018 MONGODB_REPL_SET: rs0
![Page 26: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/26.jpg)
Solu4on Overview -‐ Deploy -‐ Create Replica Sets
version: "2" services: mongodb: build: ./mongodb command: -‐-‐shardsvr -‐-‐dbpath /data/db –replSet rs0 labels: node-‐type: shard ports: -‐ "27018:27018“ links: mongoc environment: “affinity:node-‐type!=shard” MONGODB_PORT: 27018 MONGODB_REPL_SET: rs0
![Page 27: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/27.jpg)
Solu4on Overview -‐ Deploy -‐ Create Mongo Proxies
version: "2" services: mongodb: build: ./mongodb command: mongos –configdb mongoc:27017 labels: node-‐type: proxy ports: -‐ "27017:27017“ links: mongoc environment: “affinity:node-‐type!=shard”
![Page 28: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/28.jpg)
Solu4on Overview -‐ Deploy -‐ Create Nodejs containers
version: "2" services: nodejs: build: ./nodejs labels: node-‐type: nodejs ports: -‐ "27017:27017“ links: mongoc environment: “affinity:node-‐type!=nodejs” “affinity:node-‐type==proxy
![Page 29: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/29.jpg)
Solu4on Overview -‐ Deploy -‐ Reconfigure HAProxy
Extract Node.js container IPs using docker inspect and then: for i in 1..{number_of_nodejs_servers} docker exec haproxy1 \ reconfigure.sh \ -‐-‐add=<IP_of_nodejs{i}:port>
![Page 30: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/30.jpg)
Solu4on Overview -‐ Mongodb scale out
Manual Only – external to orchestra)on: • Execute “docker-‐compose scale mongod=X”. • Then use mongo command line to add the new
shard: “sh.addShard(“rs0/mongod_2:27017")
![Page 31: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/31.jpg)
Compose Solu4on Overview – Review
• Sta)c setup straighRorward, considering • Was only trivially clustered:
• Lots of cut and pas)ng needed for addi)onal cluster nodes • Addi)onal mongoc, shard replicas, mongos and nodejs
• S)ll needed to run manual step to add shards aler cluster up.
• Difficult to refer to instances in a generic way. • Stepping back to Swarm might be appropriate for
complex cases, but then DSL is lost.
![Page 32: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/32.jpg)
Docker Swarm -‐ Pros and Cons
Pros ● Easy modeling ● Placement/Affinity ● Overlay networking ● Auto healing ● Simple manual
scaling ● Highly available
Cons ● Basic infrastructure
handling ● Basic workflows ● Only dynamic
orchestration is autohealing (e.g. no autoscaling)
● No Pod concept yet ● No monitoring
![Page 33: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/33.jpg)
Kubernetes
![Page 34: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/34.jpg)
Quick Overview to Kubernetes Container cluster manager • Pods: )ghtly coupled group of containers • Replica4on controller: ensures that a specified
number of pod "replicas" are running at any one )me.
• Networking: Each pod gets its own IP address • Service: Load balanced endpoint for a set of
pods
![Page 35: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/35.jpg)
Kubernetes Architecture
![Page 36: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/36.jpg)
Sample Replication Controller apiVersion: v1beta3 kind: ReplicationController spec: replicas: 3 selector: name: mongod-‐rs1 template: metadata: labels: name: mongod-‐rs1 spec: containers: -‐ command: [mongod, -‐-‐port, 27017, -‐-‐replSet, rs1] image: example/mongod name: mongod-‐rs1
![Page 37: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/37.jpg)
Sample Service Configuration apiVersion: v1beta3 kind: Service metadata: labels: type: nodejs name: nodejs spec: ports: -‐ port: 80 targetPort: 8080 selector: type: nodejs createExternalLoadBalancer: true
![Page 38: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/38.jpg)
Solu4on Overview -‐ Deploy
• Create mongod config servers for i in 1..3 kubectl create -‐f mongod-‐configsvr{i}-‐controller.yaml kubectl create -‐f mongod-‐configsvr{i}-‐service.yaml
• Create mongos router
kubectl create -‐f mongos-‐controller.yaml kubectl create -‐f mongos-‐service.yaml
![Page 39: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/39.jpg)
Solu4on Overview -‐ Deploy -‐ Create Data nodes
for i in 1..{number_of_replica_sets} kubectl create -‐f \ mongod-‐rs{i}-‐controller.yaml # Now configure each replicate set # by picking pod to be the initial “master” # of each replica set and extract all # containers IPs using “kubectl get -‐l ...” # dynamically update replica set # members (this will kick of this process) kubectl create -‐f mongod-‐rs{i}-‐service.yaml
![Page 40: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/40.jpg)
Kubernetes -‐ Pros and Cons
Pros ● (almost) zero
configuration autoheal ● Out of the box load
balancer ● Simple (and auto)
scaling ● Built in service concept ● Rolling upgrades and
rollbacks
Cons ● Relatively complex setup ● Somewhat slower (than
Swarm) ● Limited workflows (heal,
scale, rolling update). ● No inputs/outputs ● No master model
![Page 41: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/41.jpg)
Series 2: Pure Play Orchestra)on
![Page 42: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/42.jpg)
Introduction to Terraform
• By Hashicorp • Simple (in a good way) command
line tool – Resources – Providers and provisioners – Modules – Variables and outputs
![Page 43: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/43.jpg)
Sample Configuration
resource "openstack_compute_secgroup_v2" "nodejs_security_group" { name = "nodejs_security_group" description = "security group for mongodb" rule { from_port = 22 to_port = 22 ip_protocol = "tcp" cidr = "0.0.0.0/0" } rule { from_port = "${var.nodejs_port}" to_port = "${var.nodejs_port}" ip_protocol = "tcp" cidr = "0.0.0.0/0" } }
![Page 44: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/44.jpg)
Sample Configuration
# # Create a Network # resource "openstack_networking_network_v2" "tf_network" { region = "" name = "tf_network" admin_state_up = "true" } # # Create a subnet in our new network # Notice here we use a TF variable for the name of our network above. # resource "openstack_networking_subnet_v2" "tf_net_sub1" { region = "" network_id = "${openstack_networking_network_v2.tf_network.id}" cidr = "192.168.1.0/24" ip_version = 4 }
![Page 45: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/45.jpg)
Sample Configuration (Config Server) resource "openstack_compute_instance_v2" "mongoc_host" { count = "3" region = "" name = "mongoc_host" image_name = "${var.image_name}" flavor_name = "${var.flavor_name}" key_pair = "tf-‐keypair-‐1" security_groups = ["mongo_security_group"] network { uuid = "${openstack_networking_network_v2.tf_network.id}" } ... provisioner "remote-‐exec" { scripts = [ "scripts/install_mongo.sh" "start_mongoc.sh" ] } }
![Page 46: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/46.jpg)
Sample Configuration (Shard) resource "openstack_compute_instance_v2" "mongod_host" { count = "3" region = "" name = "mongod_host" image_name = "${var.image_name}" flavor_name = "${var.flavor_name}" key_pair = "tf-‐keypair-‐1" security_groups = ["mongo_security_group"] network { uuid = "${openstack_networking_network_v2.tf_network.id}" } ... provisioner "remote-‐exec" { scripts = [ "scripts/install_mongo.sh" "start_mongod.sh" ] } }
![Page 47: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/47.jpg)
Solution Overview
• Single top level configuration file • Creates: Network, subnet, router, floating IP,
security groups, VMs, LBaaS pool • TF module to model a mongodb shard
– No easy way to specify "I want X occurrences of this module"
– Just copy and paste...
![Page 48: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/48.jpg)
Terraform - Pros and Cons
Pros ● Infrastructure &
Framework neutrality ● Solid support for
OpenStack ● Simple and elegant ● Present plan before
applying ● Support for incremental
updates
Cons ● Configurations are not
portable across cloud providers
● Hard to model non-infrastructure components
● Everything is done in the context of a single resource instance
● No auto-anything
![Page 49: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/49.jpg)
TOSCA / Cloudify
![Page 50: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/50.jpg)
What is TOSCA? TOSCA defines the
interoperable descrip)on of applica)ons; including their components, rela)onships, dependencies, requirements, and capabili)es….
![Page 51: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/51.jpg)
Cloudify – Open Source Implementa)on of TOSCA
Provision
Configure Monitor
Manage
Infrastructure
Can be used as a command line tool or as a managed service
Plugins
CM
Monitoring & Alarming
![Page 52: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/52.jpg)
Cloudify – Open Source Implementa)on of TOSCA
Provision
Configure Monitor
Manage
Monitoring & Alarming
Infrastructure
Can be used as a command line tool or as a managed service
Plugins
CM
![Page 53: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/53.jpg)
Hosted On
So:ware Component Container
(Docker Run,me Capability)
Containee (Docker Run,me Requirement)
Requirements
Capabilities
# NOT YET IN TOSCA SPEC. TO BE INVENTED… repositories: docker_hub: url: xxx credentials: yyy node_templates: docker_webserver: type: tosca.nodes.Container requirements: -‐ host: # omitted for brevity artifacts: -‐ my_image: < URI of Docker Image in Repo. > type: tosca.artifacts.impl.Docker.Image: repository: docker_repo
Container
Container
Docker Rocket
…
artifact_types: tosca.artifacts.impl.Docker.Image: derived_from: tosca.artifacts.Root description: Docker Image TAR mime_type: TBD file_ext: [ tar ]
Docker Hub (Repo.)
• URI of DockerImage • Rela,ve to Repo.
Ar4facts • Docker
Image • .TAR)
Containers Portability in TOSCA
Source: Vmware Proposal
![Page 54: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/54.jpg)
Solution Overview
Mongod-‐shard Mongo replica-‐set
Output: Mogoconfig hosts Shards endpoint
Subsitutable
*Scalable *Scalable
Input: #config instances #Shards #Replica set per shard
Input: #nodeJS instances mongodb deployment id or
MongoConfig Mogo Shards
Output: App EndPoint = Load-Balancer IP/path
Mongo cfg
*Scalable
Ini)aliza)on Ini)aliza)on
Load Balancer
NodeJS
MongoS
*Scalable
*Scalable
![Page 55: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/55.jpg)
Infrastructure setup node_templates: nodecellar_security_group: type: cloudify.openstack.nodes.SecurityGroup properties: security_group: name: nodecellar_security_group rules: -‐ remote_ip_prefix: 0.0.0.0/0 port: { get_property: [ nodecellar, port ] }
![Page 56: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/56.jpg)
Create Mongo Shards mongodb: type: tosca.nodes.mongodb.Shard directives: [substitutable] properties: count: { get_input: servers_count_in_replica_set } requirements: -‐ host: node: mongo_server capabilities: scalable: properties: min_instances: 1 max_instances: 10 default_instances: { get_input: mongodb_rs_count }
MongoDB ReplicaSet MongoDB Server
*scalable
![Page 57: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/57.jpg)
Create Compute Instances
mongo_server: type: tosca.nodes.Compute capabilities: host: properties: *host_capabilities os: properties: *os_capabilities scalable: properties: min_instances: 1 max_instances: 10 default_instances: 5
![Page 58: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/58.jpg)
Create MongoDB Replica Set mongo_db_replica_set: type: tosca.nodes.DBMS requirements: -‐ host: node: mongo_server interfaces: Standard: create: Scripts/mongodb/create.sh configure: implementation: Scripts/mongodb/config.sh inputs: mongodb_ip: { get_attribute: [mongo_server, addr] } start: Scripts/mongodb/start.sh
![Page 59: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/59.jpg)
Create NodeJS Containers
nodecellar_container: type: tosca.nodes.NodeCellarAppContainer properties: port: { get_input: nodejs_app_port } interfaces: cloudify.interfaces.lifecycle: create: inputs:
.... command: nodejs server.js environment: NODECELLAR_PORT: { get_property: [SELF, port] } MONGO_PORT: { get_property: [SELF, database_connection, port] } MONGO_HOST: { get_attribute: [SELF, database_connection, private_address] }
…..
![Page 60: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/60.jpg)
Create Load Balancer
haproxy: type: tosca.nodes.Proxy properties: frontend_port: 80 statistics_port: 9000 backend_app_port: { get_property: [ nodecellar, port ] } requirements: -‐ host: node: haproxy_frontend_host -‐ member: node: nodecellar_container
Get the web containers through relationship and update the load balancer accordingly
![Page 61: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/61.jpg)
Handling Post Deployment through Workflow & Policies
● Cloudify Workflows ● Built in workflows
o Install o Uninstall o Heal o Scale
● Discovery through graph navigation ● Remote/Local execution
Script execution in python with context to the deployment graph
cfy executions start -‐w install ...
![Page 62: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/62.jpg)
Summary TOSCA/Cloudify
Pros
● Infrastructure & Framework neutrality
● Complete Life Cycle Management – day 0,1,2 operations (monitoring, log aggregation, healing, scaling, policies)*
● Model for both infrastructure & software
Cons
● The spec is still evolving ● Cloudify isn’t 100%
complaint yet (Aria) ● Limited set of tooling
![Page 63: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/63.jpg)
Series 3: Infrastructure Centric
• Overview of Heat • Orchestra)ng NodeJS/MongoDB with Heat • Summary – Benefits/ Limita)ons
![Page 64: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/64.jpg)
What is Heat?
Heat provides a mechanism for orchestra)ng OpenStack resources through the use of modular templates.
![Page 65: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/65.jpg)
Heat Architecture
![Page 66: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/66.jpg)
Solution Overview
Mongo replica-‐set
Output: Replica set node hosts ssh-key, private ip to the init node
Input: # of replicas per shard Input:
# of nodeJS instances MongoConfig hosts Mongo Shards hosts
Output: mongos node hosts App EndPoint = Load-Balancer IP/path
mogocfg
Output: mogocfg node hosts
Input: # of config instances
Ini)alize replica-‐set-‐script
Ini)alize MongoS-‐Script
Load Balancer
NodeJS
MongoS
![Page 67: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/67.jpg)
Infrastructure setup resources: secgroup:
type: OS::Neutron::SecurityGroup properties:
name:
str_replace: template: mongodb-‐$stackstr-‐secgroup
params:
$stackstr: get_attr:
-‐ stack-‐string
-‐ value rules:
-‐ protocol: icmp
-‐ protocol: tcp port_range_min: 22
port_range_max: 22
-‐ protocol: tcp port_range_min: 27017
port_range_max: 27019
![Page 68: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/68.jpg)
Create Compute Instances mongo_host: type: OS::Nova::Server
properties: name:
str_replace:
template: $stackprefix-‐$stackstr params:
$stackprefix:
get_param: stack-‐prefix $stackstr:
get_attr:
-‐ stack-‐string -‐ value
image: get_param: image
flavor:
get_param: flavor security_groups:
-‐ get_param: security_group
![Page 69: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/69.jpg)
Create MongoDB Replica Servers
mongodb_peer_servers: type: "OS::Heat::ResourceGroup"
properties: count: { get_param: peer_server_count }
resource_def:
type: { get_param: child_template } properties:
server_hostname:
str_replace: template: '%name%-‐0%index%'
params:
'%name%': { get_param: server_hostname } image: { get_param: image }
flavor: { get_param: flavor }
ssh_key: { get_resource: ssh_key } ssh_private_key: { get_attr: [ssh_key, private_key] }
kitchen: { get_param: kitchen }
chef_version: { get_param: chef_version }
![Page 70: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/70.jpg)
Configure the Replica Servers server_setup: type: "OS::Heat::ChefSolo"
depends_on: -‐ mongodb_peer_servers
properties:
username: root private_key: { get_attr: [ssh_key, private_key] }
host: { get_attr: [mongodb_peer_servers, accessIPv4, 0] }
kitchen: { get_param: kitchen } chef_version: { get_param: chef_version }
node:
mongodb: ruby_gems:
mongo: '1.12.0'
bson_ext: '1.12.0' bind_ip: { get_attr: [mongodb_peer_servers, privateIPv4, 0] }
use_fqdn: false
replicaset_members: { get_attr: [mongodb_peer_servers, privateIPv4] } config:
replset: myreplset
run_list: [ "recipe[config_replset]" ]
![Page 71: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/71.jpg)
Create NodeJS Container nodestack_chef_run: type: 'OS::Heat::ChefSolo'
depends_on: nodestack_node
properties: ...
node:
nodejs_app:
... deployment:
id: { get_param: stack_id }
app_id: nodejs
run_list: ["recipe[apt]", "recipe[nodejs]",
"recipe[ssh_known_hosts]",
"recipe[nodejs_app]"]
data_bags: nodejs:
id: { get_param: stack_id }
nodejs_app:
password: { get_attr: [nodejs_user_password, value] } deploy_key: { get_param: deploy_key }
database_url:
str_replace:
template: 'mongodb://%dbuser%:%dbpasswd%@%dbhostname%' params:
'%dbuser%': { get_param: database_username }
'%dbpasswd%': { get_param: database_user_password }
'%dbhostname%': { get_param: db_server_ip }
![Page 72: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/72.jpg)
Summary
Pros ● Native To OpenStack ● Built-in mapping of all
the OpenStack infrastructure resource types
Cons ● Only applicable to
OpenStack ● Setup / Teardown
focused ● Limited coverage for
production aspects (monitoring, log aggregation, healing, scaling)
![Page 73: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/73.jpg)
Poten)al Synergies
• Magnum -‐ Kubernetes + Docker, Heat
• Cloudify/TOSCA + Docker + ?
• Cloudify/TOSCA + Heat
![Page 74: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/74.jpg)
Which orchestra)on tool should I choose?
![Page 75: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/75.jpg)
Final Words..
![Page 76: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/76.jpg)
The Only Constant Is Change!
![Page 77: Orchestration Tool Roundup](https://reader034.vdocuments.mx/reader034/viewer/2022042619/58a1a7ab1a28ab4a0f8c0d5c/html5/thumbnails/77.jpg)
More Change Ahead..