Download - Production Plone on OpenStack Cloud
Copyright © 2016 Mirantis, Inc. All rights reserved
software.mirantis.com
Production Ploneon OpenStack Cloud
Learn how to automate scalable multi-node deployments of Plone for OpenStack Cloud and see how easy it is to have this deployment tweaked to your taste
Copyright © 2016 Mirantis, Inc. All rights reserved
About me
● 4 Years in OpenStack Community● Now Development Manager for Murano● Started as regular developer
Sergey Melikyan
[email protected]@sergmelikyanfb.me/sergmelikyan #running #sightseeing
#hiking #motorcycles #rock
Copyright © 2016 Mirantis, Inc. All rights reserved
Production OpenStack Plone Deployments
How to deploy Plone on OpenStack
● Introduction to the Cloud and OpenStack● Basics of automation using built-in OpenStack tools● How to automate scalable multi-node deployments● Extend a cloud-ready Plone for your use-cases● Everything else you want to learn today and I can help
Copyright © 2016 Mirantis, Inc. All rights reserved
Introduction to the Cloud and OpenStack
why your head is not in the clouds yet?
Copyright © 2016 Mirantis, Inc. All rights reserved
Definition: Cloud
As described by the National Institute of Standards and Technology (NIST):
“Cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction”
Copyright © 2016 Mirantis, Inc. All rights reserved
Essential Characteristics of the Cloud*
● Provisioning of computing capabilities (compute, network, storage) without human interaction
On-demand Self-service
● Services are available over the network and accessed through standard mechanisms
Broad Network Access
● Capabilities can be elastically provisioned and released to scale rapidly
Rapid Elasticity
● Computing resources are pooled to serve multiple consumers using a multi-tenant model
● Customer generally has no knowledge over the exact location of resources
Resource Pooling
● Resource usage can be monitored, controlled, and reported
Measured Service
*According to NIST (National Institute of Standards and Technology)
Copyright © 2016 Mirantis, Inc. All rights reserved
SPI Model
● Software as a Service (SaaS)● Provides consumers with an application
running on a cloud infrastructure
● Platform as a Service (PaaS)● Provides consumers with the ability to
deploy an application to the cloud utilizing tools and platform supported by the provider
● Infrastructure as a Service (IaaS)● Provides consumers with the ability to
provision fundamental computing resources and deploy arbitrary software
Con
sum
er F
lexi
bil
ity Level o
f Ab
straction
Data CenterHardware, servers, networking
Provision CPU, RAM, VM
INFRASTRUCTURE AS A SERVICE (IAAS)
Remote login to install applications
PLATFORM AS A SERVICE (PAAS)
Browser or thin client access
SOFTWARE AS A SERVICE (SAAS)
Copyright © 2016 Mirantis, Inc. All rights reserved
Cloud Deployment Models*
*According to NIST (National Institute of Standards and Technology)
● Cloud services are only available to members of a single organization.
● Cloud infrastructure can be hosted by the same organization or a third-party.
Private Cloud
● Cloud services are offered to members of a community or organization.
● Cloud infrastructure is hosted by members of the same organization.
Community Cloud
● Cloud services are offered to the general public.● Cloud infrastructure is hosted by anyone.
Public Cloud
● Composition of two or more distinct cloud infrastructure, such as a Private+Public cloud combination.
Hybrid Cloud
Copyright © 2016 Mirantis, Inc. All rights reserved
Definition: OpenStack
As described by the OpenStack Foundation:
“Aims to produce the ubiquitous Open Source Cloud Computing platform that will meet the needs of public and private clouds regardless of size, by being simple to implement and massively scalable.”
Copyright © 2016 Mirantis, Inc. All rights reserved
History
Founded by Rackspace and NASA in 2010Mission: to enable any organization regardless of size to create and offer cloud computing services running on standardized hardware
Copyright © 2016 Mirantis, Inc. All rights reserved
Openstack Foundation
Created in 2012 to provide shared resources to help achieve the OpenStack Mission by Protecting, Empowering, and Promoting OpenStack software and the community around it, including users, developers and the entire ecosystem.
Copyright © 2016 Mirantis, Inc. All rights reserved
Basics of Automation
… using built-in OpenStack tools
Copyright © 2016 Mirantis, Inc. All rights reserved
Goal
● Install & configure development environment● Provision Plone via Heat● Provision Plone via Murano
Develop automated Plone deployment
Copyright © 2016 Mirantis, Inc. All rights reserved
Not going to happen...
What I am not going to do today:● Teach you how to develop software on python● Explain what is version control systems, specifically git● Teach you how to use linux or virtualbox
… but what I should do if I don’t know how to … ?
Copyright © 2016 Mirantis, Inc. All rights reserved
● OS: Ubuntu or CentOS● OpenStack: DevStack● Dependencies:
● git, python, pip, tox, virtualenv● OpenStack System Dependencies● IDE or text editor
Tools
Copyright © 2016 Mirantis, Inc. All rights reserved
Installing DevStack
● Requirements● 4Gb RAM● 2 CPU● 32 Gb Storage
● Recommendation● Use VirtualBox
https://etherpad.openstack.org/p/murano-development-environment
Copyright © 2016 Mirantis, Inc. All rights reserved
OpenStack System Dependencies
Ubuntu● python-dev● libxml2-dev● libxslt-dev● libffi-dev● libpq-dev● python-openssl● mysql-client
CentOS● python-devel● libxml2-devel● libxslt-devel● libffi-devel● postgresql-devel● pyOpenSSL● mysql
Copyright © 2016 Mirantis, Inc. All rights reserved
Tools: pip
pip is a package management system used to install and manage software packages written in Python. Many packages can be found in the Python Package Index (PyPI).
$ curl -O https://bootstrap.pypa.io/get-pip.py
$ sudo python -W ignore get-pip.py --force-reinstall
$ rm get-pip.py
http://pypi.python.org
Copyright © 2016 Mirantis, Inc. All rights reserved
Tools: virtualenv
virtualenv solves a very specific problem: it allows multiple Python projects that have different (and often conflicting) requirements, to coexist on the same computer.
● Installed as dependency for tox● Managed through tox
https://virtualenv.pypa.io/
Copyright © 2016 Mirantis, Inc. All rights reserved
Tools: tox
tox is a generic virtualenv management and test command line tool you can use for: checking your package installs correctly with different Python versions and interpreters or running your tests in each of the environments, configuring your test tool of choice.
$ sudo pip install tox
https://tox.readthedocs.org/
Copyright © 2016 Mirantis, Inc. All rights reserved
or short overview of how Plone looks like
Plone Architecture & Deployment
Copyright © 2016 Mirantis, Inc. All rights reserved
More: http://docs.plone.org/manage/deploying/stack.html
Architecture
Internet Zone / Plone ZODB/Blobs
Copyright © 2016 Mirantis, Inc. All rights reserved
Requirements
● OS: Ubuntu (or anything else...)● Recommended Flavor
● 2Gb RAM● 40 Gb Storage
● Software Prerequisites:● Python 2.7 (dev), built with support for expat (xml.parsers.expat),
zlib and ssl.● libz (dev), libjpeg (dev), readline (dev), libexpat (dev), libssl or
openssl (dev), libxml2 >= 2.7.8 (dev), libxslt >= 1.1.26 (dev).
Copyright © 2016 Mirantis, Inc. All rights reserved
Unified Plone UNIX Installer
● The Universal UNIX Installer lets you choose an installation mode:● a standalone mode - single Zope web application server will be
installed. ● or a ZEO cluster mode - ZEO Server and Zope instances will be
installed.
http://docs.plone.org/manage/installing/installation.html
Copyright © 2016 Mirantis, Inc. All rights reserved
Provision Plone via Heat
or ...heating the cloud!
Copyright © 2016 Mirantis, Inc. All rights reserved
Heat - OpenStack Orchestration
To orchestrate composite cloud applications using a declarative template format through an OpenStack-native
REST API.
Orchestration for Openstack
Copyright © 2016 Mirantis, Inc. All rights reserved
Orchestration vs Configuration Management
● Orchestration is a sub-category of automation, concerned with coordination of multiple component:● For example, servers, networks, volumes, etc..● Orchestration is a “higher form” of automation ● Not just simple or lower-level tasks, but multilayer applications
● Configuration Management is automation of server configuration:● Typically a declarative model, based on “fact” discovery of the server● Abstracts out the underlying implementation detail of service deployment
● Both are needed to fully automate cloud application deployment
Copyright © 2016 Mirantis, Inc. All rights reserved
Heat Capabilities
● Describes the infrastructure for a cloud application – stack (deployment):● OpenStack resources: for example, servers, networks, volumes, etc..● Relationships between resources: for example, this volume is connected to this server● In a text file in the special format – template (blueprint)
● Manages that infrastructure:● Automatically changes the infrastructure when the template is modified and re-applied● Deletes infrastructure when the stack is deleted
● Integrates with software configuration management tools such as Puppet and Chef:● For example: can create VM with puppet server and install puppet clients on VMs● Can pass parameters to cloud-init, etc.
Copyright © 2016 Mirantis, Inc. All rights reserved
Heat Orchestration Template
● Declarative definition of resources, based on YAML● Stack = deployed template, instance of the template● Structure
● Parameters● Resources● Outputs
Full Specification:http://docs.openstack.org/developer/heat/template_guide/
Copyright © 2016 Mirantis, Inc. All rights reserved
Example
heat_template_version: 2013-05-23
description: Simple template to deploy a single compute instance
resources:
my_instance:
type: OS::Nova::Server
properties:
key_name: my_key
image: cirros-0.3.0-i386-uec
flavor: m1.small
Copyright © 2016 Mirantis, Inc. All rights reserved
CloudInit
● An open-source package from Ubuntu that is the industry standard for bootstrapping cloud servers (initialization on first boot)
● Available on various Linux distributions such as Ubuntu Cloud Images and the official Ubuntu images available on EC2.
● Some of the things it configures are:● setting a default locale● setting hostname● resizing boot disk to that specified in boot flavor● adding ssh keys to user's .ssh/authorized_keys so they can log in● setting up ephemeral mount points
Copyright © 2016 Mirantis, Inc. All rights reserved
CloudInit: Example
resources:
database:
type: OS::Nova::Server
properties:
...
user_data: |
#!/bin/bash -v
yum -y install mariadb mariadb-server
systemctl start mariadb.service
Copyright © 2016 Mirantis, Inc. All rights reserved
Developing Template
● Uses “Unified Plone UNIX Installer”● Single node deployment● Allows to configure
● Size of the VM (RAM, Storage)● Setup password for Plone● Choose port on which Plone is going to be running
Copyright © 2016 Mirantis, Inc. All rights reserved
Template
http://q.melikyan.me/plone-via-heat-template
Copyright © 2016 Mirantis, Inc. All rights reserved
Links
● Project Links:● https://launchpad.net/heat ● https://wiki.openstack.org/wiki/Heat ● http://docs.openstack.org/developer/heat/
● MailingList:● [email protected] with [Heat] in the subject
Copyright © 2016 Mirantis, Inc. All rights reserved
Provision Plone via Murano
or ...going for a good stuff!
Copyright © 2016 Mirantis, Inc. All rights reserved
Murano
Introduce an application catalog to OpenStack, enabling application developers and cloud administrators to publish various cloud-ready applications in a browsable categorized
catalog.
Application Catalog for OpenStack
Copyright © 2016 Mirantis, Inc. All rights reserved
Benefits
Developer Tooling Application Catalog Cloud Orchestration
Onboarding Developers Onboarding Workloads Self-Service Deployment
IT Admins End Users
Build better cloud apps faster Deploy without assistance from ITManage what end users can deploy
Copyright © 2016 Mirantis, Inc. All rights reserved
Capabilities
● Supports both Windows and Linux● Complete application lifecycle management● Integration with configuration management tools
● Ansible, Chef & Puppet● Pluggable app definition languages
● Heat Templates● MuranoPL
Copyright © 2016 Mirantis, Inc. All rights reserved
Past
● Windows DataCenter as a Service● Fixed building blocks: AD, IIS, MS SQL, Exchange, Sharepoint● Complicated software composition● Only Windows software provisioning
Bringing Windows to OpenStack
Copyright © 2016 Mirantis, Inc. All rights reserved
LayersLayers
catalog
app interoperability layer
orchestration
configuration management
glare murano-api + dashboard
murano pl murano-engine
murano-engine
heat heat-translator third-partymurano pl
murano-agent
bash puppet powershellchef
Copyright © 2016 Mirantis, Inc. All rights reserved
Apps Interoperability
MySQL
host: instance
deploy()createDatabase()createUser()getConnectionS().destroy()
WordPress
webserver: httpddb: mysql
deploy().destroy()
WordPress: deploy() → … webserver.deploy() … db.createDatabase(<name>) db.createUser(<name>, <pass>) db.assignUser(<dname>, <uname>) ...
● Murano PL: imperative OOP DSL● Sandboxed, built on top of Python
● Everything is an object● Dependencies on interface● Decoupling
Apps form an ecosystem where complex environments can be constructed from
independent building blocks
Copyright © 2016 Mirantis, Inc. All rights reserved
YAQL
YAQL: Yet Another Query Language● Embeddable and extensible query language● Allows performing complex queries against arbitrary objects● Used by Murano, Mistral, Heat and Fuel pip install yaql
$ yaql
yaql> @load my_file.json
yaql> $.customers.groupBy($.sex)
yaql> $.customers.where($.orders.len() >= 1 or name = "John")
Find more about YAQL at https://github.com/openstack/yaql or play at http://yaqluator.com/
Copyright © 2016 Mirantis, Inc. All rights reserved
Developing Plone App
or … getting your hands dirty!
Copyright © 2016 Mirantis, Inc. All rights reserved
https://github.com/sergmelikyan/plone-training
Source Code
Copyright © 2016 Mirantis, Inc. All rights reserved
Hello World: Step 1
Adding manifest.yaml$ mkdir HelloWorld
$ vi HelloWorld/manifest.yaml
FullName: com.yourdomain.HelloWorld
Type: Application
Checkout sources for step #1 - git checkout step1
Copyright © 2016 Mirantis, Inc. All rights reserved
Hello World: Step 2
Adding HelloWorld.yaml$ mkdir Classes
$ vi Classes/HelloWorld.yaml
Name: com.yourdomain.HelloWorld
Extends: io.murano.Application
Methods:
deploy:
Body:
- $reporter: $this.find('io.murano.Environment').reporter
- $reporter.report($this, "Hello, World!")
Checkout sources for step #2 - git checkout step2
Copyright © 2016 Mirantis, Inc. All rights reserved
Hello World: Step 2
Updating manifest.yaml$ vi HelloWorld/manifest.yaml
...
Classes:
com.yourdomain.HelloWorld: HelloWorld.yaml
Checkout sources for step #2 - git checkout step2
Copyright © 2016 Mirantis, Inc. All rights reserved
HelloWorld: Pack & Upload
Pack & Upload$ zip -r hello_world.zip *$ murano package-import ./hello_world.zip
Copyright © 2016 Mirantis, Inc. All rights reserved
HelloWorld: Deploy
Deploy$ vi input.js
[{
"op": "add", "path": "/-",
"value": {
"?": {
"name": "Demo",
"type": "com.yourdomain.HelloWorld",
"id": "42"
}
}
}]http://q.melikyan.me/input-json
Copyright © 2016 Mirantis, Inc. All rights reserved
HelloWorld: Deploy
Deploy$ murano environment-create TestHello
$ murano environment-session-create <environment-id>
$ murano environment-apps-edit --session-id <session-id> <env-id> ./input.json
$ murano environment-deploy --session-id <session-id> <env-id>
$ murano environment-show <env-id>
Copyright © 2016 Mirantis, Inc. All rights reserved
Hello World: Step 3
Adding User Input$ vi Classes/HelloWorld.yaml
Properties:
username:
Usage: In
Contract: $.string().notNull()
Methods:
deploy:
Body:
- $reporter: $this.find('io.murano.Environment').reporter
- $reporter.report($this, "Hello, " + $this.username + "!")
Checkout sources for step #3 - git checkout step3
Copyright © 2016 Mirantis, Inc. All rights reserved
HelloWorld: re-upload the package
Pack & Reupload$ zip -r hello_world.zip *$ murano package-import ./hello_world.zip
Copyright © 2016 Mirantis, Inc. All rights reserved
HelloWorld: Deploy again
Deploy with an updated input$ vi input.js
[{
"op": "add", "path": "/-",
"value": {
"?": {
"name": "Demo",
"type": "com.yourdomain.HelloWorld",
"id": "42"
},
"username": "Alice"
}
}]
http://q.melikyan.me/input-json-updated
Copyright © 2016 Mirantis, Inc. All rights reserved
HelloWorld: Deploy again
Deploy$ murano environment-create TestHello2
$ murano environment-session-create <environment-id>
$ murano environment-apps-edit --session-id <session-id> <env-id> ./input.json
$ murano environment-deploy --session-id <session-id> <env-id>
$ murano environment-show <env-id>
Copyright © 2016 Mirantis, Inc. All rights reserved
Hello World: Step 4
Adding UI wizard$ mkdir UI$ vi UI/UI.yaml
Application:
?:
type: com.yourdomain.HelloWorld
username: Alice
Checkout sources for step #4 - git checkout step4
Copyright © 2016 Mirantis, Inc. All rights reserved
Hello World: Step 4 (continued)
Application:
?:
type: com.yourdomain.HelloWorld
username: $.step1.username
Forms:
- step1:
fields:
- name: username
type: string
description: Username of the user to say 'hello' to
required: true
Checkout sources for step #4 - git checkout step4
Copyright © 2016 Mirantis, Inc. All rights reserved
Plone: Goals
● Provision a virtual machine in OpenStack (VM);● Configure VMs network connectivity and security;● Download a distribution of Plone from Internet to the
virtual machine;● Install the distribution and configure some of its
parameters with user input.
Copyright © 2016 Mirantis, Inc. All rights reserved
Plone: Preparation
Preparation$ mkdir PloneApp$ vi PloneApp/manifest.yaml
FullName: com.yourdomain.Plone
Name: Plone CMS
Description: Simple Plone Deployment
Type: Application
Author: John Doe
Checkout sources for step #5 - git checkout step5
Copyright © 2016 Mirantis, Inc. All rights reserved
Plone: Preparation (continued)
Add Plone.yaml$ mkdir PloneApp/Classes$ vi PloneApp/Classes/Plone.yaml
Namespaces:
=: com.yourdomain
std: io.murano
apps: io.murano.applications
Name: Plone
Extends: apps:SingleServerApplication
Checkout sources for step #6 - git checkout step6
Copyright © 2016 Mirantis, Inc. All rights reserved
Plone: Preparation (ends)
Checkout sources for step #6 - git checkout step6
Updating manifest.yaml$ vi PloneApp/manifest.yaml
...
Classes:
com.yourdomain.Plone: Plone.yaml
Require:
io.murano.applications:
Copyright © 2016 Mirantis, Inc. All rights reserved
Plone: Properties
Adding properties$ vi PloneApp/Classes/Plone.yaml
Properties:
deploymentPath:
Contract: $.string()
Default: '/opt/plone'
adminPassword:
Contract: $.string().notNull()
listeningPort:
Contract: $.int()
Default: 8080
Copyright © 2016 Mirantis, Inc. All rights reserved
Plone: Properties
Adding UI.yaml server:
?:
type: io.murano.resources.LinuxMuranoInstance
name: $.instanceConfiguration.hostname
image: $.instanceConfiguration.image
flavor: $.instanceConfiguration.flavor
assignFloatingIp: $.instanceConfiguration.assignFloatingIp
Checkout sources for step #7 - git checkout step7
Copyright © 2016 Mirantis, Inc. All rights reserved
Plone: Configuration Management
● Use “Unified Plone UNIX Installer” ● Use almost same script as we did for Heat template
http://q.melikyan.me/install-plone-sh
Copyright © 2016 Mirantis, Inc. All rights reserved
Plone: Configuration
Checkout sources for step #8 - git checkout step8
Add install-plone.sh$ mkdir PloneApp/Resources
<download and put install-plone.sh in the directory above>
Copyright © 2016 Mirantis, Inc. All rights reserved
Plone: Deploy
Checkout sources for step #9 - git checkout step9
Adding logic$ vi PloneApp/Classes/Plone.yaml
...
Methods:
onInstallServer:
...
Body:
- $file: sys:Resources.string('install-plone.sh').replace({"$1" => $this.deploymentPath, "$2" => $this.adminPassword, "$3" => $this.listeningPort})
- conf:Linux.runCommand($server.agent, $file)
Copyright © 2016 Mirantis, Inc. All rights reserved
Bonus: Logo & Security Rules
● Please checkout step10 and take a look● Try to analyze what we are doing and why● Explain
Checkout sources for step #10 - git checkout step10
Copyright © 2016 Mirantis, Inc. All rights reserved
Ready to use applications!
DBaaS Containers
PaaS
CI/CD
CMS Web/ App Servers
Copyright © 2016 Mirantis, Inc. All rights reserved
Screencasts
● Openstack Application Catalog. Quick introduction● HA & Autoscaling for Applications● Docker Integration● Support for Heat based application● Composing Murano Application Package
Please, find more screencasts published on wiki: https://wiki.openstack.org/wiki/Murano/Screencasts
Copyright © 2016 Mirantis, Inc. All rights reserved
Links
● Project Links:● https://launchpad.net/murano ● https://wiki.openstack.org/wiki/Murano ● http://docs.openstack.org/developer/murano/
● MailingList:● [email protected] with [Murano] in the subject
Weekly IRC meeting is held on Tuesdays at 17:00 UTC on #openstack-meeting-alt at Freenode.
Copyright © 2016 Mirantis, Inc. All rights reserved
Credits
● Alexander Tivelkov & Evgeniy Mashkin● for amazing articles and developing Plone app for Murano
● David Flanders & Philip Bauer● for opportunity being here and present this training