opencms days 2013 - opencms cloud extensions
DESCRIPTION
In this session Stephan Hartmann presents Metamesh's OpenCms Cloud eXtensions (OCCX) implementation, which is the foundation to provide a cloud-ready, distributable and cluster-ready version of OpenCms that implements an integration with Open Data Grid Platforms. Arash Kaffamanesh will present OCCX in action running on Eucalyptus Private Cloud and Amazon EC2 and shows you the first cloudified version of OpenCms with Auto-Scaling and Multi-Workplace support for your OpenCms SaaS initiatives.TRANSCRIPT
Metamesh’s OpenCms Cloud eXtensions
The OpenCms Elastic Cloud Cluster
Stephan Hartmann
e-mail: [email protected] http://www.metamesh.de
Arash Kaffamanesh
e-mail: [email protected] http://www.cloudssky.com
A bit about us Stephan Hartmann ● Senior Java Architect and Developer for 15 years ● Working with OpenCms since 2000 ● Working on large scale Java Enterprise Portals
Arash Kaffamanesh ● Open Cloud Solution Architect ● Working with OpenCms since 2002 (Kaitain Release) ● Building the first OpenCms CloudSite Offering
Agenda ● What is “Cloud” ● Why Cloud ● Requirements of Cloud Readiness of a Java WebApp ● Problems ● Solutions ● About OCCX ● OCCX and Data Grid Platforms ● OpenCms Clustering on Hybrid Clouds with OCCX ● OpenCms Cluster on Eucalyptus, AWS and ProfitBrikcs ● OpenCms Quick- and Auto- Scaling with OCCX ● Classic- and Elastic- Load Balancing
What is “Cloud” ● Fuzzy Word ● Virtual Server -> Cloud Server? ● SaaS really new? (ASP in 90ies) ● Cloud == Internet? ● Red Cloud (SaaS 2.0) - Blue Cloud (PaaS, SaaS 3.0) ● The PaaS Marketing Lie
Why Cloud • High Availability • Fault-Tolerance • Performance • Scalability, Auto-Scaling ("Slashdot effect"), true
Horizontal Scaling • Deployment Automation - Fast Deployments (HA) • Staging • Administration, Maintenance • Development, DevOps • Cost saving, increase agility
Requirements for Cloud-Readiness of a Java WebApp • What the Servlet Spec says (<distributable/>, not
enough) • Container Support for Distribution of WebApp • Container Support for Distribution of Http Sessions • NO! Distribution of Application Scope ⇒ No support for
Shared Memory • Objects have to be Serializable • Shared Persistent Storage (DB) • Transactions
Problems ● Multiple Processes (Java VMs) have no shared Memory
for e.g. maintaining state of transient objects ● Instance, Class and Application Scope Variables ● Singleton Objects only per VM ● Comparison (==, != vs. equals()) ● Local Caches ● Serialization
Problems in the Case of OpenCms ● Flex-Cache (Presentation Layer) ● Core-Cache (Persistence Layer) ● Static Export ● Events ● Serialization E.g. CmsGroup, CmsUser, CmsProject ● Map<String, Object> E.g. CmsPrincipal.m_additionalInfo ● Local Objects E.g. CmsBroadcast ● Comparison E.g. CmsLockType, CmsResourceState ● Search Indexes ● BUG: Resources have Cache, but it is not used, File Content no Cache at
all
Example: Comparison of Objects
JSP
Request Phases of OpenCms
Flex Cache
Request Response
Core Cache
DB
Solution 1 (no Code Changes in Core) ● Only one Workplace Server (Master) ● Distribute Events from Workplace Server to Slaves ● Slaves Flush all Caches on DB-Change Events ● Static Export On-Demand ● Rebuild Search Indexes Regularly or on Startup Drawbacks ● No Workplace if Master fails ● No shared Cache between Nodes
Solution 2 (with Code Changes in Core)
● Flex-Cache: no Solution -> Throw it away or use Solution 1 ● Core-Cache: Make all cached Objects serializable and use distributed
Cache ● Static Export: no Problem with On-Demand ● Events: Distribute Events to all Cluster Nodes. Publish-Event: flush Flex
Cache, don't flush Core Cache ● Comparison: Use equals or compare Primitives ● Search Indexes: Rebuild regularly or on Startup or use external Solr Server
Advantages ● Data Grid to enlarge Heap Space
Fast Deployments ● GIT Repos that holds the
complete WebApp ● Developer checks in to
trunk / branch ● Branch for Test and Production ● Update on each node and restart
if necessary (Puppet or Chef)
How it works ● RFS Driver ● Mount RFS files and folders into
VFS ● Serve JSPs directly from RFS ● Reads all Module data as well ● NO import necessary!!!
VCS Rep. Developer
Test
Production
Metamesh’s OpenCms Cloud eXtensions
The OpenCms Elastic Cloud Cluster
Part II: Demo
Stephan Hartmann e-mail: [email protected]
http://www.metamesh.de
Arash Kaffamanesh e-mail: [email protected] http://www.cloudssky.com
Date: 2013, October 1st
Recorded Video:
http://cloudssky.com/en/events/opencms-days-2013-opencms-cloud-extensions.html
DEMO Multi Workplace Support • Single Bare-Metal Cluster
o Bare-Metal Cloud • Eucalyptus Cluster
o Quick-Scale o Auto-Scale
• Get Unlimited with Amazon EC2 / AWS with Auto-Scaling • Run on ProfitBricks (The German IaaS Company)
o Apply awesome Vertical Scaling o Manual horizontal scaling with free load balancers o Half the Price o Twice the Speed (compared to Amazon)
About OpenCms Cloud eXtensions ● Developed by Stephan Hartmann, CEO Metamesh ● Provides Multi Workplace Support for OpenCms ● Provides a Cloud Ready Version of OpenCms ● Supports elasticity in OpenCms Clusters ● Supports building OpenCms based hybrid cloud solutions ● Empowers Clouds Sky’s OpenCms CloudSite offering ● Is going to engage and innovate the next generation of OpenCms
OCCX Multi Workplace Support
New York
Cologne
Tokyo
Mexico Singapour
● Multi- Workplace support for international companies ○ support low latency in different geographic regions for:
■ authoring and user productivity ■ high availability
● Comes with Zero Configuration ○ all instances have the same configuration ○ no master/slave design - all nodes are equal ○ fault tolerance - removing one node won't hurt others ○ every node can serve the OpenCms Workplace at ANY Time, ANY Where
with sticky sessions
Single Bare-Metal Cluster
OCCX Data Grid Cache Platform
Bare-Metal Cluster
DB Matser DB Slave
OCCX1 OCCX2
HAProxy Load Balancer
Guests
DB Cluster
Infinispan
Eucalyptus ClusterQuick Scale
OCCX Data Grid Cache Platform
Eucalyptus Cluster
DB Matser DB Slave
OCCX1-VM OCCX2-VM
HAProxy Load Balancer
Guests
DB Cluster
OCCX3-VM
Quick Scale
Eucalyptus ClusterAuto-Scale
OCCX Data Grid Cache Platform
Eucalyptus Cluster
DB Matser DB Slave
OCCX1-VM OCCX2-VM
Eucalyptus Elastic Load Balancer
Guests
DB Cluster
OCCX3-VM
Auto Scale
EC2 Elastic Load Balancer
Get Unlimited with EC2/AWS
OCCX Data Grid Cache Platform
Eucalyptus Cluster Bare-Metal Cluster Amazon EC2 Cluster
DB Matser DB Slave
OCCX1-VM OCCX2-VM OCCX1 OCCX2 OCCX1-EC2 OCCX2-EC2
HAProxy Load Balancer
Guests
DB Cluster
Eucalyptus OCCX Dashboard (before auto-scaling)
Eucalyptus OCCX Dashboard (Quick scale action I)
Eucalyptus OCCX Dashboard (Quick scale action II)
Eucalyptus OCCX Dashboard (after Quick scale action)
New OCCX Instance
New OCCX Instance running in less than 30 seconds
Eucalyptus CloudWatch Alarm for Auto-Scaling
Create an OCCX Auto-Scaling Group on Amazon EC2 Create the Launch Congif: sh-3.2# as-create-launch-config OCCXLC --key occx --image-id ami-xxxxxxx --instance-type m1.xlarge --group sg-33134d4f --region eu-west-1 OK-Created launch config sh-3.2# as-create-auto-scaling-group OCCX_ScalingGroup --launch-configuration OCCXLC --availability-zones eu-west-1a --region eu-west-1 --min-size 2 --max-size 5 OK-Created AutoScalingGroup
Verify OCCX Cluster Nodes
ubuntu@occx1-ec2:~$ grep "cluster" /var/lib/tomcat6/webapps/opencms/WEB-INF/logs/opencms.log ... 29 Sep 2013 09:50:40,628 [metamesh.cloud.corecache.MemoryMonitor: 323] CLUSTER - cluster members: [csky07-64460, csky08-15213, occx2-ec2-54095, occx1-ec2-32746]
Deploy with ProfitBricks DCD
OCCX’s Status, Challenges left
➢ Runs on Clouds Sky’s OpenCms 8.5 CloudSite offering on ProfitBricks ➢ Use DevOps Tools such as Puppet / Chef for deployment ➢ Needs still some performance tuning ➢ Image Cache / Sync shall be improved ➢ Hybrid Solutions needs security and stability improvements ➢ Improve auto-discovery on EC2 ➢ Get Elastic Load Balancing and Auto-Scaling to work as it should ➢ Stress Testing still not performed ➢ Licensing issues in discussion ➢ Is ready for new pilots ➢ Upgrade OCCX for OpenCms 9.0 ➢ Implement 100% Availability
Thanksgivings to:
● Metamesh ○ Stephan Hartmann
● Alkacon Software ○ Martina Alger ○ Alexander Kandzior ○ Rüdiger Kurz ○ Michael Emmerich ○ Andreas Zahner ○ Tobias Herrmann ○ Georg Westenberger ○ and other members of Alkacon Software’s team
● ProfitBricks ○ Chris Meyer ○ Christian Studier
● Eucalyptus Systems
Thank You for Your attention!