migrating it to the cloud - zenoss in amazon web services

© 2016 All Rights ReservedCONFIDENTIAL #GALAXZ16 @nickclarkturner

#GALAXZ16

Migrating IT to the Cloud

Nick Turner @nickclarkturner

Zenoss in Amazon Web Services


Topics we’ll be covering…

• Benefits of the Cloud• Benefits, Feasibility, and Objections

• Zenoss in the Cloud• Internal use

• Zenoss monitoring Zenoss

• Deploying Zenoss in AWS

• Challenges of Migrating to the Cloud• Overall and Zenoss-Specific

• Cloud Governance and Cost Controls

• Recommended Reading


Benefits of the Cloud


Benefits of the Cloud• Amazon Elastic Compute Cloud was introduced in August 2006

• Does it make sense to move X to the cloud?• Does your compute load burst? How scalable is your infrastructure?

• Elastic Infrastructure (Availability Zones, Regions, Virtual Private Clouds, Route53)

• Storage (S3, EBS, Snapshots, AMI)

• How critical is uptime? Do you have SPOFs? How redundant is my infrastructure?

• Available Zones, S3 Cross-Region replication, Elastic Load Balancing

• How truly international is your co-location or managed datacenter partner?

• Does your datacenter meet regulatory/compliance needs?

• SOC, PCI, HIPAA, ISO, etc…


Benefits of the Cloud• But I don’t have the expertise…

• A lot of transferable concepts from Physical to Cloud or Cloud to Cloud

• Managed AWS Services (Rackspace, Datapipe, Connectria, etc...)

• What are the knowledge retention realities of your current infrastructure?

• Documentation, cross training, and wikis vs. Tribal

• But CapEx vs. OpEx... Short-term vs. Long-term• What are the financial needs of the business?

• Available capital? EBITDA obsession? Time value of money?

• Try and level the financial playing ground with a discounted cash flow analysis or other tools

• The realities of your CapEx investment reaching EOL or maximum capacity

• Will needed capital be there in 5 years when your compute ot storage needs it?


Zenoss in the Cloud


Zenoss in the Cloud• We followed the department by department lift and shift model

• Different Accounts with Consolidated Billing

• Automation isn’t easy, but it is your friend.

• SaltStack, Ansible, Chef, Cloud Formation, YAML, Boto Python

• Architected each solution around use-case:

• QA/Engineering

• Short term use, frequent version changes, build and destroy, contractor access

• Training

• Short term use and destroy, components differ based on curriculum, multi-region, externally accessible

• Demo

• Short term and random simultaneous usage, need to showcase all product features with historical data

• ZaaS

• Long term use, highly available, geographically redundant data retention, minimize SPOFs where arch allows


Zenoss in the Cloud• Zenoss-as-a-Service (Zenoss Monitoring Zenoss)

• Visibility, Uniformity, Quick Recovery

• Critical ZaaS ZenPacks

• AWS ZenPack

• Linux Monitor ZenPack

• AWS RDS Monitoring

• ControlCenter ZenPack

• RabbitMQ ZenPack

• PagerDuty ZenPack

• Slack Notifications

• Web-Based Synthetic Transactions

http://wiki.zenoss.org/ZenPack:Amazon_Web_Services

http://wiki.zenoss.org/ZenPack:Linux_Monitor

http://wiki.zenoss.org/ZenPack:AWS_RDS_Monitoring

http://wiki.zenoss.org/ZenPack:ControlCenter

http://wiki.zenoss.org/ZenPack:RabbitMQ

http://wiki.zenoss.org/ZenPack:PagerDuty_Official

http://wiki.zenoss.org/ZenPack:PagerDuty_Official

http://wiki.zenoss.org/ZenPack:Slack_Notifications

http://wiki.zenoss.org/ZenPack:Web-Based_Synthetic_Transactions

http://wiki.zenoss.org/ZenPack:Web-Based_Synthetic_Transactions


Zenoss in the Cloud

• Deploying Zenoss in AWS• Requirements have changed from 4x to 5x, but on par costs of running Zenoss Service Dynamics

(Resource Manager, Impact, and Analytics) are similar.

• For 4x ZSD depending on the number of Managed Resources would vary from:

• RM = m4.xlarge -> m4.2xlarge -> m4.4xlarge, Impact = c4.2xlarge, Analytics = m4.2xlarge with RDS on db.m4.large all backed with EBS volumes.

• For 5x the perspective shifts to resource pools

• CC Resource Pool = m4.xlarge, Host Pool = m4.2xlarge xN, Analytics = m4.2xlarge, all with EBS

• Nice instance comparative resource - http://www.ec2instances.info

• Zenoss working on deploying an AMI to the Marketplace

Image courtesy of https://www.botmetric.com/blog/how-to-choose-aws-instance-family-for-your-business-workloads/

http://www.ec2instances.info/

http://www.ec2instances.info/

https://www.botmetric.com/blog/how-to-choose-aws-instance-family-for-your-business-workloads/



Challenges Migrating to the Cloud


Challenges of Migrating to the Cloud

• Building out an infrastructure• Cloud Service Accounts/Architecture

• Too small (Account per service/customer – Automation/Management Limitations)

• One Size Fits All (Security concerns, Account Limitations)

• Seek guidance from vendor.

• Migration Strategy

• All at once or re-architecture

• Complexity may cause analysis paralysis

• Lift and Shift sensible workloads

• Re-inventing the wheel? Are processes similar are efforts duplicated?


Challenges of Migrating to the Cloud

• Zenoss in AWS Considerations

• Networking (How Will Zenoss communicate with collectors or what it is monitoring?)

• Connecting VPCs, L2TPw/IPSEC VPN tunnels, or OpenVPN tunnels

• In ZaaS we define a pool per remote collector and utilize OpenVPN for connectivity

• Are your services communicating internally or externally?

• The route traffic takes when talking with other systems can impact security and performance

• Should you offload processes like mail, database (RDS/HBASE), log aggregation, etc…

• Do you need multiple environments ”Test” and “Prod”?

• Plan for growth!

• Zenoss in Azure?

• On “paper” everything looked good, but the distributed infrastructure was not as seamless as stated and resulted in the application crashing.


Cloud Governance and Cost Controls



• Considerations for Cost Control Tool Selection

• Purchasing On-Demand versus Reservations

• Complexity (No-Upfront, Partial Upfront, Full-Upfront)

• Dynamic Environment (m1 paravirtual (PV) today m4 hardware virtual machine (HVM) tomorrow)

• Multiple Accounts / Different AZs

• What is the best buy for my budget? What if I choose poorly?

• Are they assisting you or is it up to you?

• Tagging is your friend

• The more you tag environments upon build for Environment, Department, Customer, etc… the easier it is to manage costs and the more governance tools can do for you.

• Helps you determine what to shut off and what to keep on. What can be deleted and what should be retained.



• After evaluating others went with Cloud Health Tech

• Reserved Instance Recommendations

• Cost Tracking and Projections

• Cleaning out the noise (RI Prepay, Unused RI)

• Detailed Historical Service Data

• Multiple Accounts, Different Perspectives (Tagging for Environment, Department, Customer)

• Automated Reserved Instances/Health Checks

• Automated Modifications (Dynamic AZ RI changes)

• Unattached volume clean up

• Stopping instances

• Securing Environment (Open Ports)

• Instance Utilization Recommendations

migrating it to the cloud - zenoss in amazon web services

Software