guide - migrating from heroku to aws using cloudformation

Download Guide - Migrating from Heroku to AWS using CloudFormation

Post on 13-Dec-2014




2 download

Embed Size (px)


Step by step guide to migrating from Heroku to Amazon AWS using AWS CloudFormation. Presented at the Australian AWS User Group in Melbourne at the October Meetup.


  • 1. Rob Linton Jasondb a cloud based,CTO & Founder scalable NoSQL database for social and mobile applications. Phone:[AUS] + 61 418 510 155 Email: 2Migrating from Heroku toAmazon AWS using CloudFormationIn guide number one we looked at how we migrated Amazon EBS backed AMIsbetween AWS Regions using the tools that already exist on a standard Unix AMI.(Available on at this guide we look at migrating a Ruby on Rails application to Amazon AWS.This guide, while focussed on Ruby on Rails, can be used as a generic introduction onsetting up any application in the Amazon AWS stack.This guide was presented at the Australian AWS Users Group in Melbourne on the26thOctober, 2011.A brief summary of HerokuHeroku is commonly referred to as a Platform as a Service (PaaS) provider, in that theyoffer more than just infrastructure services. Heroku offer a turnkey solution for hosting,originally, Ruby on Rails (RoR) applications, although they now offer other languagessuch as Node.js, Clojure, Java, Python and Scala.Herokus success lies in the fact that as a Ruby on Rails developer, you could focus onyour application and leave everything else such as the hosting and scaling up to someoneelse. Having done a substantial amount of development over the years (although not inRoR), I can attest to the attraction of any services that eases the deployment of yourapplication.

2. Why migrate from Heroku?There may be a number of reasons why you may want to migrate from Heroku. WhileHeroku is great in the way it takes the load of managing the deployment and scaling awayfrom you, there are always compromises.In this case Heroku, while reducing the complexity, also reduces the flexibility.There are just some things that you cannot do in Heroku. Also as your application startsto gain momentum, it is usually cheaper to host your application on AWS as the numberof users increase.The usual steps for deploying to HerokuThe usual steps for deploying to Heroku are as follows:Install the required software1. Install Ruby on Rails2. Install Git3. Create an Heroku account4. Install the Heroku Gem$ gem install herokuCreate your hello world application:$ rails hello$ cd hello$ rails generate controller helloOpen the file config/routes.rb. Almost at the bottom (line #57) is this line:# match :controller(/:action(/:id(.:format)))Remove the # in front so the line looks like:match :controller(/:action(/:id(.:format)))Create a file named index.html.erb in app/views/hello containing the text Hello world.Start the server.2 3. $ rails serverUse Git to manage your project:$ cd PATH/TO/MY_APP$ gitinit Initialized empty Git repository in .git/$ git add .$ git commit -m "new app"Create a ssh key pair:$ cd ~/.ssh$ mkdirkey_backup$ cpid_rsa* key_backup$ rmid_rsa*$ ssh-keygen -t rsa -C ""Open the file with a text editor and cut & paste into your account settings onthe Heroku web site.Configure Git$ gitconfig --global "FirstnameLastname"$ gitconfig --global ""Create your Heroku application$ heroku createUpload your application to Heroku$ git push heroku master3 4. Set up your application database$ heroku rake db:migrateSetting up the Amazon AWS EnvironmentBefore we begin, lets make sure we have the pre-requisites in place.Make sure you have created an AWS account and have enabled access to the followingproducts:1. Elastic Compute Cloud (EC2)2. Relational Database Service (RDS)3. Elastic Load Balancing (ELB)4. CloudWatch5. CloudFormation6. Simple Notification Service (SNS)Also before you start make sure you have created a key/pair and download theprivate key to a safe place on your local computer.Architecture overviewHere is asimple overview of the architecture for the rails application on AWS.4 5. Overview architecture of the rails applicationAs you can see from the diagram above, the layout is relatively simple.The load balancer will have two rails servers, which in turn will use aRelationalDatabase Instance (RDS) to store its data.Step by stepNow lets put the pieces in place.To create the application stack we are going to use AWS CloudFormation.CloudFormation is a great way to speed up the creation of commonly used applicationstacks such as Ruby on Rails applications, Wordpress sites and even Drupal CMS sites.5 6. Create a new stackClick on the Create New Stack button, this will pop up the Create Stack dialog. Create stack dialogGive your application a name (Hello World) and select the sample template of Rails HelloWorld Application (SSH enabled).At the bottom of the dialog, expand the Show Advanced Options. This exposes someextra options that you can select such as setting a time out on the creation or adding aSimple Notification Service (SNS) topic.6 7. For this example we will be adding a SNS topic that notifications can be sent to.This next dialog looks pretty daunting, but is effect just asking for some standard bits ofinformation such as user names and passwords.Create stack optionsI recommend that you change the rails database password, the rails database user and theoperator email address.Make sure you scroll down and get the last option to enter a key pair name, asthis is hidden at the bottom of the dialog.Enter the key pair name you created earlier.Once you have entered these pieces of information click on Continue.The final dialog shows a summary of the parameters that will be used to create the Rubyon Rails stack.7 8. Create stack summary screenThis is the last point where you can cancel the create stack. By clicking on the CreateStack button, amazon will go ahead and create an Auto Scaled sample Ruby application,backed by an Amazon RDS instance.Note the Notification section at the bottom, at this point the SNS notification willhave already have been created and you should have received an email similar tothe one below.8 9. SNS notificationClicking on the link should display the following message. ConfirmedDuring the Ruby on Rails stack creation, the following dialog will be displayed. 9 10. Creating stackDuring the creation process you will receive a number of email notifications via the SNSservice. Email notificationsAfter your rails stack has been completed, the status will change toCREATE_COMPLETE.10 11. Resources createdSelecting the Resources tab will show you all of the resources that the CloudFormationstack created for you. Because it is important that you understand what actuallyhappened, Ill take you through each of the resources created.Here is an overview diagram of what was created above: 11 12. CloudFormation stack after creationEC2SecurityGroupThis is the AWS security Group used to restrict access to your rails hosts. For AWS,security groups work on individual servers, so for example, A single EC2 instancecreated as part of the CloudFormation stack can only be accessed via port 8888, or port22 (ssh), even if the source of the access was a second EC2 instance in the same group.ElasticLoadBalancerThe ELB accepts traffic in on port 80 and redirects it out on port 8888 to each of therunning rails servers. Note how it changes the port number to ensure that each of the railsservers are not accessible on port 80.12 13. Note however, that if you knew the direct address of one of the EC2 instances,this does not prevent you from connecting to the instance directly by using thefollowing syntax in your browser. database security group restricts access to the RDS instance by only allowing EC2instances that are members of a specific EC2 security group to connect. In this case onlymembers of the Rails security group can connect to RDS.SampleDatabaseThis is the RDS MySQL instance that was created for you. By default a db.m1.smallinstance was created.Also by default a Multi AZ RDS deployment was done. Selecting a Multi AZdeployment allows a hidden replica to be created in a different availability zone, allowingfor fail-over of your RDS instance if one availability zone goes down.AlarmTopicThis is the Simple Notification Service (SNS) that was created as part of the stackcreation. As part of the creation process I received 22 notifications in the space of around30 minutes, so I would recommend that you create a mail rule to move these off to theirown folder in your mail program.TooManyUnhealthHostsAlarmYou can see the details for this alarm by clicking on it in the CloudWatch tab.13 14. Alarm detailsAs you can see from the highlighted section above, this alarm will trigger if the numberof unhealthy hosts > 0 for 1 minute.The action for this alarm is to send a notification to the SNS service that we set up earlier.RequestLatencyAlarmHighThis is the second alarm. This alarm will trigger if the latency is greater than 1 secondfrom the ELB to a Rails instance for longer than 1 minute. If this alarm triggers a SNSnotification will be sent.LaunchConfigThis is a component of the Auto Scaling. The launch configuration defines the type ofinstances that will be started as well as what AMI to launch. This has been taken care offor you so you will not need to edit this unless you would like to upgrade to a largerinstance type or if you would like to use a Golden Image AMI(We see later on how touse Golden Images)WebServerGroupThe Web Server Group defines which availability zones to start any new instances in aswell as the maximum and minimum instance numbers. The default configuration for the14 15. web server group that was set up is a minimum of 1 with a maximum of 3. Unfortunatelythis is not visible in the Web GUI, other than by reading the template. The followingsection of the template shows the WebServerGroup details. "WebServerGroup": {"Properties": {"LoadBalancerNames": [{"Ref": "ElasticLoadBalancer"}],"M