running on amazon ec2
DESCRIPTION
Platforms like Amazon EC2 promise scalable and redundant systems for a couple of pennies. As soon as you start to build complex systems or migrate existing apps there are many knobs to set. This talk will explain how you can create and deploy reliable and redundant applications to EC2 and will point out all the little things you need to know, like how to automatically provision new servers with tools like Chef.Presented by Jonathan Weiss at PHP UK Conference 2011 in London.TRANSCRIPT
![Page 1: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/1.jpg)
Running on Amazon EC2
Jonathan Weiss
![Page 2: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/2.jpg)
Scalarium
Amazon EC2 Cluster Management
Auto-Config
Self-Healing
Auto-Scaling
One-click-deployment
www.scalarium.com
2
![Page 3: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/3.jpg)
3
![Page 4: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/4.jpg)
Amazon Elastic Compute Cloud
Virtual Server via API call
Pay per hour
Different instance types in different regions
4
![Page 5: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/5.jpg)
EC2 Features
Spot instances
Elastic IP
Elastic Load Balancing
CloudWatch
Auto Scaling
Virtual Private Cloud
Elastic Block Storage
5
![Page 6: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/6.jpg)
6
Running on EC2 Setting up an app
![Page 7: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/7.jpg)
Running on EC2
Steps to complete
Choose region and availability zone
Define architecture
Choose instance types and AMI
Boot & configure instances
Deploy application
…
Profit
7
![Page 8: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/8.jpg)
Region and Availability Zone
Regions Individual EC2 installations
Multiple datacenters a.k.a availability zones
Small differences in pricing
US-east usually gets features first
Availability Zone
Datacenter of instances & volumes
Differently named for different accounts
Traffic within the same zone is free
Different Instance type availability
8
![Page 9: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/9.jpg)
Existing Availability Zones
US-EAST-1
us-east-1a
us-east-1b
us-east-1c
us-east-1d
9
US-WEST-1
us-west-1a
us-west-1b
us-west-1c
EU-WEST-1
eu-west-1a
eu-west-1b
AP-SOUTHEAST-1
ap-southeast-1a
ap-southeast-1b
![Page 10: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/10.jpg)
Single AZ Deployment
Most common and simple setup
Not very different from typical setup
10
![Page 11: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/11.jpg)
Multi AZ Deployment
11
![Page 12: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/12.jpg)
Architecture
12
![Page 13: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/13.jpg)
Architecture
13
![Page 14: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/14.jpg)
Architecture
14
![Page 15: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/15.jpg)
Elastic IP
Static, public IP in contrast to instance IP
Associate to any instance (~15s)
Paying when not in use
Use for DNS of load balancer
15
![Page 16: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/16.jpg)
Elastic Block Store EBS
Network based block device: 1GB – 1TB
Persistent & redundant
Can be used by one instance at a time
Incremental Snapshots
RAID for better performance
16
![Page 17: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/17.jpg)
Elastic Load Balancer ELB
Managed Load Balancer
Auto-Scales via Round Robin DNS
Can start/stop instances
Slower and not as flexible as instance + HAProxy
The only way to scale past 1 Gbit/s!
17
![Page 18: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/18.jpg)
Relational Data Store RDS
Managed MySQL instance
Choose instance size,
pay ~ 30% premium
Hot-Failover and read replication
available
Define a 4h maintenance window
Better performance than single EBS-backed instance
18
![Page 19: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/19.jpg)
Instance Types
19
RAM Cores x ECU Arch Hourly Cost (US-East)
Monthly Cost (US-East)
Micro
Tiny 613 MB Burst 1 x 2 32bit/64bit $0.02 $15
Standard
Small 1.7 GB 1 x 1 32bit $0.085 $62
Large 7.5 GB 2 x 2 64bit $0.34 $248
Extra Large 15 GB 4 x 2 64bit $0.68 $496
High-Memory
Extra Large 17.1 GB 2 x 3.25 64bit $0.50 $366
Double Extra Large 34.2 GB 4 x 3.25 64bit $1.00 $732
Quadruple Extra Large 68.4 GB 8 x 3.25 64bit $2.00 $1464
High-CPU
Medium 1.7 GB 2 x 2.5 32bit $0.17 $124
Extra Large 7 GB 8 x 2.5 64bit $0.68 $496
1 ECU = 1.0 – 1.2 GHz Opteron/Xeon from 2007
![Page 20: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/20.jpg)
Recommendation
Start with Small for load balancer
Medium for App server
Large for DB server
Tiny for monitoring and utility
Grow to
XL-HighCPU for load balancer
XL-HighCPU for App server
XL-HighMemory for DB server
20
![Page 21: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/21.jpg)
Reserved Instances
Pay amount upfront and you get:
Hourly price ~ 33%
Guaranteed availability
24/7 instance cost: ~ 55%
Recommended!
21
![Page 22: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/22.jpg)
Running on EC2
Steps to complete
Choose region and availability zone
Define architecture
Choose instance types
Boot & configure instances
Deploy application
…
Profit
22
![Page 23: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/23.jpg)
23
Instance Configuration From blank Linux to your PHP app server
![Page 24: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/24.jpg)
Instance Bootstrap
24
2 Approaches
![Page 25: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/25.jpg)
Instance Bootstrap
Images
One image for every server type
All software pre-installed
Few moving parts
Fast boot
25
2 Approaches
![Page 26: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/26.jpg)
Instance Bootstrap
Images
One image for every server type
All software pre-installed
Few moving parts
Fast boot
26
Scripted
One image to rule them all
Smallest common denominator
Acquires configuration on start
Installs & configures during boot process to fit role
All parts moving
Slower boot
2 Approaches
![Page 27: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/27.jpg)
Instance Bootstrap
Images
One image for every server type
All software pre-installed
Few moving parts
Fast boot
27
Scripted
One image to rule them all
Smallest common denominator
Acquires configuration on start
Installs & configures during boot process to fit role
All parts moving
Slow boot
2 Approaches
![Page 28: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/28.jpg)
Chef
Open Source Provisioning Tool
Configures your servers
Cookbooks & recipes
Ruby DSL
28
![Page 29: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/29.jpg)
Chef
29
![Page 30: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/30.jpg)
Chef Solo
30
![Page 31: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/31.jpg)
MemcachedSolo
31
![Page 32: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/32.jpg)
Chef Solo Run
32
Configuration JSON
Run Chef Solo
![Page 33: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/33.jpg)
Deploying
Deploy resource Compatible with Capistrano layout
Supports Capistrano hooks via deploy/hook_name.rb
Async!
33
![Page 34: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/34.jpg)
Chef Experiences
Great so far, but
Not idempotent
Unintuitive 2 phases
Documentation could be better
Example Cookbooks often so so
34
![Page 35: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/35.jpg)
Running on EC2
Steps to complete
Choose region and availability zone
Define architecture
Choose instance types
Boot & configure instances
Deploy application
…
Profit
35
![Page 36: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/36.jpg)
The Project
Browser game hosted on EC2
1.2 million Daily Active Users (DAU)
~ 130 server
~ 120.000 requests per minute on average
![Page 37: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/37.jpg)
Flexibility and Scalability
![Page 38: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/38.jpg)
Flexibility and Scalability
![Page 39: Running on Amazon EC2](https://reader033.vdocuments.mx/reader033/viewer/2022051210/54b75fb04a7959f71f8b4646/html5/thumbnails/39.jpg)
© Peritor GmbH - Alle Rechte vorbehalten
Peritor GmbH Blücherstr. 22, Hof III Aufgang 6 10961 Berlin
Tel.: +49 (0)30 69 20 09 84 0 Fax: +49 (0)30 69 20 09 84 9
Internet: www.peritor.com E-Mail: [email protected]
Q&A