Deploying and Scaling with AWS Matt Conway, VP Engineering

matt@snapmylife.com

About Us

 Company Profile •  Founded 2007 •  Tier 1 venture-backed •  Experienced team of eleven in

Needham, MA •  Developer of direct-to-consumer

mobile services

 SnapMyLife Service •  Mobile photo sharing community •  Best mobile user experience •  Premium demographic •  Rapidly growing local & global

audience

2

Key Stats

•  Launched April 2008

•  Mobile website (no download required), iPhone web/app, Internet

•  1.5+ million unique visitors / month

•  15+ million monthly page views (Aug 2008)

•  Over 90% traffic from mobile devices

•  Visitors from every country in the world

•  400,000+ registered users

Screenshots

Mobile iPhone

Desktop

Overview

 EC2 •  Entire web serving stack •  Daemons for background/batch processing •  Internal infrastructure

 S3 •  Cheap CDN •  Backups for EC2

 Mechanical Turk •  Content filtering •  Content classification

4

EC2 Costs

 Xen based cluster on leased hardware •  Time sink •  Not as flexible •  Slow to provision (Days vs Minutes) •  More expensive (~$1.8K/month)

 EC2/S3 •  We hardly ever think about it, just use it •  Unlimited playground without fear of breakage •  Clone your entire production stack in minutes •  Works out cheaper (~$1.2K/month)

5

EC2/S3 Architecture

 Fairly standard architecture  DNS – Web – App – Memcached - Mysql  Assume and plan for failure  Use a role based architecture to easily scale

roles independently of each other (e.g. add an app server or db slave)

 Roles scale in as well as out – incredibly useful to have production cloned to a single instance for testing new features.

6

EC2 Instance Allocation

  Static Vs Dynamic instance allocation •  We went static route •  Our traffic graph pretty flat due to global nature •  Easier, dynamic cost savings not worth the extra

complexity for us •  We watch our load carefully, given a critical alert, we

can scale up in a matter of minutes by increasing the instance count for the impacted roles.

  All instance configuration lives in our source tree. New instances to handle any combination of roles is only a couple of commands and a few minutes away

  We open sourced our framework for doing so: The rubber project on github - http://github.com/wr0ngway/rubber/wikis

7

Mechanical Turk

 All of our content gets classified through turk  Some of our content gets tagged  We interact with turk solely through

automation – large quantities preclude use of web interface

 Results were better than anticipated   Incredibly quick turnaround, 24x7  Very hard to match this performance given

the hiring constraints of a startup

8

Mturk costs

 Depends a lot on how you do it  Keep questions simple, pay less  Multiple questions, better result  Example of a months usage •  ~64K picture moderations (x3) •  ~28K pictures tagged •  Total Cost ~$3.3K (AWS commission $1.1K)

9

Things we’d like to see

 A reliable way to send mail from EC2  Faster S3 response - New AWS CDN

should address this  Automatic billing for Mturk  A better plan for large volume mturk usage –

commission is normally 10%, but the half cent minimum makes it 50% if all you do are 1c HITs

10

Questions?

 Contact Matthew Conway VP Engineering matt@snapmylife.com

11