Vimeo on EC2

1

Vimeo?

The best place for video creators to put their video

The best experience for everyone:

• Video quality

• Video player

• Community

• Tools for sharing and privacy

Free Users Plus Users

500mb upload per week 5GB per week

Basic toolsAdvanced stats, customization, advanced privacy options

Wait for transcoding No wait for transcoding

Free $60/year

2

S3, 2007

It was there

High cost, small servers in managed hosting

S3 had more features (query string auth – store and serve)

Why AWS?

3

EC2, 2008

Everyone uploads on Sundays

Needed auto-scaling: two peak days per week, peak was 4x normal encoding load

Free & fast transfer between EC2 and S3

The Workflow

4

Raw sourcefile

User’s Browser

• Metadata

• Verified

EC2 Upload

Machine

Sourcefilestored

S3

• HD

• SD

• Mobile

• Thumbnails

EC2 Transcoder

Playable video files

CDN

Reserved capacity is cheaper

Reserve capacity for paying customers

• Guaranteed availability, even when Amazon runs out of on-demand

Reserve capacity for 100% utilization

Reserve capacity for 75% utilization

Still need on-demand for elasticity

On-demand, Reserved

5

Upload

m1.large instances

Nginx + nodejs + SQS

Upload/Transcoding Infrastructure

6

Transcoding

c1.xlarge instances

Reserved, On demand and spots

Spots for low priority jobs. Instances will shut down without warning

Leverage open source libraries (boto in python)

Keep it Simple

Onetime spot requests, One instance per request, across all availability zones

Bidding strategy

Spots@Vimeo

7

Thinking of spots for your Database and Web servers?

Failover to On demand when spot market is saturated

Never kill spots. Let Amazon do it.

Watch out for open spot requests. Add expiry to your requests.

Keep tweaking your Algorithm.

Conceptualization the Obvious

8

Some months we save 50% over pure on-demand instances

Guaranteed availability for our plus users

Results

9