How AWS Drives Argyle Social

A bit about Argyle Social

• What does Argyle do?

– Social Media management, engagement, & Analytics

• We are a small & very efficient dev team

– Adam (general), Mike (platform), & Josh (design)

Social

AWS is Startup Friendly

RDS

SimpleDB

Feb ‘10

‘The Server’

MySql

Dec ‘10

App ServerSimple

DBMySql SQS

SimpleDB

May ‘10

‘The Server’

MySql

SQS

Servers Galore

RDS

Jun ‘11

Jobs

SimpleDB

SQS

SocialSocialSocialStream

MySqlMySql

App ServerURL

Redir

S3 Support

Our current setup

• EC2 – 9 Servers

• SQS – 5 Queues

• SimpleDB – 1 Domain

• RDS – 1 DB; 5MM records

• Support & S3 – Just starting out

• Let’s dig into SQS, SimpleDB, & Support

SQS – Simple Queue Service

• Used for processing:– Twitter Streams

– Email queue

– Filtering for significant items

– Log Processing for Goals

• Very easy to get started:

How it works

Process Everything

Queue Everything

Process MessageProcess Message

Process MessageProcess Message

Queue

Argyle SQS Usage

AWS Service Land

Interweb

Streams RDS Database

Twitter Site Streams

Queue

Twitter Processor

Twitter Processor

Twitter Processor

Twitter Stream Reader

Twitter Stream Reader

Twitter Stream Reader

Argyle Web Interface

Users

SQS – What we learned

• Not super fast– Lag between pushing and seeing an item on pull

• Visibility timeout is key– Make sure you have time to finish and delete– ‘at least once delivery’ you may get something twice

• Not guaranteed FIFO• Default size

– Restrictive Sizing can lead to errors– 1kB to 64kB

• Getting Queue Url– Create it and store it, don’t look it up

SQS - Conclusions

• Very easy to get bullet-proof queues

– No administration costs

• Expect some limitations that rolling or running your own won’t incur

– MSMQ, Spread, etc

Simple DB

• Easy storage & retrieval of Key/Value pairs

• Eventual consistent read vs. consistent read

• Grows with your data

• NoSQL N00bs: take your time

– This isn’t relational, which is obvious, but may take some time to really wrap your head around

– No JOIN or GROUP BY, etc

Argyle SimpleDB Usage

Redirector

Memcached

SimpleDB

Redirecting a URL

ar.gy/123 ar.gy/12

AWS Service Land

Interweb

Cached URL Non-cached URL

Click log

Redirector

Memcached

SimpleDB

ar.gy/ftw

Shortening a URL

SDB – Scratch Pad

*This makes your life easier, especially when starting out or debugging

SDB – Lessons Learned

• Be aware of the limits:

Source: http://en.wikipedia.org/wiki/Amazon_SimpleDB#Limitations

SDB – Lessons Learned

• We’ve had some availability issues– Sometimes no response

– Sometimes no write capabilities

• Storage of big items is possible, but must be linked out

• Can separate domains to increase performance

SimpleDB - Conclusions

• Great, easy storage

• Again, reduced maintenance costs

• Have to change your mindset from RDBMS

AWS Support

• We’re new to it

• We don’t expect much

• It can be helpful toget nagging questionsresolved

• This is your company on the line, right?

AWS Support

• We ‘lost’ a server; 4.5 hours after submitted:

• Clarification on “underlying host issue” pls; 40 minutes to reply

Support Conclusions

• Well worth the security

• Rumor has it that upgrading to higher levels isn’t worth it – any experiences out there?

– (Update: we got good feedback around higher levels of service from Adzerk)

• You still have to design for failure

Q & A

• About Argyle

• About your own AWS setup