![Page 1: Mobile + HA + Cloudqcontokyo.com/data_2014/BS1-1_EugeneCiurana.pdf · Mobile + HA + Cloud Eugene Ciurana!! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin!](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98d97b7511a59e711a12e/html5/thumbnails/1.jpg)
Mobile + HA + Cloud
Eugene Ciurana!!pr3d4t0r - irc.freenode.net!##java, ##security, #awk, #python, #bitcoin!irc.oftc.net: #tor, #tor-dev, #[email protected]
![Page 2: Mobile + HA + Cloudqcontokyo.com/data_2014/BS1-1_EugeneCiurana.pdf · Mobile + HA + Cloud Eugene Ciurana!! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin!](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98d97b7511a59e711a12e/html5/thumbnails/2.jpg)
About Eugene...• 15+ years building HA, mission-critical
systems!
• State-of-the-art engineering for some of the biggest and brightest worldwide!
• Open source evangelist and author
• Not a web guy...!
• Adviser to several venture capital funds in the US, Asia, and Europe!
• Now providing business and technology development advise to mobile and enhanced reality companies worldwide
![Page 3: Mobile + HA + Cloudqcontokyo.com/data_2014/BS1-1_EugeneCiurana.pdf · Mobile + HA + Cloud Eugene Ciurana!! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin!](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98d97b7511a59e711a12e/html5/thumbnails/3.jpg)
About Eugene…2007 - 2014
SOA!Cloud Computing
![Page 4: Mobile + HA + Cloudqcontokyo.com/data_2014/BS1-1_EugeneCiurana.pdf · Mobile + HA + Cloud Eugene Ciurana!! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin!](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98d97b7511a59e711a12e/html5/thumbnails/4.jpg)
Very Important
Please Ask Questions!!(don’t be shy...)
![Page 5: Mobile + HA + Cloudqcontokyo.com/data_2014/BS1-1_EugeneCiurana.pdf · Mobile + HA + Cloud Eugene Ciurana!! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin!](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98d97b7511a59e711a12e/html5/thumbnails/5.jpg)
Mobile HA and Cloud• Bootstrapping a mobile startup almost
always includes a cloud component!
• Cloud services and servers (Saas and PaaS)!
• Main reason? Battery life!!
• Processing and net I/O == battery drain
![Page 6: Mobile + HA + Cloudqcontokyo.com/data_2014/BS1-1_EugeneCiurana.pdf · Mobile + HA + Cloud Eugene Ciurana!! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin!](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98d97b7511a59e711a12e/html5/thumbnails/6.jpg)
Mobile HA and Cloud
• Cloud services are Always On!
• Until they aren’t!
• App should always appear to be live for the user!
• All services must be self-healing
![Page 7: Mobile + HA + Cloudqcontokyo.com/data_2014/BS1-1_EugeneCiurana.pdf · Mobile + HA + Cloud Eugene Ciurana!! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin!](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98d97b7511a59e711a12e/html5/thumbnails/7.jpg)
Mobile HA and Cloud• HA in mobile != HA for desk or web
apps!
• Assume the device is a cache!
• The service provider is The Law
![Page 8: Mobile + HA + Cloudqcontokyo.com/data_2014/BS1-1_EugeneCiurana.pdf · Mobile + HA + Cloud Eugene Ciurana!! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin!](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98d97b7511a59e711a12e/html5/thumbnails/8.jpg)
Mobile HA and Cloud
![Page 9: Mobile + HA + Cloudqcontokyo.com/data_2014/BS1-1_EugeneCiurana.pdf · Mobile + HA + Cloud Eugene Ciurana!! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin!](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98d97b7511a59e711a12e/html5/thumbnails/9.jpg)
Which Cloud Provider?
• SaaS - interfacing with ready-made services; Salesforce.com!
• PaaS - Google App Engine, CloudHub!
• IaaS - Amazon Web Services, MS Azure
![Page 10: Mobile + HA + Cloudqcontokyo.com/data_2014/BS1-1_EugeneCiurana.pdf · Mobile + HA + Cloud Eugene Ciurana!! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin!](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98d97b7511a59e711a12e/html5/thumbnails/10.jpg)
Which Cloud Provider?
• Your architecture will be a mix of mobile, web app, services, and database!
• Decisions: run your own data center, IaaS, or Paas?!
• No brainer answer: AWS EC2!
• Keep an eye on that bill!
![Page 11: Mobile + HA + Cloudqcontokyo.com/data_2014/BS1-1_EugeneCiurana.pdf · Mobile + HA + Cloud Eugene Ciurana!! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin!](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98d97b7511a59e711a12e/html5/thumbnails/11.jpg)
A Word About Cloud Costs
• Don’t treat your cloud services as a substitute to a data center or dedicated co-located servers!
• Leverage spot and reserved instances!
• Otherwise your costs will balloon like mad!
![Page 12: Mobile + HA + Cloudqcontokyo.com/data_2014/BS1-1_EugeneCiurana.pdf · Mobile + HA + Cloud Eugene Ciurana!! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin!](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98d97b7511a59e711a12e/html5/thumbnails/12.jpg)
Cloud Gives You Everything• At a price - be judicious
![Page 13: Mobile + HA + Cloudqcontokyo.com/data_2014/BS1-1_EugeneCiurana.pdf · Mobile + HA + Cloud Eugene Ciurana!! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin!](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98d97b7511a59e711a12e/html5/thumbnails/13.jpg)
Pricing Horror Story• Successful app - no capacity planning!
• Daily bill? $70,000 USD/day
• Used all AWS stack services!
• Lots of servers, all regular instances
![Page 14: Mobile + HA + Cloudqcontokyo.com/data_2014/BS1-1_EugeneCiurana.pdf · Mobile + HA + Cloud Eugene Ciurana!! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin!](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98d97b7511a59e711a12e/html5/thumbnails/14.jpg)
Avoid Pricing Surprises
• Invest time in understanding your cloud provider’s pricing model!
• Use Linux/open source wherever possible!
• Understand the implications of using standard database, caching, etc. vs. using AWS’s Elastic Cache, Dynamo, RDS, etc.!
• Amazon Web Services best? ELB, SSL termination
![Page 15: Mobile + HA + Cloudqcontokyo.com/data_2014/BS1-1_EugeneCiurana.pdf · Mobile + HA + Cloud Eugene Ciurana!! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin!](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98d97b7511a59e711a12e/html5/thumbnails/15.jpg)
Typical Application Architecture
• iOS or Android!
• App server!
• Message broker!
• Database!
• Caching
RoR, PHP, CherryPy - hip
Whassat??
MySQL, mongoDB, Dynamo, RDS
Later....
Android is the cool! Bad
![Page 16: Mobile + HA + Cloudqcontokyo.com/data_2014/BS1-1_EugeneCiurana.pdf · Mobile + HA + Cloud Eugene Ciurana!! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin!](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98d97b7511a59e711a12e/html5/thumbnails/16.jpg)
Success!
• You built a popular app!
• You think/know you can scale because it’s all “on the cloud”!
• Nope! You’ll have to rework a lot of stuff -- better plan ahead
![Page 17: Mobile + HA + Cloudqcontokyo.com/data_2014/BS1-1_EugeneCiurana.pdf · Mobile + HA + Cloud Eugene Ciurana!! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin!](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98d97b7511a59e711a12e/html5/thumbnails/17.jpg)
Scalable Application Architecture
• iOS or Android!
• App server!
• Message broker!
• Database!
• Caching
Mule Integration, Spring - robust
ActiveMQ, RabbitMQ
Neo4J, MySQL cluster, mongoDB - NO Dynamo
Memcached, Redis
iOS - better monetization
![Page 18: Mobile + HA + Cloudqcontokyo.com/data_2014/BS1-1_EugeneCiurana.pdf · Mobile + HA + Cloud Eugene Ciurana!! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin!](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98d97b7511a59e711a12e/html5/thumbnails/18.jpg)
Managing Your Cloud• Find the meanest, leanest, toughest,
smartest macho hombre DevOps guy you can hire!
• Chef, Puppet!
• Leverage cloud DNS!
• Don’t forget monitoring!
• Zabbix > Nagios > AWS monitoring!
• New Relic > AWS monitoring
Plan deployment via
configuration - avoid VM image-based deployments!
Hard and expensive to manage
![Page 19: Mobile + HA + Cloudqcontokyo.com/data_2014/BS1-1_EugeneCiurana.pdf · Mobile + HA + Cloud Eugene Ciurana!! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin!](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98d97b7511a59e711a12e/html5/thumbnails/19.jpg)
App Interface - SOA
• Your mobile app talks to the servers via an API!
• Your servers talk to one another over the same API!
• Build around services, no tight coupling!
![Page 20: Mobile + HA + Cloudqcontokyo.com/data_2014/BS1-1_EugeneCiurana.pdf · Mobile + HA + Cloud Eugene Ciurana!! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin!](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98d97b7511a59e711a12e/html5/thumbnails/20.jpg)
App Interface
• Data exchange? JSON!
• JASON-LD or other specilizations OKi!
• Don’t be too granular!
• Treat data as resources!
• RESTful
![Page 21: Mobile + HA + Cloudqcontokyo.com/data_2014/BS1-1_EugeneCiurana.pdf · Mobile + HA + Cloud Eugene Ciurana!! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin!](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98d97b7511a59e711a12e/html5/thumbnails/21.jpg)
App Interface• All APIs must be stateless!
• The mobile app or the server keep state, but no session management!
• Round robin load balancing!
• Cache, cache, cache, and cache!
• Even if your DB supports all colors of “smart caching” - it won’t scale
raml.org
![Page 22: Mobile + HA + Cloudqcontokyo.com/data_2014/BS1-1_EugeneCiurana.pdf · Mobile + HA + Cloud Eugene Ciurana!! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin!](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98d97b7511a59e711a12e/html5/thumbnails/22.jpg)
Database• Define your data model in advance and
plan for massive growth!
• All your operations must be designed and implemented for eventual consistency!
• Think of full replication!
• Use a DAO of some sort - don’t talk to it directly
![Page 23: Mobile + HA + Cloudqcontokyo.com/data_2014/BS1-1_EugeneCiurana.pdf · Mobile + HA + Cloud Eugene Ciurana!! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin!](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98d97b7511a59e711a12e/html5/thumbnails/23.jpg)
Database
Brewer’s!CAP!
Theorem!!
Pick any two!
Pick Any Two
C A
P
Consistency Availability
Partition tolerance
RelationalKey-Value
Column-OrientedDocument-Oriented
Graph
RDBMs (Oracle, MySQL), Aster Data, Green Plum, Vertica
Dynam
o, Vo
ldemort
, Tok
yo C
abine
t, KAI
, Cas
sand
ra,
Simple
DB, C
ouch
DB, R
iak, T
itan
mongoDB, Terrastore, Datastore, Hypertable, Hbase,
Redis, Berkeley DB, MemcacheDB, Neo4J
![Page 24: Mobile + HA + Cloudqcontokyo.com/data_2014/BS1-1_EugeneCiurana.pdf · Mobile + HA + Cloud Eugene Ciurana!! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin!](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98d97b7511a59e711a12e/html5/thumbnails/24.jpg)
Caching
• Nobody’s ever been fired for using Memcached!
• Redis if the app needs access to collections, counters, and other complex data structures!
• Roll your own servers - more management, but finer-grained control
![Page 25: Mobile + HA + Cloudqcontokyo.com/data_2014/BS1-1_EugeneCiurana.pdf · Mobile + HA + Cloud Eugene Ciurana!! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin!](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98d97b7511a59e711a12e/html5/thumbnails/25.jpg)
Summly Logical Architecture
EngineRESTful services
Aggregatorcontent summarization
Back EndSummly admin
Front Endlight summaries
SummarizationEngine mongoDB
L1 Cache
mobiledevicesmobile
devicesmobiledevicesmobile
devicesmobiledevices
content publishercontent publisher
content publishercontent publisher
content publisher
Firewall
![Page 26: Mobile + HA + Cloudqcontokyo.com/data_2014/BS1-1_EugeneCiurana.pdf · Mobile + HA + Cloud Eugene Ciurana!! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin!](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98d97b7511a59e711a12e/html5/thumbnails/26.jpg)
Summly Physical Architecture
Aggregator 0Web Crawler
Aggregator 1English
Cache 0
LB
Web 0 Web 1
shardMaster
Slave
shardMaster
Slave
Aggregator 2English
Aggregator 3French
Aggregator 4Spanish
Aggregator 5Spanish
Engine ClusterRESTful API provider (mobile, web, and admin console)
LB
LB LB Bad Idea!
![Page 27: Mobile + HA + Cloudqcontokyo.com/data_2014/BS1-1_EugeneCiurana.pdf · Mobile + HA + Cloud Eugene Ciurana!! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin!](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98d97b7511a59e711a12e/html5/thumbnails/27.jpg)
Summly Physical Architecture
AggregatorMaster
Aggregator 1English
LB
Web 0 Web 1
shardMaster
Slave
shardMaster
Slave
Aggregator 2English
Aggregator 3French
Aggregator 4Spanish
Aggregator 5Spanish
Engine ClusterRESTful API provider (mobile, web, and admin console)
Message Broker (ActiveMQ)
Cache 0
![Page 28: Mobile + HA + Cloudqcontokyo.com/data_2014/BS1-1_EugeneCiurana.pdf · Mobile + HA + Cloud Eugene Ciurana!! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin!](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98d97b7511a59e711a12e/html5/thumbnails/28.jpg)
Almost Done
Are there any questions?
![Page 29: Mobile + HA + Cloudqcontokyo.com/data_2014/BS1-1_EugeneCiurana.pdf · Mobile + HA + Cloud Eugene Ciurana!! pr3d4t0r - irc.freenode.net! ##java, ##security, #awk, #python, #bitcoin!](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec98d97b7511a59e711a12e/html5/thumbnails/29.jpg)
Mobile + HA + Cloud
Thanks for Coming!
Download this presentation from:!http://ciurana.eu/qcon2014/TOK/mobileHA
Eugene Ciurana!!pr3d4t0r - irc.freenode.net!##java, ##security, #awk, #python, #bitcoin!irc.oftc.net: #tor, #tor-dev, #[email protected]