some advice from the guy who handle your applications uptime - scalaio 2013
TRANSCRIPT
![Page 1: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/1.jpg)
HOW TO SCALE YOUR APP SOME ADVICE FROM THE GUY WHO HANDLES YOUR APP UPTIME
QUENTIN ADAM AT SCALA.IO
@WAXZCE2013
![Page 2: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/2.jpg)
MY DAY TO DAY WORK : CLEVER CLOUD, MAKE YOUR APP RUN ALL THE TIME
![Page 3: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/3.jpg)
WHEN YOU NEED TO SCALE
THERE ARE 2 WAYS
![Page 4: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/4.jpg)
GROWING AND GROWING UNTIL YOU EXPLODE OR BECOME WEIRD
![Page 5: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/5.jpg)
OR SPLIT THE WORK AND MAKE YOUR SOFTWARE WORK AS A TEAM
![Page 6: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/6.jpg)
Build an army of fat app
YOU CAN DO BOTH
![Page 7: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/7.jpg)
SO WE NEED TO BE ABLE TO DISPATCH THE WORK
SCALE OUT
• Many workers doing the same thing
• No SPOF
• Growing is more easy
• Introduce best practice
SCALE UP
• 1 Fat instance
• 1 Fat application
• SPOF (single point of failure)
• Hard to maintain
• Always has a limit
• Short term meaning
BEST LONG
TERM
SOLUTION
![Page 8: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/8.jpg)
SO, HOW TO SCALE OUT ?JUST SOME FACTS
![Page 9: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/9.jpg)
SPLIT PROCESS AND STORAGE
Storage• Databases• Files• Sessions• Events• …
Code• Can be replicated• Stateless• Process
![Page 10: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/10.jpg)
Picking one instance or another doesn’t matter
STATELESSNESS IS THE KEY
![Page 11: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/11.jpg)
CONSIDER MORE THINGS AS DATA• User account
• Users data
• Files
• Sessions
• Events
![Page 12: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/12.jpg)
TRUST YOUR MIDDLEWARE
![Page 13: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/13.jpg)
USE AN EVENT BROKER TO MODULARIZE YOUR APP
• AMQP
• Celery
• 0MQ
• Redis
• JMS
• Even some http chunk or websocket
• Some case : hadoop, akka…
• …
![Page 14: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/14.jpg)
CRON + FS IS NEITHER AN EVENT QUEUE NOR A JOB SCHEDULER
![Page 15: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/15.jpg)
CHOOSE YOUR DATASTORE WISELYYOU CAN SHOULD USE MANY DATASTORES
![Page 16: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/16.jpg)
DATASTORE CHOICES ARE DRIVEN BY USAGE
Make decisions based on
needs
Do I need atomicity of requests ?
Do I need concurrent access ?
Do I mostly read or write ?
Do I need relational ?
Do I need big storage capacity ?
Do I need high
availability ?
![Page 17: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/17.jpg)
USE ONLINE DATABASE / BE READY TO TEST IN JUST A FEW MINUTES NO NEED TO TRASH YOUR COMPUTER
![Page 18: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/18.jpg)
COMMON MISTAKES
![Page 19: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/19.jpg)
DO NOT USE THE FILE SYSTEM AS A DATASTORE
File system are POSIX compliant
• POSIX is ACID• POSIX is powerful but is a bottleneck • File System is the nightmare of ops • File System creates coupling (host provider/OS/language)• SPOF-free multi tenant File System is a unicorn
STORE IN DATABASE, OR IN A DATASTORE LIKE S3/RIAKCS DEDICATED TO FILE MANAGEMENT
![Page 20: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/20.jpg)
LOGS IN FILES I HATE IT
![Page 21: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/21.jpg)
USE STREAMING I/O TO STREAM DATA DIRECTLY TO DATABASE
HTTP Post dataTemporarily store as file
or in memory
Store it into your storage
backend
Say OK to client
![Page 22: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/22.jpg)
USE STREAMING I/O TO STREAM DATA DIRECTLY TO DATABASE
HTTP Post data
Directly stream your data to
Storage backend
Say OK to client
![Page 23: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/23.jpg)
DO NOT USE MEMORY AS DATABASELIKE : SHARED / GLOBAL VARIABLE, CACHE “IN THE CODE”, INTENSIVE SESSION USAGE…
![Page 24: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/24.jpg)
DO NOT USE A VARIABLE FOR MORE THAN ONE REQUEST
![Page 25: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/25.jpg)
F(X) = X * 2F(2) = 4^ WE ASSUME THATFOR SAME INPUT, SAME OUTPUT
![Page 26: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/26.jpg)
You have to understand that : you’re functional programing addicts ;-)
IT’S LIKE MATH FUNCTION
![Page 27: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/27.jpg)
Example : GET should not change data on server
RESPECT HTTP
![Page 28: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/28.jpg)
data will be lost
CODE WILL FAIL
![Page 29: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/29.jpg)
CAREFUL USE OF DARK MAGIC
![Page 30: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/30.jpg)
DO NOT CREATE MONSTERS
![Page 31: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/31.jpg)
MAKE HARD COMPUTATION ASYNC
![Page 32: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/32.jpg)
SPLIT THE CODE : MODULES
• Smallest code base
• Deploy as service for each other
• Focus on best technology for a problem
![Page 33: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/33.jpg)
SCALE YOUR TEAMMODULARIZE YOUR TEAM
![Page 34: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/34.jpg)
ALWAYS USE A REVERSE PROXY
Y U NO USE ONE ?
![Page 35: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/35.jpg)
DO NOT BUILD “THE SERVER” WITH NO DOC
![Page 36: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/36.jpg)
USE PROCESS DEPLOYMENT
![Page 37: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/37.jpg)
THE GOOD WAY OF DEPLOY AN APP :
git push
![Page 38: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/38.jpg)
EASY MOVING OR INCIDENT MANAGEMENT
![Page 39: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/39.jpg)
KEEP CALM UNDER FIRE
![Page 40: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/40.jpg)
TRACK BUG & GET METRICS
![Page 41: Some advice from the guy who handle your applications uptime - scalaIO 2013](https://reader036.vdocuments.mx/reader036/viewer/2022070313/554bb315b4c90594278b4626/html5/thumbnails/41.jpg)
I’m @waxzce on twitter
I’m the CEO of
a non exclusive scala PaaS provider, give it a try ;-)
THX FOR LISTENING & QUESTIONS TIME
sponsors