scaling django to the sky

SCALING DJANGO TOSCALING DJANGO TOTHE SKY THE SKY

HOW SMALL DECISIONS CAN MAKE A BIGHOW SMALL DECISIONS CAN MAKE A BIGDIFFERENCEDIFFERENCE

!# Created by = ; Social =Naren Arya @Narenarya3

PyDelhi 2016 http://narenaryan.github.io/PyDelhi-Infrastructure-proposal/?print-pdf#/

1 of 22 Friday 12 February 2016 08:30 AM

WHOAMI WHOAMII am a

Python & JavaScript programmerLinux & Books loverDevOps Engineer by professionBlogging & Chess are favorite hobbies



PART-1PART-1 DESIGN OPTIMIZATION DESIGN OPTIMIZATION

++

STACK OPTIMIZATION STACK OPTIMIZATION====

SPEED SPEED



THUMB RULE OF MVCTHUMB RULE OF MVC

In most cases web application wastes it's time in interactingwith database. So Database is the slowest part of any web

framework.



EVIL SIDE OF DJANGO ORMEVIL SIDE OF DJANGO ORMMyth: Django ORM is dead easy to use in a web application

Truth: Django ORM can make a web application dead slow



TIPS FOR CODE OPTIMIZATIONTIPS FOR CODE OPTIMIZATIONThe techniques those can reduce the DB hits siginificantly:

Advanced ORM implementationsQuery cachingDjango template cachingNon-blocking codeAlternate data stores



ORM FACTORSORM FACTORSCACHED PROPERTIESCACHED PROPERTIES

Use cached proerty to avoid multiple db queries



In a single query we can fetch the related objects.Don'tmake multiple queries for every thing.

PREFETCH AND SELECT RELATEDPREFETCH AND SELECT RELATED



A novice developer's simple mistake at 20 places can costyou more servers

HOW EASILY ONE CAN ADD AHOW EASILY ONE CAN ADD ABOTTLENECK TO DJANGOBOTTLENECK TO DJANGO



like

Which can have greater performance bottlenecks

MISTAKES! MISTAKES!MISTAKES! MISTAKES!We all are doing basic mistakes

... bad design ...

... writing inefficient code

not analyzing latest documentation...



OTHER FACTORSOTHER FACTORS////

Non Blocking Django CodeUse Queue and Job systemsSpawn non-blocking IO from Django

Caching internal componentsQuery CachingTemplate Caching



OPTIONS FOR NON-BLOCKINGOPTIONS FOR NON-BLOCKINGCODECODE

RabbitMQ + Celery1. asyncio (Python 3) || future-requests (python 2.7+)2.



OPTIONS FOR QUERY CACHINGOPTIONS FOR QUERY CACHING

by Tobias McNulty by Jason Moiron

Cache-MachineJohnny-Cache



OPTIONS FOR TEMPLATEOPTIONS FOR TEMPLATECACHINGCACHING

Must use Django template cachingCache-Machine provide support for Jinja2 templating



PART-2PART-2STACK OPTIMIZATIONSTACK OPTIMIZATION

“If you are going to kick a tiger inthe ass, you’d better have a plan

to deal with the teeth” - Tomclancy



AN IDEAL STACKAN IDEAL STACKPyDelhi 2016 http://narenaryan.github.io/PyDelhi-Infrastructure-proposal/?print-pdf#/


ESSENTIAL COMPONENTSESSENTIAL COMPONENTSLoad balancersWeb accelaratorsCache backendsDatabase SQL vs NoSQL



OPTIONS FOR STACKOPTIONS FOR STACKThese open source so�ware proved worthy in powering

huge web sites like Instagram, Disqus.

Load balancers - HAProxyWeb accelarators - VarnishCache backends - Redis, MemcachedDatabase SQL vs NoSQL - MongoDB, PostgreSQL JSONstore



With help of floating IP we can create a cluster of loadbalancers in front of Django application servers

IMPORTANCE OF LOADIMPORTANCE OF LOADBALANCINGBALANCING



Which is better web accelarator?

NGINX VS VARNISHNGINX VS VARNISH



APPLES ARE NOTAPPLES ARE NOTAIRPLANESAIRPLANES

Databases

MONGODB VS POSTGRE SQLMONGODB VS POSTGRE SQLJSON STOREJSON STOREDatastores

REDIS VS MEMCACHEDREDIS VS MEMCACHED



THANK YOU DELHI THANK YOU DELHI For giving me this wonderful opportunity

- -

self.python_blogself.personal_blog



scaling django to the sky

Technology