scaling lamp doesn't have to suck

35
scaling LAMP doesn’t have to suck! @stevencorona (800 million api calls/day)

Upload: stephen-corona

Post on 08-May-2015

338 views

Category:

Technology


5 download

DESCRIPTION

Scale PHP Effortlessly— Learn the 80/20 of owning your LAMP stack and get back to coding! I spent a year sleeping next to my laptop, rolling over to restart Apache, as I scaled Twitpic to 50m users. Save your sanity and learn from my mistakes.

TRANSCRIPT

Page 1: Scaling LAMP doesn't have to suck

scaling LAMPdoesn’t have to

suck!@stevencorona

(800 million api calls/day)

Page 2: Scaling LAMP doesn't have to suck

let’s learn how to scale php

!

...in 40 minutes

Page 3: Scaling LAMP doesn't have to suck

LAMPLINUX

APACHEMYSQL

PHP

Page 4: Scaling LAMP doesn't have to suck

a better LAMPLINUX +TUNING

NGINXPHP-FPM

PERCONA

NSQREDIS

Page 5: Scaling LAMP doesn't have to suck

nginx loadbalancer

nginx php-fpm

MySQL Master

nginx php-fpm

nginx php-fpm

nsq

MySQL Slavecache

MySQL Slave

Page 6: Scaling LAMP doesn't have to suck

tuning linux•linux is mostly sane •use latest kernel •distro doesn’t matter •(i like ubuntu)

Page 7: Scaling LAMP doesn't have to suck

top 3 settingsopen_files

network/sysctl tuning i/o scheduler

!

most settings you find online are outdated

Page 8: Scaling LAMP doesn't have to suck

open filesdefault is 1024

on linux, each socket = open file !

/etc/security/limits.conf!soft nofile 65535!hard nofile 65535

Page 9: Scaling LAMP doesn't have to suck

sysctl tuning!

thousands of settings newer kernels autotune network

!

/etc/sysctl.conf!net.core.somaxconn=65536!

net.ipv4.ip_local_port_range=2000 62000!

Page 10: Scaling LAMP doesn't have to suck

i/o schedulerlinux has swappable scheduler

default is cfq !

$ echo “deadline” > \!/sys/block/sda/queue/scheduler

!

+50% IOPS on MySQL

Page 11: Scaling LAMP doesn't have to suck

nginxopen-source HTTP server

swap out apache easy + sane config

nginx.org

Page 12: Scaling LAMP doesn't have to suck

nginxmost popular webserver

of top 1000 sites !

15,000 requests/second with 20% CPU and100MB RAM

Page 13: Scaling LAMP doesn't have to suck

nginx

Page 14: Scaling LAMP doesn't have to suck

nginxload balancer http cache

fastcgi proxy web server

Page 15: Scaling LAMP doesn't have to suck

php

php-fpm is king standard with php5.3

Page 16: Scaling LAMP doesn't have to suck

php-fpmuse static worker pool

4x number of CPU cores !

pm=static!pm.max_children=128

Page 17: Scaling LAMP doesn't have to suck

phpphp 5.5 is much faster

than 5.4 and 5.3 !

USE IT!

Page 18: Scaling LAMP doesn't have to suck

php!

php 5.5 includes Zend Opcache

(no more APC) !

10-20% faster than APC in real world

Page 19: Scaling LAMP doesn't have to suck

php

horizontal scaling watch out for sessions!

Page 20: Scaling LAMP doesn't have to suck

php + mysqlmysql persistent

connections are GOOD to use.

!

despite what the internet says

Page 21: Scaling LAMP doesn't have to suck

percona!

open source mysql fork !

patches from twitter and google

percona.com

Page 22: Scaling LAMP doesn't have to suck

perconadrop-in replacement

!

faster at scale, more reliable

Page 23: Scaling LAMP doesn't have to suck

percona

new transactions per minute (more is better)

Page 24: Scaling LAMP doesn't have to suck

percona•only use innodb •disable query cache •enable thread pool

thread_handling=pool-of-threads!

Page 25: Scaling LAMP doesn't have to suck

perconastop wasting time

use SSDs !

Page 26: Scaling LAMP doesn't have to suck

redisin the old days, we used

memcache !

redis is better !

redis.io

Page 27: Scaling LAMP doesn't have to suck

redis•data persisted to disk •hot cache •500,000 GET/second

Page 28: Scaling LAMP doesn't have to suck

redisconfig is great out of box

!

avoid stalls on EC2 use HVM instances

Page 29: Scaling LAMP doesn't have to suck

redislots of php libraries

!

use pecl-redis c extension

pecl install redis

Page 30: Scaling LAMP doesn't have to suck

nsqmessage queue server in golang by

bit.ly !

we use it to process 800 million events/day

!

https://github.com/bitly/nsq

Page 31: Scaling LAMP doesn't have to suck

nsq•nsqphp is best php library •pub/sub model •do work async •api calls •defer long-running work

!

https://github.com/davegardnerisme/nsqphp

Page 32: Scaling LAMP doesn't have to suck

nsq

Page 33: Scaling LAMP doesn't have to suck

nsq

DON’T USE MYSQL AS A WORK QUEUE!

Page 34: Scaling LAMP doesn't have to suck

don’t scale code!

your code is fast enough !

scale infrastructure

Page 35: Scaling LAMP doesn't have to suck

thanks!check out my book—

SCALING PHP

http://scalingphpbook.com