building a website to scale to 100 million page views per day and beyond
DESCRIPTION
TRANSCRIPT
![Page 1: Building a Website to Scale to 100 Million Page Views Per Day and Beyond](https://reader034.vdocuments.mx/reader034/viewer/2022051400/54c672584a79594a538b46cb/html5/thumbnails/1.jpg)
Building a Website To Scale
www.ManwinJobs.com
Target: 200 Million page views per day and beyond!By Eric Pickup
[email protected] Twitter: EricPickupYP
![Page 2: Building a Website to Scale to 100 Million Page Views Per Day and Beyond](https://reader034.vdocuments.mx/reader034/viewer/2022051400/54c672584a79594a538b46cb/html5/thumbnails/2.jpg)
www.ManwinJobs.com
Contents
1. The Context
2. The Requirements
3. The Architecture
4. The Good and the Bad
![Page 3: Building a Website to Scale to 100 Million Page Views Per Day and Beyond](https://reader034.vdocuments.mx/reader034/viewer/2022051400/54c672584a79594a538b46cb/html5/thumbnails/3.jpg)
www.ManwinJobs.com
What are we talking about?
YP First Launched
1 Million daily
visitors
100,000 Uploads
Aug2006
Apr2007
Dec2007
Feb2008
Apr2011
100 million daily
page views
Acquired by Manwin
![Page 4: Building a Website to Scale to 100 Million Page Views Per Day and Beyond](https://reader034.vdocuments.mx/reader034/viewer/2022051400/54c672584a79594a538b46cb/html5/thumbnails/4.jpg)
www.ManwinJobs.com
Traffic In Perspective
Source: Alexa.com
Alexa global rank 95
100 Gb/s – 3 full DVDs streamed every single second
![Page 5: Building a Website to Scale to 100 Million Page Views Per Day and Beyond](https://reader034.vdocuments.mx/reader034/viewer/2022051400/54c672584a79594a538b46cb/html5/thumbnails/5.jpg)
www.ManwinJobs.com
The Context
Written in PERL with a very complex architecture
First few months dedicated to learning the site, maintain it, and plan the re-write.
Re-write started in August 2011 and was originally planned for a delivery in mid-November.
Actually launched at the end of January.
![Page 6: Building a Website to Scale to 100 Million Page Views Per Day and Beyond](https://reader034.vdocuments.mx/reader034/viewer/2022051400/54c672584a79594a538b46cb/html5/thumbnails/6.jpg)
www.ManwinJobs.com
The requirements
1
2
3
4
Support 200 million+ daily requests
100% transparent to users
Six years of legacy data
Even faster site
![Page 7: Building a Website to Scale to 100 Million Page Views Per Day and Beyond](https://reader034.vdocuments.mx/reader034/viewer/2022051400/54c672584a79594a538b46cb/html5/thumbnails/7.jpg)
www.ManwinJobs.com
The Architecture
![Page 8: Building a Website to Scale to 100 Million Page Views Per Day and Beyond](https://reader034.vdocuments.mx/reader034/viewer/2022051400/54c672584a79594a538b46cb/html5/thumbnails/8.jpg)
www.ManwinJobs.com
The Architecture
![Page 9: Building a Website to Scale to 100 Million Page Views Per Day and Beyond](https://reader034.vdocuments.mx/reader034/viewer/2022051400/54c672584a79594a538b46cb/html5/thumbnails/9.jpg)
www.ManwinJobs.com
The Architecture
Fast and reliable load-balancing.
Intelligent load distribution.
Performs health-checks
![Page 10: Building a Website to Scale to 100 Million Page Views Per Day and Beyond](https://reader034.vdocuments.mx/reader034/viewer/2022051400/54c672584a79594a538b46cb/html5/thumbnails/10.jpg)
www.ManwinJobs.com
The Architecture
![Page 11: Building a Website to Scale to 100 Million Page Views Per Day and Beyond](https://reader034.vdocuments.mx/reader034/viewer/2022051400/54c672584a79594a538b46cb/html5/thumbnails/11.jpg)
www.ManwinJobs.com
The Architecture
Reverse proxy optimized for better speed
Reduces web and database server load
Very rich and flexible configuration
![Page 12: Building a Website to Scale to 100 Million Page Views Per Day and Beyond](https://reader034.vdocuments.mx/reader034/viewer/2022051400/54c672584a79594a538b46cb/html5/thumbnails/12.jpg)
www.ManwinJobs.com
The Architecture
Cache management (what, for how long)
Edge Side Includes (ESIs)
Health check on Web servers
![Page 13: Building a Website to Scale to 100 Million Page Views Per Day and Beyond](https://reader034.vdocuments.mx/reader034/viewer/2022051400/54c672584a79594a538b46cb/html5/thumbnails/13.jpg)
www.ManwinJobs.com
The Architecture
![Page 14: Building a Website to Scale to 100 Million Page Views Per Day and Beyond](https://reader034.vdocuments.mx/reader034/viewer/2022051400/54c672584a79594a538b46cb/html5/thumbnails/14.jpg)
www.ManwinJobs.com
The Architecture
Custom logging of page views
Used for tasks like view counters or related videos
Between 8GB and 15GB of logs per hour!
![Page 15: Building a Website to Scale to 100 Million Page Views Per Day and Beyond](https://reader034.vdocuments.mx/reader034/viewer/2022051400/54c672584a79594a538b46cb/html5/thumbnails/15.jpg)
www.ManwinJobs.com
The Architecture
![Page 16: Building a Website to Scale to 100 Million Page Views Per Day and Beyond](https://reader034.vdocuments.mx/reader034/viewer/2022051400/54c672584a79594a538b46cb/html5/thumbnails/16.jpg)
www.ManwinJobs.com
The Architecture
High-performance HTTP server.
PHP-FPM
External CDNs for Static files like CSS, images and JS
![Page 17: Building a Website to Scale to 100 Million Page Views Per Day and Beyond](https://reader034.vdocuments.mx/reader034/viewer/2022051400/54c672584a79594a538b46cb/html5/thumbnails/17.jpg)
www.ManwinJobs.com
The Architecture
![Page 18: Building a Website to Scale to 100 Million Page Views Per Day and Beyond](https://reader034.vdocuments.mx/reader034/viewer/2022051400/54c672584a79594a538b46cb/html5/thumbnails/18.jpg)
www.ManwinJobs.com
The Architecture
FPM hosts our framework of choice: Symfony2.
Fast and feature rich.
A wealth of bundles already available.
![Page 19: Building a Website to Scale to 100 Million Page Views Per Day and Beyond](https://reader034.vdocuments.mx/reader034/viewer/2022051400/54c672584a79594a538b46cb/html5/thumbnails/19.jpg)
www.ManwinJobs.com
The Architecture
![Page 20: Building a Website to Scale to 100 Million Page Views Per Day and Beyond](https://reader034.vdocuments.mx/reader034/viewer/2022051400/54c672584a79594a538b46cb/html5/thumbnails/20.jpg)
www.ManwinJobs.com
The Architecture
A messaging component
Designed for large scale deployments
ActiveMQ to do writes (MySQL and Redis)
![Page 21: Building a Website to Scale to 100 Million Page Views Per Day and Beyond](https://reader034.vdocuments.mx/reader034/viewer/2022051400/54c672584a79594a538b46cb/html5/thumbnails/21.jpg)
www.ManwinJobs.com
The Architecture
Partially implemented with mitigated results.
Too rigid for a site requiring constant changes.
Gains not justifying Java and a separate infrastructure.
![Page 22: Building a Website to Scale to 100 Million Page Views Per Day and Beyond](https://reader034.vdocuments.mx/reader034/viewer/2022051400/54c672584a79594a538b46cb/html5/thumbnails/22.jpg)
www.ManwinJobs.com
The Architecture
![Page 23: Building a Website to Scale to 100 Million Page Views Per Day and Beyond](https://reader034.vdocuments.mx/reader034/viewer/2022051400/54c672584a79594a538b46cb/html5/thumbnails/23.jpg)
www.ManwinJobs.com
The Architecture
Ability to manage pools of servers with health checks. We maintain 2 pools:
Write pool with fail-over to backup-Master. Read pool with all servers except Master.
![Page 24: Building a Website to Scale to 100 Million Page Views Per Day and Beyond](https://reader034.vdocuments.mx/reader034/viewer/2022051400/54c672584a79594a538b46cb/html5/thumbnails/24.jpg)
www.ManwinJobs.com
The Architecture
![Page 25: Building a Website to Scale to 100 Million Page Views Per Day and Beyond](https://reader034.vdocuments.mx/reader034/viewer/2022051400/54c672584a79594a538b46cb/html5/thumbnails/25.jpg)
www.ManwinJobs.com
The Architecture
Open source, advanced key-value store
Read operations on Redis are FAST
Primary data source
![Page 26: Building a Website to Scale to 100 Million Page Views Per Day and Beyond](https://reader034.vdocuments.mx/reader034/viewer/2022051400/54c672584a79594a538b46cb/html5/thumbnails/26.jpg)
www.ManwinJobs.com
The Architecture
Updated in real time as with MySQL.
Redis Sorted Sets for all lists.
Pipelining is VERY important for performance.
![Page 27: Building a Website to Scale to 100 Million Page Views Per Day and Beyond](https://reader034.vdocuments.mx/reader034/viewer/2022051400/54c672584a79594a538b46cb/html5/thumbnails/27.jpg)
www.ManwinJobs.com
The Architecture
Persistence needs tuning.
RDB does a snapshot but is very IO extensive.
AOF does incremental backups and is IO pain-free.
![Page 28: Building a Website to Scale to 100 Million Page Views Per Day and Beyond](https://reader034.vdocuments.mx/reader034/viewer/2022051400/54c672584a79594a538b46cb/html5/thumbnails/28.jpg)
www.ManwinJobs.com
The Architecture
![Page 29: Building a Website to Scale to 100 Million Page Views Per Day and Beyond](https://reader034.vdocuments.mx/reader034/viewer/2022051400/54c672584a79594a538b46cb/html5/thumbnails/29.jpg)
www.ManwinJobs.com
The Architecture
Very normalized database since not used directly for site.
Some tables have over 100 million rows.
Used to populate Redis lists for new features
![Page 30: Building a Website to Scale to 100 Million Page Views Per Day and Beyond](https://reader034.vdocuments.mx/reader034/viewer/2022051400/54c672584a79594a538b46cb/html5/thumbnails/30.jpg)
www.ManwinJobs.com
The good and the bad
Main reasons for the delays:
Decisions concerning some of the technologies to use.
Learning curve for new technologies longer than expected.
Data transfer and restructuring in MySQL and Redis
Staffing issues.
![Page 31: Building a Website to Scale to 100 Million Page Views Per Day and Beyond](https://reader034.vdocuments.mx/reader034/viewer/2022051400/54c672584a79594a538b46cb/html5/thumbnails/31.jpg)
www.ManwinJobs.com
The good and the bad
Was it a success?
Launch without any downtime
New site about 10% faster
Valuable expertise gained
A GOOD SUCCESS STORY WITH LESSONS LEARNED