kill the fail whale for your api
TRANSCRIPT
KILL THE
FAIL
WHALE
FOR
YOUR
API@MarkCheshire
@3scale
#APISBERLIN
25 April, 2015
@MarkCheshire @3scale #apisberlin
Unmatched Flexibility and Scale
3SCALE API MANAGEMENT DESIGNED FOR
SPEED AND RELIABILITY
2
@MarkCheshire @3scale #apisberlin
Unmatched Flexibility and Scale3
Various – Data-as-a-Service, Business Service, etc
Energy Telecom – Include Tier1 Telco Op. Infrastructure
Retail / E-Commerce
Media Health / Nutrition
Transport GVT. / Administration Education Music Finance
Charities
600 APIS POWERED BY 3SCALE
@MarkCheshire @3scale #apisberlin
Unmatched Flexibility and Scale
APIS ARE CRITICAL INFRASTRUCTURE
4
@MarkCheshire @3scale #apisberlin
Unmatched Flexibility and Scale
FAIL WHALE WAS A BIG DEAL FOR TWITTER
AVOID THE SAME FATE FOR YOUR API
5
@MarkCheshire @3scale #apisberlin
Unmatched Flexibility and Scale
UNDERSTAND PERFORMANCE EXPECTATIONS FOR YOUR API
• Average throughput in requests per second
• Peak throughput
(what is the most traffic that you get over a certain period?)
• Throughput distribution
– by API endpoint
(do you have any endpoint that gets substantially more traffic than any
others?)
– by users
(a few generate most traffic or is it more evenly distributed)
6
@MarkCheshire @3scale #apisberlin
Unmatched Flexibility and Scale
D
LADDER TO LOAD TESTING NIRVANA
7
Dumb repetitive load generation
Simulated traffic patterns
Real log traffic
Live traffic
@MarkCheshire @3scale #apisberlin
Unmatched Flexibility and Scale
Tool Ideal use case
JMeter functional testing of complex user flows in a web application
wrk find what is the maximum load your API can handle
Vegeta test how an API behaves at a steady rate of X reqs/s
Loader.ioSaaS tool. Similar in target to wrk but with zero setup and nice reports
BlazemeterSaaS tool, based on JMeter (allows you to upload JMeter test plans and run them from the cloud)
LOAD TESTING TOOLS COMPARED
@MarkCheshire @3scale #apisberlin
Unmatched Flexibility and Scale
L
FIRST PASS – ESTABLISH THE BASELINE
WHAT IS THE MAX LOAD THE API CAN SUPPORT?
9
Loader.io
30 sec test10K req/sec
<40ms response time
Only occasional CPU peaks
@MarkCheshire @3scale #apisberlin
Unmatched Flexibility and Scale
WRK TO FIND THE REAL MAXIMUM THROUGHPUT
10
60 sec test16K req/secSustained ~1.4B req/day!
@MarkCheshire @3scale #apisberlin
Unmatched Flexibility and Scale
THE TEST CONFIGURATION
11
@MarkCheshire @3scale #apisberlin
Unmatched Flexibility and Scale
L
UH OH! AFTER ADDING THE GATEWAY
WE MEET THE FAIL WHALE
12
Error code 500sTimeoutsTest terminates
@MarkCheshire @3scale #apisberlin
Unmatched Flexibility and Scale
TYPICAL BOTTLENECK
MAXIMIZE THE NUMBER OF AVAILABLE SIMULTANEOUS CONNECTIONS
Increase range of ports allowed for TCP traffic. Default: [1,024 - 5,000]
(net.ipv4.ip_local_port_range)
Increase default number of available file descriptors per process (ulimit)
Make connections available for reuse as quickly as possible by reducing the
default duration of the TIME_WAIT state (net.ipv4.tcp_fin_timeout)
If your API server has concurrency problems, having Nginx (or an API Gateway) in
front and enabling keepalives can help reduce the number of connections:
http://nginx.com/blog/http-keepalives-and-web-performance/
13
@MarkCheshire @3scale #apisberlin
Unmatched Flexibility and Scale
L
REFINE AND ITERATE
UNTIL YOU REACH THE DESIRED PERFORMANCE TARGET
14
10K req/sec
<40ms response time
@MarkCheshire @3scale #apisberlin
Unmatched Flexibility and Scale
L
ALL THE BENEFITS OF AN API GATEWAY TO PROTECT YOUR API
WITH A MINIMAL PERFORMANCE HIT
15
@MarkCheshire @3scale #apisberlin
Unmatched Flexibility and Scale
BLOG POST COMING SOON AT:
WWW.3SCALE.NET
16
@MarkCheshire @3scale #apisberlin
Unmatched Flexibility and Scale
PALE ALE IS BETTER THAN THE FAIL WHALE
17
@MarkCheshire @3scale #apisberlin
Unmatched Flexibility and Scale
CREDITS
• Fail Whale aka “Lifting a Dreamer” by Yiying Lu http://www.whatisfailwhale.info/
• http://pixshark.com/people-walking-up-stairs-clipart.htm
• Fail Whale Pale Ale label by Brian Cook, Manchester, Connecticut, USA http://www.whatisfailwhale.info/
18
• http://gwan.com/en_apachebench_httperf.html
• https://www.usenix.org/legacy/publications/library/proceedings/usits97/full_papers/banga/banga.pdf
• http://www.hpl.hp.com/research/linux/httperf/wisp98/httperf.pdf
• http://making.fiftythree.com/load-testing-an-unexpected-journey
• https://engineering.gosquared.com/optimising-nginx-node-js-and-networking-for-heavy-workloads
• https://support.3scale.net/overview#how-it-works
• http://www.3scale.net/
RESOURCES