1 how to process thousands of requests per second a story of unexpected success july, 2015
TRANSCRIPT
1
HOW TO PROCESS THOUSANDS OF REQUESTS PER SECOND
A STORY OF UNEXPECTED SUCCESSJULY, 2015
2
FALL 2010: ...A NEW PROJECT
WITH OLD PROBLEMS. OLD CODEBASE.
THE NEW CHALLENGE.
3
FUNCTIONAL REQUIREMENTS
CLIENTS
BANK ECOSYSTEM
STORAGE / LOGS
SMS BANKING SERVICES
SMS GATESMS / USSD
4
NONE-FUNCTIONAL REQUIREMENTS
STORAGE / LOGS
SMS BANKING SERVICES
SMS GATE
PERFORMAINCE 4000+ records per sec
AVAILABILITY NONE-STOP on re-config & scale
SCALABILITY aka PLUG-AND-SCALE
RESILIENCE 10 000 000+ records as a start
CONSISTENCY Guaranteed Delivery
5
NONE-FUNCTIONAL REQUIREMENTS
PERFORMAINCE 4000+ records per sec
AVAILABILITY NONE-STOP on re-config & scale
SCALABILITY aka PLUG-AND-SCALE
RESILIENCE 10 000 000+ records as a start
CONSISTENCY Guaranteed Delivery
ON DESSERT LEGACY CODE
6
UNEXPECTED SUCCESS
7
MESSAGE QUEUING SOLUTIONS
How to process thousands of requests per second? SCALE UP, SCALE OUT…
IBM X6 Oracle SPARC
DELL PowerEdgeIntel Server System
HP BladeSystem
Microsoft Azure
AWS
Google Cloud
Rackspace
Oracle Exalogic Elastic Cloud
IBM MQ
Oracle Tuxedo
Tibco EMS / ActiveMatrix
MSMQ
OpenAMQ
RabbitMQActiveMQ
Apache Qpid
8
TASK HANDLER
“THROUGHPUT”
“REQUEST PROCESSING”
REQUESTS
QUEUE: INCOMINGARTEFACTS
SERVICES
9
ORACLE: WORK ITEMS SELECTION
QUEUE: INCOMING
WHERE ... mod (dbms_rowid.ROWID_BLOCK_NUMBER(rowid), M) = N
10
SQL SERVER: WORK ITEMS SELECTION
QUEUE: INCOMING
ID Data State
… … …
... … FREE = 0 … WHERE [State] = 0
11
SQL SERVER: WORK ITEMS SELECTION
QUEUE: INCOMING
ID Data State
… … …
... … IN PROGRESS = 1
UPDATE …SET [State] = 1WHERE [State] = 0
12
SQL SERVER: WORK ITEMS SELECTION
QUEUE: INCOMING
ID Data State
… … …
... … IN PROGRESS = 1
... ... IN PROGRESS = 1
... … FREE = 0
UPDATE TOP (...)SET [State] = 1WHERE [State] = 0OUTPUT inserted.ID, inserted.Data …
13
SQL SERVER: WORK ITEMS SELECTION
QUEUE: INCOMING
ID Data State
… … …
... … IN PROGRESS = 1
... ... IN PROGRESS = 1
... … FREE = 0
... … LOCKED / INSERTING = 0
... … LOCKED / INSERTING = 0
UPDATE TOP (...)WITH (READPAST)SET [State] = 1WHERE [State] = 0OUTPUT inserted.ID, inserted.Data …
14
SQL SERVER: WORK ITEMS SELECTION
UPDATE TOP (...)WITH (READPAST)SET [State] = 1WHERE [State] = 0OUTPUT inserted.ID, inserted.Data …
PREVENT LOCKS ESCALATION
#SQLSERVER
15
THREAD POOL (Simple approach)
REQUESTS
QUEUE: INCOMINGARTEFACTS
SERVICES
THREAD POOLSERVICE API
WORKER THREAD
16
MASTER WORKER #1/4CACHE
NGINX (God-like approach)
REQUESTS
Sockets (OS Kernel)
ARTEFACTS
SERVICES
17
LEGACY: WIN SERVICELEGACY: LOCKS
RESOLUTION
…our unexpected success -)
REQUESTS
QUEUE: INCOMING
SQL Server
PARSING
WORKER #1/4 M, N
18
LEGACY: LOCKS RESOLUTION
DEMO
QUEUE: INCOMING
WORKER #1/4 M, N
SQL Server