websocket microservice vs. rest microservice

Download WebSocket MicroService vs. REST Microservice

If you can't read please download the document

Post on 15-Apr-2017




2 download

Embed Size (px)


Rick Hightower - Mammatus Technology Inc.WebSocket MicroService vs. REST MicroserviceHow does the performance of a REST microservice compare to a WebSocket MicorService?

Comparison of SpeedWebSocket is faster and uses less resources for more callsREST is slower due to each connection needing to wait for a response before it can send another request

REST/HTTP vs WebSocketWith REST you have to wait for a reply before you can send another request (over the same connection)Unless you use pipelining which seems to work better in benchmarks than the real worldWith WebSocket you can stream requests and stream responses and then marry them togetherStill request/reply but WebSocket can send responses while it is getting other requests

Example uses QBitQBit supports REST and WebSocket RPCQBit allows you to stream requests over the wire and receive responses over the wire. It employs micro-batching to batch requests and responses into larger chunks to minimize IO and thread hand-off costsQBit allows async REST calls and async WebSocket calls. (As well as internal services based on queues)

wrk Testing tool for REST/HTTPTo perf test REST code we use wrk wrk load tester capable of generating significant load when run on a single multi-core CPUwrk load tester uses multithreaded and event notification systems epoll and kqueue wrk maximizes the number of HTTP requests per second

Trade Service

wrk LUA script

Caveats About the TestWe can tweak server a bit and reduce the flush rate or reduce the batch size to get higher throughput with lower connectionsWe can also tweak the OS so we can have more ephemeral ports available and then use a lot more connectionsexperience tells me that we can get close to 90K TPS or so on a MacBook Pro We could test from two machines one of those machines being a Linux server with a tweaked stackThis test has disadvantage of all being run on same machinebut same disadvantage that the WebSocket version will have so it is somewhat fairWe could also employ HTTP pipelining to increase the throughput, great trick for benchmarks rarely works in production environments with real clientsOn commodity Linux server, we can get close to 150K TPS to 200K TPS from experience

WebSocket TestThis will be a bit harderNo tool like wrk to test WebSocket RPC calls so we will write our ownRPC calls are QBit centric and rely on JSON plus some custom ASCII protocol Some overhead in the protocol because we have to match responses to clients and requests need unique identifier (as well as the client needing a unique identifier)

To use WebSocket we will need Async interface to service

WebSocket PERF Test code

WebSocket Perf code continued

How does the WebSocket microservice do?

1 Million messages a second.

700K messages for a single WebSocket connection.

7.5x better than REST version.

OverheadRunning code through perf showed it was waiting for IO a lot. Better IO. Better NIC card. More TCP/IP stack tuning. You can expect better throughput. QBit has to marry requests with responses/client id combinations. There is overhead in doing this. QBit also uses JSON/ASCII protocol which could be optimized to reduce IO (gzip it or use message pack).Custom more efficient streaming can be implemented if needed (from experience, this is another 2x to 4x)But in general, WebSocket will do better than HTTP

More CaveatsWhen using WebSocket, you also need service discovery, and you need to implement your own failover You cant just throw a bunch of microservices behind a load balancerQBit integrates with DNS and consul.io to provide service discovery (also allows you to use push)

Follow upThis slide deck as a blog post: http://rick-hightower.blogspot.com/2015/12/websocket-microservice-vs-rest.htmlThe QBit project: https://github.com/advantageous/qbitMammatus Technologies http://www.mammatustech.com/Rick Hightowerhttps://twitter.com/RickHighhttps://www.linkedin.com/in/rickhigh

There is a lot more to QBit than this

QBit supports reactive stream over WebSocket

QBit implements Service Actors

Check out QBit