Sharding and Load Balancing in Scala

Twitter's Finagle

Geoff BallingerTechnical Advisor at Mobile Acuity Ltd.

• some ML and Lisp in the early 90s• C, C++ and Java since – embedded and server• apps on BB, Android and iOS

• Mobile• Startups• Scaling• Integration• Deployment & Ops

>>> Background

>>> Finagle

https://blog.twitter.com/2011/finagle-protocol-agnostic-rpc-system

• Finagle is an extensible RPC system for Scala• Protocol-agnostic and asynchronous• Created, used and published by Twitter• Built on netty and thus NIO• Heavy use of Futures etc• http://twitter.github.io/finagle/

• Multiple HTTP workers – part of the result• Each shard is one or more identical workers• Sharder scatters requests to the shards and

gathers the results• Load balance between workers• Fault tolerant

• Chance to play w/ Scala in anger!• (Simple rational reconstruction)

>>> A simple sharder?

object Worker extends App { try { // Collect args val port = Integer.parseInt(args(0)) val value = args(1) // Define our service - just return the value provided! val service = new Service[HttpRequest, HttpResponse] {

def apply(req: HttpRequest) = { val buffer = ChannelBuffers.copiedBuffer(value,

Charset.forName("UTF-8")) val response = new DefaultHttpResponse(req.getProtocolVersion,

HttpResponseStatus.OK) response.setHeader(HttpHeaders.Names.CONTENT_TYPE, "text/plain") response.setHeader(HttpHeaders.Names.CONTENT_LENGTH,

buffer.readableBytes()) response.setContent(buffer)

Future.value(response) }

}

>>> Worker (1)

// Run this worker over the required portval server = ServerBuilder().codec(Http()).bindTo(new InetSocketAddress(port)).name("Shard").build(service)

} catch { case t: Throwable => { System.err.println(t.getMessage()) System.err.println("Usage: Worker <port> <value>") } }}

>>> Worker (2)

// Collect port argval port = Integer.parseInt(args(0))

// Build up shards from host specs on remaining command lineval shards = args.tail.map(spec => { ClientBuilder() .codec(Http()) .hosts(spec) .hostConnectionLimit(10) .retryPolicy(policy) .build()})

>>> Sharder (setup)

// Define our service - scatter to the shards and gather the resultsval service = new Service[HttpRequest, HttpResponse] { def apply(req: HttpRequest) = { Future.collect(shards.map(shard => { // Scatter shard(req).map(resp => resp.getContent().toString(CHARSET)) })) .map(resps => { // Gather val bf = ChannelBuffers.copiedBuffer(resps.reduceLeft(_+":"+_), CHARSET)

val resp = new DefaultHttpResponse(req.getProtocolVersion, HttpResponseStatus.OK)

resp.setHeader(HttpHeaders.Names.CONTENT_TYPE, "text/plain") resp.setHeader(HttpHeaders.Names.CONTENT_LENGTH, bf.readableBytes()) resp.setContent(bf)

resp }) }}

>>> Sharder (service)

>>> Demo time!

>>> Reflections and Opinions

• Concise and expressive• Shoulders of giants• Poorly documented• Exposes too many lower APIs• Twitter vs Scala vs Akka Futures!

https://github.com/geoffballinger/simple-sharder

geoff@geoffballinger.co.uk@geoffballinger

http://www.geoffballinger.co.uk

>>> Thanks for Listening!