Scala + Akka
+
ning/async-http-client
Or: How I stopped worrying about network
latency and drank the reactive cool-aid
Intro
$ whoami
RSS is dead, long live RSS!
● In constant decline over the years as
consumer-facing tech
● Still very popular between “content services”
● RSS/Atom feeds @ HSo Follow feeds on behalf of users
o Post to social networks based on content
o ~250K active feeds, ~400K messages daily
Scaling webfeeds using PHP
But latency isn’t the problem; blocking is.
Enter Scala + Akka + async HTTP client
Akka: a short detour for the uninitiated
“Toolkit and runtime for building highly
concurrent, distributed, and fault tolerant
applications on the JVM”
● Implementation of the Actor model
● Scala, Java
● Adheres strictly to the Reactive Manifesto:o Responsive, Resilient, Elastic, Message-driven
Akka (continued)
Source: http://www.scottlogic.com/blog/rdoyle/assets/ActorModel.png
Ning’s async-http-client
… in an actor
Actors in Scala webfeed service
So, what did this buy us?
Now
BeforeBefore ● 120 processes
● 6 X m3.xlarge
● 2 processes
● 2 X m3.xlarge
90 webfeed/sec 300 webfeed/sec
CPU utilization & load average - barely breaking a sweat!
Some lessons learned
● You can't ignore back pressureo Pull-based approach
o Akka Streams to the rescue?
● scala.xml.pull.XMLEventReader is not
production ready
● Be weary of the resulting load on the rest of
your infrastructure!
Some lessons learned (continued)
● Actors should only ever do one thing (and do
it well)o Easy to modify the actor topology (and it will
happen!)
o Easier to describe supervisor’s responsibilities
● Let it crash!
Thanks!
http://akka.io/
https://github.com/AsyncHttpClient/async-http-client
We’re hiring!
https://hootsuite.com/about/careers
@yanik_berube
