scala + akka + ning/async-http-client - vancouver scala meetup february 2015
Post on 15-Jul-2015
Embed Size (px)
Scala + Akka+ning/async-http-clientOr: How I stopped worrying about network latency and drank the reactive cool-aid
TODO: add owly loves scala picIntro$ whoami-Who I am-What does Hootsuite do-What the Publishing team does-Mention move to service-oriented/microservices architecture-Scala+Akka at Hootsuite(not sure I need some slides for this material)
RSS is dead, long live RSS!In constant decline over the years as consumer-facing techStill very popular between content servicesRSS/Atom feeds @ HSFollow feeds on behalf of usersPost to social networks based on content ~250K active feeds, ~400K messages daily
-So, what am I going to talk about? RSS/Atom feeds?-XML? Is this guy for real?-Constant decline over the years as consumer-facing technology (graph)-But still used a great deal between "content services"-How are RSS/Atom feeds used at Hootsuite?-Webfeeds at Hootsuite; how much?
Scaling webfeeds using PHP
-Show a rough sketch of php+gearman+cron-Latency, latency, latency everywhere!But latency isnt the problem; blocking is.Talk about the example of mailing a letter or getting your car/bike fixedWaiting around for something that someone else is doing to be finished is simply wasteful.Enter Scala + Akka + async HTTP clientAkka: a short detour for the uninitiatedToolkit and runtime for building highly concurrent, distributed, and fault tolerant applications on the JVMImplementation of the Actor modelScala, JavaAdheres strictly to the Reactive Manifesto:Responsive, Resilient, Elastic, Message-driven
Source: http://www.scottlogic.com/blog/rdoyle/assets/ActorModel.pngactors (encapsulate state and behaviour - true encapsulation)mailboxesimmutable messagessupervisory hierarchyconcurrency and distribution is handled by the framework
Nings async-http-clientNings async-http-client-library written in Java but some Scala wrappers exist-we use Plays wrapper for convenience but access the underlying ning client for greater control of the configuration-easy to configure-returns a Future to map, compose, etc. (a familiar construct to Scala hackers)
in an actorCommon pattern for handling futures in Akka actors.Actors in Scala webfeed service
So, what did this buy us?
Before120 processes6 X m3.xlarge2 processes2 X m3.xlarge-Graphs show rough representation of throughput in # of message sent by units of time-Greater throughput with less computational capacity
90 webfeed/sec300 webfeed/sec-Number of webfeeds processed per second.-This is after throttling the service because of the pressure it imposed on other parts of the system
CPU utilization & load average - barely breaking a sweat!Some lessons learnedYou can't ignore back pressurePull-based approachAkka Streams to the rescue?scala.xml.pull.XMLEventReader is not production readyBe weary of the resulting load on the rest of your infrastructure!
Some lessons learned (continued)Actors should only ever do one thing (and do it well)Easy to modify the actor topology (and it will happen!)Easier to describe supervisors responsibilitiesLet it crash!Thanks!http://akka.io/https://github.com/AsyncHttpClient/async-http-client
Were hiring! https://hootsuite.com/about/careers