WEBINAR

Understanding Akka Streams, Back Pressure and Asynchronous Architectures

by Konrad Malawski (@ktosopl)

Konrad `ktoso` Malawski

Akka Team, Reactive Streams TCK,

Persistence, HTTP

Konrad `@ktosopl` Malawski

akka.iotypesafe.comgeecon.org

Java.pl / KrakowScala.plsckrk.com

GDGKrakow.pl lambdakrk.pl

“Stream”

“Stream” What does it mean?!

Akka Streams

Akka Streams && Reactive Streams

Why back-pressure?

?

Why back-pressure?

So you’ve built your app and it’s awesome.

Why back-pressure?

Let’s not smash it horribly under load.

What is back-pressure?

?

What is back-pressure?

No no no…! Not THAT Back-pressure!

No no no…! Not THAT Back-pressure!

What is back-pressure?

Publisher[T] Subscriber[T]

Back-pressure explained

Fast Publisher Slow Subscriber

Push model

Subscriber usually has some kind of buffer.

Push model

Push model

Push model

What if the buffer overflows?

Push model

Use bounded buffer, drop messages + require re-sending

Push model

Kernel does this!Routers do this!

(TCP)

Use bounded buffer, drop messages + require re-sending

Push model

Increase buffer size… Well, while you have memory available!

Push model

Push model

DEMO

Reactive Streams explained

Reactive Streams explained in 1 slide

Fast Publisher will send at-most 3 elements. This is pull-based-backpressure.

Reactive Streams: “dynamic push/pull”

JEP-266 – soon…!public final class Flow { private Flow() {} // uninstantiable

@FunctionalInterface public static interface Publisher<T> { public void subscribe(Subscriber<? super T> subscriber); }

public static interface Subscriber<T> { public void onSubscribe(Subscription subscription); public void onNext(T item); public void onError(Throwable throwable); public void onComplete(); }

public static interface Subscription { public void request(long n); public void cancel(); }

public static interface Processor<T,R> extends Subscriber<T>, Publisher<R> { }}

Reactive Streams: goals

1) Avoiding unbounded buffering across async boundaries

2)Inter-op interfaces between various libraries

Reactive Streams: goals1) Avoiding unbounded buffering across async boundaries

2) Inter-op interfaces between various libraries

Argh, implementing a correct RS Publisher or Subscriber is so hard!

Reactive Streams: goals1) Avoiding unbounded buffering across async boundaries

2) Inter-op interfaces between various libraries

Argh, implementing a correct RS Publisher or Subscriber is so hard!

Reactive Streams: goals1) Avoiding unbounded buffering across async boundaries

2) Inter-op interfaces between various libraries

Argh, implementing a correct RS Publisher or Subscriber is so hard!

You should be using Akka Streams abstractions instead!

Akka Streams

Streams complement Actors, they do not replace them.

Actors – distribution (location transparency) Streams – back-pressured + more rigid-blueprint

Akka is a Toolkit, pick the right tools for the job.

Runar’s excellent talk @ Scala.World 2015

Asynchronous processing toolbox:

Power

Constraints

Akka is a Toolkit, pick the right tools for the job.

Asynchronous processing toolbox:

Constraints

Power

Akka is a Toolkit, pick the right tools for the job.

Single value, no streaming by definition.Local abstraction. Execution contexts.

Power

Constraints

Akka is a Toolkit, pick the right tools for the job.

Mostly static processing layouts.Well typed and Back-pressured!

Constraints

Power

Akka is a Toolkit, pick the right tools for the job.

Plain Actor’s younger brother, experimental.Location transparent, well typed.Technically unconstrained in actions performed

Constraints

Power

Akka is a Toolkit, pick the right tools for the job.

Runar’s excellent talk @ Scala.World 2015

Location transparent. Various resilience mechanisms.

(watching, persistent recovering, migration, pools)

Untyped and unconstrained in actions performed.

Constraints

Power

Akka Streams

streams

Akka Streams in 20 seconds: // types: Source[Out, Mat] Flow[In, Out, Mat] Sink[In, Mat]

// generally speaking, it's always: val ready = Source(???).via(flow).map(_ * 2).to(sink)

val mat: Mat = ready.run() // the usual example: val f: Future[String] = Source.single(1).map(_.toString).runWith(Sink.head)

Proper static typing!

Akka Streams in 20 seconds:

// types: _Source[Int, Unit] Flow[Int, String, Unit] Sink[String, Future[String]]

Source.single(1).map(_.toString).runWith(Sink.head)

Akka Streams in 20 seconds:

// types: _Source[Int, Unit] Flow[Int, String, Unit] Sink[String, Future[String]]

Source.single(1).map(_.toString).runWith(Sink.head)

Materialization

Gears from GeeCON.org, did I mention it’s an awesome conf?

What is “materialization” really?

What is “materialization” really?

What is “materialization” really?

What is “materialization” really?

Akka Streams & HTTP

streams& HTTP

Akka HTTP

Joint effort of Spray and Akka teams. Complete HTTP Server/Client implementation.

Learns from Spray’s 3-4 years history.

Since the beginning with streaming as first class citizen.

Side note: Lagom also utilises Akka Streams for streaming.

Streaming in Akka HTTPDEMO

http://doc.akka.io/docs/akka/2.4.7/scala/stream/stream-customize.html#graphstage-scala “Framed entity streaming” https://github.com/akka/akka/pull/20778

HttpServer as a: Flow[HttpRequest, HttpResponse]

Streaming in Akka HTTPDEMO

http://doc.akka.io/docs/akka/2.4.7/scala/stream/stream-customize.html#graphstage-scala “Framed entity streaming” https://github.com/akka/akka/pull/20778

HttpServer as a: Flow[HttpRequest, HttpResponse]

HTTP Entity as a: Source[ByteString, _]

Streaming in Akka HTTPDEMO

http://doc.akka.io/docs/akka/2.4.7/scala/stream/stream-customize.html#graphstage-scala “Framed entity streaming” https://github.com/akka/akka/pull/20778

HttpServer as a: Flow[HttpRequest, HttpResponse]

HTTP Entity as a: Source[ByteString, _]

Websocket connection as a: Flow[ws.Message, ws.Message]

Persistence Query (experimental)

“The Query Side” of Akka Persistence

Persistence Query (experimental)

Persistence Query Journals

akka/akka-persistence-cassandra 0.16 akka/akka @ leveldb-journal 2.4.8

dnvriend/akka-persistence-jdbc 2.3.3 scullxbones/akka-persistence-mongo 1.2.5

…and more, that I likely forgot about.

Implementation of “data-pump” pattern.

Akka + Kafka = BFF

Reactive Kafka

+

Started by Krzysiek Ciesielski & Adam Warski @ SofwareMill.com

“ACKnowladged streams”

happy ACKing!

Kafka + Akka = BFF

Akka is Arbitrary processing.

Kafka is somewhat more than a message queue, but very focused on “the log”.

Spark shines with it’s data-science focus.

Kafka + Akka = BFF

Kafka + Akka = BFF

Streams talking to Actors &&

Actors talking to Streams

Streams <=> Actors inter-op

Source.actorRef (no back-pressure) Source.queue (safe)

Sink.actorRef (no back-pressure) Sink.actorRefWithAck (safe)

Exciting times ahead!

Next steps for Akka

Completely new Akka Remoting (goal: 1M+ msg/s (!)),(it is built using Akka Streams).

More integrations for Akka Streams stages, also dynamic fan-in/out A.K.A. “the Hub”.

Reactive Kafka polishing and stable release with SoftwareMill.

“Confirmed Streams” work from Reactive Kafka generalised.

Akka Typed likely to progress again.

Of course, continued maintenance of Cluster and others.

Upgrade your grey matterTwo free O’Reilly eBooks by Lightbend

DOWNLOADNOWDOWNLOADNOW

lightbend.com/pov

Reactive RoundtableWorld Tour by Lightbend

lightbend.com/reactive-roundtable

Proof of Value ServiceAccelerate Project Success

Q/A

ktoso @ lightbend.com twitter: ktosopl

github: ktosoteam blog: letitcrash.com

home: akka.io