S

Y

N

C

H

R

O

N

O

U

S

ASY

NC

HRO

NOUS

and

programming in

Łukasz Nawojczyk

contact@lukeahead.net

@LukeAheadNET

www.sproutigy.com

C10K

Handle more than 10.000 concurrent connections

What’s already in Java?

Multi-threading:

Threads

Future

ExecutorService, ScheduledExecutorService, ForkJoinPool (Java 7+)

BlockingQueue: ArrayBlockingQueue + LinkedBlockingQueue

CopyOnWriteArrayList, ConcurrentHashMap

...

Non-blocking:

NIO2 (Java 7+)

Where’s the problem

with async programming?

HARD TO DESIGN

HARD TO DEVELOP

HARD TO HANDLE ERRORS

HARD TO READ CODE

HARD TO DEBUG

HARD TO TEST

HARD TO PROFILE

HARD TO VISUALIZE

HARD TO DOCUMENT

Open Source Community does not sleep

ParSeq

RxJavaVertX

Reactor

Guava

Akka

...and much more...

Futures

Java Future (since Java 5)

boolean isDone()

V get()

V get(long timeout, TimeUnit unit)

cancel()

Guava Listenable Future

Guava Settable Future

CompletableFuture (since Java 8)

[EXAMPLE CODE]

Why not Futures?

Java Futures are straight-forward to use for a single level of asynchronousexecution but they start to add non-trivial complexity whenthey're nested (prior to Java 8 CompletableFuture).

Conditional asynchronous execution flows become difficult to optimallycompose (particularly as latencies of each request vary at runtime) usingFutures. It can be done of course, but it quickly becomes complicated (and thuserror prone) or prematurely blocks on 'Future.get()', eliminating the benefit ofasynchronous execution.

Source: Ben Christensen http://techblog.netflix.com/2013/02/rxjava-netflix-api.html

Callbacks

Simple Theory:

Task A -> Task B -> Task C

Real Implementation:

TaskC = ...

TaskB = ...

TaskA = ...

TaskA.start(

TaskB.start(

TaskC.start()

)

) Pyramid of Doom

Callback (listener) example

indexRequestBuilder.execute(new ActionListener<IndexResponse>() {

@Override

public void onResponse(IndexResponse indexResponse) {

//TODO

}

@Override

public void onFailure(Throwable throwable) {

//TODO

}

});

Elasticsearch example

MIX Future+Callbacks in reality

Elasticsearch example – ListenableActionFuture

indexRequestBuilder.execute().

boolean isDone()

IndexResponse get()

IndexResponse get(long timeout, TimeUnit unit)

addListener(ActionListener<IndexResponse> listener)Guava ListenableFuture is similar but has addCallback() instead

[EXAMPLE CODE]

Awaitility

await().atMost(5,

SECONDS).until(customerStatusIsUpdated());

https://github.com/jayway/awaitility

for testing asynchronous code

[EXAMPLE CODE]

What’s in JavaScript ?

No multithreading, but can work asynchronously

e.g. AJAX requests

Asynchronous events are based callbacks

jQuery provides promises & deferred (not good implementation)

Standard interoperable Promise A+: https://promisesaplus.com/

Multiple implementations

Promise

listener for success event

listener for failure event

listener for done event (both success and failure)

allows to specify next Promise (ability to build a chain of Promises)

! watch out for fluent chaining A.then(B).then(C)it may be A->B->C or A->[B,C] depending on implementation

Implementations:

JDeferred - https://github.com/jdeferred/jdeferred (based on jQuery)

RxJava Promises - https://github.com/darylteo/rxjava-promises (based on Promise A+)

Promise + Deferred

Promise begin() {

Deffered deferred = ...

startAsync(

aresult -> {

if (aresult.failure()) deferred.failure();

else deferred.success();

}

);

return deferred.promise();

}

Promise promise = begin();promise.done( ... );promise.fail( ... );promise.always( ... );

promise.then( doNext );

CLIENT

or use Java 8 CompletableFuture instead if you don’t like Deferred concept

[EXAMPLE CODE]

Reactor

“On a recent laptop with a dual-core processor,it's possible to process over 15,000,000 events per second with theRingBufferDispatcher and over 25,000,000 events per second in a single thread.”

reactor.on(Selectors.object("test"), new Consumer<Event<String>>() {

@Override

public void accept(Event<String> ev) {

log.info("Received event with data: " + ev.getData());

}

});

reactor.notify("test", Event.wrap("BLABLA"));

[EXAMPLE CODE]

ParSeq

Parallelization of asynchronous operations (such as IO)

Serialized execution for non-blocking computation

Code reuse via task composition

Simple error propagation and recovery

Execution tracing and visualization

https://github.com/linkedin/parseq

ParSeq (2)

final Task<String> googleContentType = getContentType("http://www.google.com");final Task<String> bingContentType = getContentType("http://www.bing.com");

final Task<String> contentTypes =Task.par(googleContentType, bingContentType)

.map("concatenate", (google, bing) -> "Google: " + google + "\n" + "Bing: " + bing + "\n");

private Task<String> getContentType(String url) {return HttpClient.get(url).task()

.map("getContentType", response -> response.getContentType());}

[EXAMPLE CODE]

RxJava

Observable<T>

Error Strategies:

onErrorReturn

onErrorResumeNext

Asynchronous Iterator

Event Iterable (pull) Subscription (push)

retrieve data T next() onNext(T)

discover error throws Exception onError(Exception)

complete returns onCompleted()

Asynchronous programming with observable streams

Akka

http://akka.io/

Actor Model

multiple actors

supervisors

Hello Librarian!

Have you read

“Java in 1 minute

for dummies“ ?

Yes, sure.

I’m too lazy to read it.

Could you summarize

this book for me?

No problem.

System.out.println(“Hello World”);

Now you know Java.

[EXAMPLE CODE]

Event-driven / Message-driven

WAIT FOR EVENT

HANDLE EVENT

EVENT QUEUE

EVENT LOOP

Vert.x

Simple concurrency model. All code is single threaded, freeing from

the hassle of multi-threaded programming.

Asynchronous programming model for writing truly scalable non-

blocking applications.

Distributed event bus that spans the client and server side. The event

bus even penetrates into in-browser JavaScript allowing to create so-

called real-time web applications.

Vert.x simple HTTP server

VertxFactory.newVertx().createHttpServer().requestHandler(req -> {

if (req.path().equals("/test")) {

req.response().end("Hello World");

}

else {

String file = req.path().equals("/") ? "index.html" : req.path();

req.response().sendFile("webroot/" + file);

}

}).listen(8080);

[EXAMPLE CODE]

Copying streams

byte[] buf = new byte[4096];

while (true) {

int r = inputStream.read(buf);

if (r == -1) break;

outputStream.write(buf, 0, r);

}

Plain Old Java way Vert.x async way

Pump.createPump(input, output).start();

or

req.response().sendFile("webroot/" + file);

ALL IN JAVA CODE

processing every 4KB!

MOSTLY HANDLED BY JVM AND OS

no blocking threads

Apache Commons IOUtils.copy()Guava ByteStreams.copy()

WARNING!Using Event-loop

for long synchronous processing will lead to:

For long processing

use dedicated workers

outside main event-loop.

VERY SLOW QUEUE

Reactive Streamshttp://ww.reactive-streams.org/https://github.com/reactive-streams/reactive-streams-jvm/

Akka

Ratpack

Reactive Rabbit

Reactor

RxJava

Slick

Vertx 3

public interface Publisher<T> {public void subscribe(Subscriber<? super T> s);

}

public interface Subscriber<T> {public void onSubscribe(Subscription s);public void onNext(T t);public void onError(Throwable t);public void onComplete();

}

public interface Subscription {public void request(long n);public void cancel();

}

backpressure

handling

Reactive Manifesto Responsive: The system responds in a timely manner if at all possible. (...)

Resilient: The system stays responsive in the face of failure. (...)

Elastic/Scalable: The system stays responsive under varying workload. (...)

Message Driven: (...) asynchronous message-passing to establish a boundary

between components that ensures loose coupling, isolation, location

transparency, and provides the means to delegate errors as messages. (...)

http://www.reactivemanifesto.org/

fail-fast / let it crash

Main targets

for asynchronous processing

Network I/O

Disk I/O

Calling external APIs

Scaling

Scale UP (vertical)

adding more resources on single machine (CPU, memory, disk space...)

Scale OUT (horizontal)

adding more machines

requires shared contract and serializable messages

Look for async-supporting libraries/frameworks that allows

not only vertical scaling but also helps with horizontal scaling.

Examples: Akka, Hazelcast, Vert.x, Atmosphere...

Hungry? Want even more?

Atmosphere

server - https://github.com/Atmosphere/atmosphere

client - https://github.com/Atmosphere/wasync

Ratpack - http://ratpack.io/

LMAX Disruptor - https://lmax-exchange.github.io/disruptor/

Hazelcast - http://hazelcast.com/

Vert.x EventBus - http://vertx.io/core_manual_java.html#event-bus-api

C10K & C10Mhttp://highscalability.com/blog/2013/5/13/the-secret-to-10-million-concurrent-connections-the-kernel-i.html

TO SUM UP...

Is asynchronous == multithreaded?

No! (not only)

Non-blocking I/O

Event Queue/Loop

Can we mix sync + async?

Sure!

SYNC or ASYNC

API

that supports

both sync and async

SYNC or ASYNC

caller (client) service

?

Enough talking

QUESTIONS?