Connecting Akka to theRest of the World

System integration with

Akka and Apache Camel

by Martin Krasser

About me• Freelance software engineer, architect

– eHealth application integration (Java, Scala, Groovy)– Bioinformatics, Cheminformatics (C++, Java)– …

• Open source committer– Akka– Camel– Scalaz-Camel– …

• Java since 1998• Scala since 2009

• Twitter: @mrt1nz

Overview

• Introduction– Camel– Akka

• Akka-Camel integration– Consumer actors– Producer actors– Actor components

• Comparison– Camel routes– Akka-Camel routes– Scalaz-Camel routes

Apache Camel

• Integration framework written in Java– Open source, Apache 2 license– http://camel.apache.org

• Enterprise Integration Patterns (EIP)– G. Hohpe, B. Woolf

• Domain-specific language (DSL)– Internal: Java, Scala– External: XML/Spring

• Pluggable protocol and API bindings– HTTP, FTP, JMS, File … and approx. 100 more– Camel “components” http://camel.apache.org/components.html

Apache Camel

• Usage in Scala applications

– Camel Java/Scala DSL• http://camel.apache.org

– Akka-Camel integration module• http://akka.io/docs/akka-modules/1.1.2/modules/camel.html

– Scalaz-Camel DSL• https://github.com/krasserm/scalaz-camel

Apache Camel

• Architecture

Apache Camel

• Routing example

JMSEndpoint

HTTPEndpoint

Filter

D D

HTTP

http://example.org/docs

jms:queue:docs

Java DSL:

from("jms:queue:docs") .filter().xpath("/person[@name='Jon']") .to("http:example.org/docs");

Scala DSL:

"jms:queue:docs" when(_.in == "…") to("http:example.org/docs");

Apache Camel

• Messaging endpoints– Configured via endpoint URI

• "jms:queue:docs"• "http://example.org/docs"• …

– Endpoint consumer: from(uri)– Endpoint producer: to(uri)

• Message processors– Configured via fluent API

• filter().xpath("…")• transform(body().append("…"))• …

– Custom Java/Scala code (Processor interface)

…

Akka

• Platform for event-driven, scalable and fault-tolerant architectures on the JVM

• Concurrency– Actors, agents, dataflow, STM

• Scalability– Remote actors, cluster membership

• Fault tolerance– Local and remote actor supervision

CoreServices

Akka• Add-on modules

– akka-camel • Camel integration (untyped actors)

– akka-camel-typed• Camel integration (typed actors)

– akka-kernel • Microkernel

– akka-scalaz• Scalaz support

– akka-spring • Spring integration

– …

Akka• Actor model

– Mathematical model of concurrent computation– Carl Hewitt, 1973

• Akka actor– Encapsulates state– Encapsulates behavior– Exchanges messages– Mailbox– Event-driven– Single-threaded

• API– Scala– Java

Akka

• Actor model benefits– Easier to reason about

– Raised abstraction level– Easier to avoid

• Race conditions

• Dead locks

• Starvation

• Live locks

Akka Actors

• Receive messagesimport akka.actor.Actor

// actor definitionclass ExampleActor extends Actor { def receive = { case "test" => … case _ => … }}

// create actor reference (ActorRef)val actor = Actor.actorOf[ExampleActor]

// start actoractor.start

Akka Actors

• Send messages

– Fire-Forget

actor ! "test"

Akka Actors

• Send messages

– Send-And-Receive-Eventually

actor !! "test" match { case Some(result) => … // process result case None => … // timeout}

uses Future under the hood (with timeout)

Akka Actors

• Send messages

– Send-And-Receive-Future

val future = actor !!! "test"future.awaitval result = future.get

returns Future directly

Overview

• Introduction– Camel– Akka

• Akka-Camel integration– Consumer actors– Producer actors– Actor components

• Comparison– Camel routes– Akka-Camel routes– Scalaz-Camel routes

Akka-Camel Integration

• Exchange messages with Akka actors via– HTTP, FTP, JMS, File … and approx. 100 more– Any Camel component can be used (pluggable)

• Integration layer of Akka– Consumer actors: receive message from Camel endpoints– Producer actors: produce messages to Camel endpoints– Message exchange patterns (MEPs): in-out and in-only

• Leverages Camel’s asynchronous routing engine– Asynchronous completion of message exchange– No threads blocked waiting for response messages, for example

Consumer Actors

• Basic example – TCP consumer•

– Camel component: camel-mina– MEP: in-out

import akka.actor._import akka.camel._

class TcpConsumer extends Actor with Consumer { def endpointUri = "mina:tcp://localhost:6200?textline=true" def receive = { case Message(body, headers) => { self.reply("received %s" format body) } }}

Consumer Actors

• Basic example – TCP consumer activation

– Asynchronous activation of consumer endpoint

import akka.actor._import akka.camel._

// Consumer actors require a running CamelServiceCamelServiceManager.startCamelService

// Activate TCP endpoint (asynchronously)Actor.actorOf[TcpConsumer].start

Consumer Actors

• Basic example – TCP consumer activation

– Wait for endpoint activation

– Useful for testing purposes

import akka.actor._import akka.camel._

// Consumer actors require a running CamelServiceval service = CamelServiceManager.startCamelService

// Wait for TCP endpoint activationservice.awaitEndpointActivation(1) { Actor.actorOf[TcpConsumer].start}

Consumer Actors

• Basic example – TCP consumer deactivationimport akka.actor._import akka.camel._

val service: CamelService = …val tcpConsumer: ActorRef = …

// deactivate TCP endpoint (asynchronously)tcpConsumer.stop

// … or wait for TCP endpoint deactivationservice.awaitEndpointDeactivation(1) { tcpConsumer.stop}

Consumer Actors

• Basic example – HTTP consumer

– Camel component: camel-jetty– MEP: in-out– Jetty continuations used internally– Camel type converter (msg.bodyAs[…])

class HttpConsumer extends Actor with Consumer { def endpointUri = "jetty:http://localhost:8080/example" def receive = { case msg: Message => { self.reply("received %s" format msg.bodyAs[String]) } }}

Consumer Actors

• Basic example – JMS consumer

– Camel component: camel-jms

– MEP: in-only

class JMSConsumer extends Actor with Consumer { def endpointUri = "jms:queue:example" def receive = { case Message(body, headers) => ... }}

Consumer Actors

• Basic example – IMAP consumer

– Camel component: camel-mail– MEP: in-only

class MailConsumer extends Actor with Consumer { def endpointUri = "imap://admin@mymailserver.com?password=secret" def receive = { case Message(body, headers) => ... }}

Consumer Actors

• Basic example – Scheduled consumer

– Camel component: camel-quartz– MEP: in-only

class ScheduledConsumer extends Actor with Consumer { def endpointUri = "quartz://example?cron=0/2+*+*+*+*+?" def receive = { case tick => ... }}

Consumer Actors

• Acknowledgements

– Camel component: camel-file– MEP: in-only– Application-level acknowledgement (Ack)

import akka.camel.Ack...

class FileConsumer extends Actor with Consumer { override def autoack = false // default is true def endpointUri = "file:messages/in?delete=true" def receive = { case Message(body, headers) => { // ... self.reply(Ack) // delete consumed file } }}

Consumer Actors

• Failure replies

– Application-level negative acknowledgement (Failure)– Redelivery works also with JMS endpoints, for example– Failure replies can also be used with in-out MEP

import akka.camel.Failure...

class FileConsumer extends Actor with Consumer { override def autoack = false // default is true def endpointUri = "file:messages/in?delete=true" def receive = { case Message(body, headers) => { // ... self.reply(Failure(reason)) // redeliver file } }}

Consumer Actors

• Failure replies

– Should not be made within receive

– Let consumer actors crash (on Exception)

– Use a supervisor for failure replies

Consumer Actors• Supervised consumerclass SupervisedFileConsumer extends Actor with Consumer { override def autoack = false def endpointUri = "file:messages/in?delete=true" def receive = { case Message(body, headers) => { // if exception thrown: actor is restarted or stopped // else self.reply(Ack) // delete file } } override def preRestart(reason: scala.Throwable) { self.reply_?(Failure(reason)) // redeliver file } override def postStop() { self.reply_?(Failure(…)) // or Ack to delete file }}

Consumer Actors• Supervised consumer

– Restart on any Exception– 5 restart attempts within 10 seconds

import akka.actor._ import akka.config.Supervision._

// Create consumer actor referenceval consumer = Actor.actorOf[new SupervisedFileConsumer]

// Create supervisor for consumer actorval supervisor = Supervisor( SupervisorConfig( OneForOneStrategy( List(classOf[Exception]), 5, 10000), Supervise(consumer, Permanent) :: Nil))

Consumer Actors

• Simplified failure reporting– override def blocking = true

– Pros• No need for setting up a supervisor• No need to catch exception within receive• No need to self.reply(Failure(…))

– Cons• Endpoint communicates with actor via !!• Thread blocked waiting for Ack

Consumer Actors

• Simplified failure reportingclass FileConsumer extends Actor with Consumer { override def blocking = true override def autoack = false def endpointUri = "file:messages/in?delete=true" def receive = { case Message(body, headers) => { // if exception thrown: endpoint receives it // else self.reply(Ack) // delete file } }}

Typed Consumer Actors

• Basic example – HTTP/JMS consumerimport akka.actor._import akka.camel._import org.apache.camel.{Body, Header}

trait TypedConsumer { @consume("jetty:http://localhost:8080/example") // in-out def foo(s: String): String

@consume("jms:queue:example") // in-only def bar (@Body s: String, @Header("priority") p: Integer) : Unit}

class TypedConsumerImpl extends TypedActor with TypedConsumer { def foo(s: String) = "received %s" format s def bar(s: String, p: Integer) = println("received %s (priority = %d)" format(s, p))}

Typed Consumer Actors

• Annotations

– Consumer endpoints•@consume(endpointURI)

– Parameter bindings•@Body•@Header•…

Typed Consumer Actors

• Basic example – HTTP/JMS consumer activationimport akka.actor._import akka.camel._

val service = CamelServiceManager.startCamelService

// Activate HTTP and JMS endpoints (asynchronously)TypedActor.newInstance( classOf[TypedConsumer], classOf[TypedConsumerImpl])

// … or wait for HTTP and JMS endpoint activationservice.awaitEndpointActivation(2) { TypedActor.newInstance( classOf[TypedConsumer], classOf[TypedConsumerImpl]) }

Producer Actors

• Basic example – HTTP producer

– Camel component: camel-jetty

– MEP: in-out– receive inherited from Producer trait– Jetty’s async HTTP client used internally

import akka.actor._import akka.camel._

class HttpProducer extends Actor with Producer { def endpointUri = "jetty:http://localhost:8080/example"}

Producer Actors

• Basic example – HTTP producer activationimport akka.actor._import akka.camel._

// Producer actors require an initialized CamelContextCamelServiceManager.startCamelService

// activate producer actorval producer = Actor.actorOf[HttpProducer].start

// POST test message to http://localhost:8080/exampleproducer !! "test" match { case Some(m: Message) => … case Some(f: Failure) => … case None => …} // ! and !!! can also be used

Producer Actors

• Basic example – JMS producer

– Camel component: camel-jms– MEP: in-only

class JmsProducer extends Actor with Producer { def endpointUri = "jms:queue:example" override def oneway = true}

Producer Actors

• oneway = false (default)– Initiates an in-out ME with endpoint– Replies result to initial sender

• oneway = true– Initiates an in-only ME with endpoint– No reply to sender

• receiveAfterProduce– Change default reply behaviour

Producer Actors

• Custom replies

class JmsProducer extends Actor with Producer { def endpointUri = "jms:queue:example" override def oneway = true override def receiveAfterProduce = { case m: Message => self.reply("enqueue succeeded") case f: Failure => self.reply("enqueue failed") } }

Producer Actors

• Forward results

– Chaining of producer actors (pipelines)– Related: Future composition

class HttpProducer extends Actor with Producer { val target: ActorRef = … def endpointUri = "jetty:http://localhost:8080/example" override def receiveAfterProduce = { case msg => target forward msg } }

Producer Actors

• Future composition

– Producer pipeline

val producer1 = Actor.actorOf[HttpProducer1].startval producer2 = Actor.actorOf[HttpProducer2].start

// monadic future composition (non-blocking)val future = for { m1: Message <- producer1 !!! Message("test") m2: Message <- producer2 !!! m1} yield m2

// blocks until result is availableval result: Message = future.get

Actor Components

• Are Camel components– Can be used in any Camel route

• actor Camel component– Send messages to untyped actors– Provided by akka-camel module

• typed-actor Camel component– Send messages to typed actors– Provided by akka-camel-typed module– Extension of Camel’s bean component

• Used by akka-camel internally– Routes to consumer actors

Actor Components

• actor endpoint URI– actor:uuid:[<actor-uuid>][?<params>]

• Parameters– autoack: Boolean

• System or application-level acknowledgements– blocking: Boolean

• Use ! or !! for sending messages to actor

• Supported message headers– CamelActorIdentifier

• Dynamic routing to actors

Actor Components

• Example

– actor receives messages of type Message

import akka.actor._

// can be any actor (no need for Consumer)val actor = Actor.actorOf[SomeActor].start

// ...

// Camel route from JMS endpoint to actorfrom("jms:queue:example") .to("actor:uuid:%s?autoack=false" format actor.uuid)

Overview

• Introduction– Camel– Akka

• Akka-Camel integration– Consumer actors– Producer actors– Actor components

• Comparison– Camel routes– Akka-Camel routes– Scalaz-Camel routes

Comparison

• Criteria– Connectivity– Message processing– Route composition– Other

• High-level• Incomplete

Camel Routes

• Connectivity– Camel components/endpoints

• Message processing– Processor interface

• Predefined processors (all known EIPs)• Custom processors

– Concurrent execution of Processor instances– Mutable messages

Camel Routes

• Route composition– Camel DSL (Java, Scala, XML)

• Other– No built-in mechanism for distributing routes (except

via endpoints)– Distribution addressed by ServiceMix and

FuseSource Fabric, for example

Akka-Camel Routes

• Connectivity– Camel components/endpoints managed by

• Consumer actors• Producer actors

• Message processing– Actor

• Predefined (a few in akka.actor.routing)• Custom

– Sequential execution of actor instance– Immutable messages

Akka-Camel Routes

• Route composition– Wiring actors (low-level)– Future composition– …– No integration DSL (yet)

• Other– Easy to implementing stateful EIPs (aggregator,

resequencer …)– Strong built-in mechanisms for distribution, scalability

and fault-tolerance– Basis for a distributed and scalable Enterprise Service

Bus (?)

Scalaz-Camel Routes

• Connectivity– Camel components/endpoints

• Message processing– Scala functions

• Predefined (some EIPs)• Custom

– Concurrent execution– Immutable messages

Scalaz-Camel Routes

• Route composition– Based on functional programming concepts

• Message processors chained via Kleisli composition (>=>)

– Continuation-based approach• scala.Responder used as continuation monad

– Direct-style DSL

• Other – Configurable concurrency strategies– Camel processors can be re-used– Akka integration

Scalaz-Camel Routes

• Example

– Details at https://github.com/krasserm/scalaz-camel

// custom message processorval validate: MessageProcessor = ...

// Kleisli composition of route (+ implicit conversions)val vroute = validate >=> oneway >=> to("jms:queue:valid") >=> { m: Message => m.setBody("order accepted") }

// consume messages via HTTP and error handling routesfrom(jetty:http://localhost:8080/orders) attempt { vroute } fallback { case e: ValidationException => { m: Message => ... } >=> failWith(e) case _ => ...}

THANK YOU!