beyond horizontal scalability: concurrency and messaging using spring

Beyond Horizontal Scalability: Concurrency & Messaging Using Spring

Bruce Snyder, Senior Software Engineer, SpringSource/VMware

Friday, July 8, 2011

Problem Statement

• Software systems are growing larger than ever before – Virtual Machines – Physical Machines– More CPUs– Multi-core CPUs– More requests– More data

• With so many hardware changes, shouldn’t our software practices change too?


Typical Application Layers

3


Typical Code Construction

• Command-and-control scheme– One method calls another– Blocking calls

• Assumptions – One task takes place at a time – The order of operations in known – The provider of a particular function is known – All execution happens in a single JVM

4


Typical Component Interaction

• Single memory space • Using the call stack • Sequential execution

• Assumptions – No separation between:

• Knowing what needs to happen next • Knowing which method to invoke

– Interactions are always known at compile time

5


Typical Assumptions

• All work must take place in a single call stack • Example:

– Receive request– Verify data– Save data – Query data– Generate PDF– Send email – Render response

6


• The higher the number of assumptions, the more tightly coupled the system

7


Typical Client Request

8


Typical Request Growth

9


Typical Strategy For Horizontal Scaling

10


Typical Assumptions

• All work must take place on a single machine • Example:

– Receive request– Verify data– Save data – Query data– Generate PDF– Send email – Render response

11


Treating the Symptoms

• Horizontal scale is good to a point – Treat the symptoms instead of fixing the cause of the illness

• Throughput of functions should not equal response time!

• This is not the case at forward-thinking companies – Amazon.com

12


The Problem With Distributed Systems

• Most developers don’t understand them – Call stack provides specific interaction style – Most object-oriented systems focus on structure vs.

interaction

• Interaction becomes more important than structure – Mediator pattern - encapsulates how objects interact – Observer pattern - notifies dependent objects of state change

• The real point of service-orientated design – Composition rules are rather simple – Loosely coupled interaction becomes very important

13


Is There a Better Solution?

• Insert a level of indirection – Remove direct interaction between components – Extract the interaction into a separate element – Enterprise Integration Patterns (EIP)

• http://enterpriseintegrationpatterns.com/

• Simplify the rules of component interaction – Remove the coordination and continuation style of interaction– Just send data

14


http://enterpriseintegrationpatterns.com


Concurrency

• Concurrency is a style of asynchronous execution

15


Concurrency With Spring

• Spring provides the TaskExecutor interface – Abstracts the execution of a Runnable, i.e., asynchronous work– SimpleAsyncTaskExecutor

• Creates a new thread for each task – ThreadPoolTaskExecutor

• Uses a JDK 1.5 ThreadPoolExecutor to create a pool of threads– org.springframework.scheduling.concurrent package

• Many scheduling related utility classes – WorkManagerTaskExecutor

• Delegates to a JCA 1.5 CommonJ WorkManager

16


JDK ExecutorService

17

ExecutorService executorService = Executors.newSingleThreadExecutor();executorService.execute(task);executorService.shutdown();

executorService = Executors.newFixedThreadPool(10);executorService.execute(task);executorService.shutdown();

int corePoolSize = 5;int maxPoolSize = 10;long keepAliveTime = 5000;executorService = new ThreadPoolExecutor(corePoolSize, maxPoolSize,

keepAliveTime, TimeUnit.MILLISECONDS, new LinkedBlockingQueue<Runnable>());

executorService.execute(task);executorService.shutdown();


Spring TaskExecutor

18

<bean id="myTask" class="org.bsnyder.spring.concurrency.MyTask" />

<bean id="springConcurrencyExample" class="org.bsnyder.spring.concurrency.SpringConcurrencyExample" /> <bean id="simpleAsyncTaskExecutor" class="org.springframework.core.task.SimpleAsyncTaskExecutor" p:daemon="false" />

<bean id="taskExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor" p:corePoolSize="5" p:maxPoolSize="10" p:queueCapacity="30" />


Spring TaskExecutor

19

public class SpringConcurrencyExample {! @Autowired! private Runnable task;! @Autowired! private SimpleAsyncTaskExecutor simpleAsyncTaskExecutor;! @Autowired! private ThreadPoolTaskExecutor threadPoolTaskExecutor;!! public void runExamples() {! ! simpleAsyncTaskExecutor.execute(task);! ! threadPoolTaskExecutor.execute(task);! }!! public static void main(String[] args) {! ! ApplicationContext context = ! ! ! new ClassPathXmlApplicationContext("/META-INF/spring/executor-context.xml",

SpringConcurrencyExample.class);! ! SpringConcurrencyExample example =

context.getBean(SpringConcurrencyExample.class);! ! example.runExamples();! }}


Messaging

• Messaging is a style of communication • Often used for integration purposes

20


Messaging With Spring

• JMS Support – JmsTemplate for sync send and receive – DefaultMessageListenerContainer and

SimpleMessageListenerContainer for async receive • javax.jms.MessageListener• org.springframework.jms.listener.SessionAwareMessageListener• org.springframework.jms.listener.adapter.MessageListenerAdapter

• AMQP Support – RabbitTemplate for sync send and receive – SimpleMessageListenerContainer for async receive

• org.springframework.amqp.core.MessageListener• org.springframework.amqp.rabbit.listener.adapter.MessageListenerAdapter

21


Spring JmsTemplate

22

<bean id="connectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory" p:brokerURL="tcp://localhost:61616" /> <bean id="destination" class="org.apache.activemq.command.ActiveMQQueue"> <constructor-arg value="FOO.TEST" /> </bean> <bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate" p:connectionFactory-ref="connectionFactory" p:defaultDestination-ref="destination" />

<bean id="messageProducer" class="org.bsnyder.spring.jms.producer.SimpleMessageProducer" p:jmsTemplate-ref="jmsTemplate" />

Synchronous


Spring JmsTemplate

23

// Use the default destination jmsTemplate.convertAndSend("Hello World!");

// Use a different destinationjmsTemplate.convertAndSend(“TEST.BAR”, “Hello World!”);

// Use a different destinationString textMessage1 = (String) jmsTemplate.receiveAndConvert();

// Use a different destinationString textMessage2 = (String) jmsTemplate.receiveAndConvert(“TEST.BAR”);

Synchronous


DefaultMessageListenerContainer

• XML configuration

24

<bean id="connectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory" p:brokerURL="tcp://localhost:61616" />

<bean id="messageListener" class="org.bsnyder.spring.jms.listener.SimpleMessageListener" />

<jms:listener-container concurrency="5-10"> <jms:listener destination="FOO.TEST" ref="messageListener"/> </jms:listener-container>

Asynchronous


Spring RabbitTemplate

25

<bean id="rabbitAdmin" class="org.springframework.amqp.rabbit.core.RabbitAdmin"> <constructor-arg ref="connectionFactory"/> </bean> <bean id="connectionFactory" class="org.springframework.amqp.rabbit.connection.SingleConnectionFactory" p:host="localhost" p:port="5672" p:username="guest" p:password="guest"/>

<bean id="rabbitTemplate" class="org.springframework.amqp.rabbit.core.RabbitTemplate" p:connectionFactory-ref="connectionFactory" p:routingKey="TEST.FOO" p:queue="TEST.FOO" />

<bean id="messageProducer" class="org.bsnyder.spring.amqp.producer.SimpleMessageProducer" p:rabbitAdmin-ref="rabbitAdmin" p:rabbitTemplate-ref="rabbitTemplate"/>

Synchronous


Spring RabbitTemplate

26

// Use the default destination rabbitTemplate.convertAndSend("Hello World!");

// Use a different destinationrabbitTemplate.send(“TEST.FOO”, “TEST.FOO”, message);

// Use a different destinationMessage message1 = rabbitTemplate.receiveAndConvert();

// Use a different destinationString textMessage2 = (String) rabbitTemplate.receive(“TEST.FOO”);

Synchronous


SimpleMessageListenerContainer

27

<bean id="connectionFactory" class="org.springframework.amqp.rabbit.connection.SingleConnectionFactory" p:host="localhost" p:port="5672" p:username="guest" p:password="guest"/>

<bean id="rabbitAdmin" class="org.springframework.amqp.rabbit.core.RabbitAdmin"> <constructor-arg ref="connectionFactory"/> </bean>

<bean id="rabbitTemplate" class="org.springframework.amqp.rabbit.core.RabbitTemplate" p:connectionFactory-ref="connectionFactory" p:routingKey="TEST.FOO" p:queue="TEST.FOO" />

<bean id="messageListener" class="org.bsnyder.spring.amqp.listener.SimpleMessageListener" p:rabbitTemplate-ref="rabbitTemplate"/> <bean id="listenerContainer" class="org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer" p:connectionFactory-ref="connectionFactory" p:queueName="TEST.FOO" p:concurrentConsumers="5" p:listener="messageListener" />

Asynchronous


Best of Both Worlds

• What if concurrency and messaging was provided together?

28


Spring Integration

• A framework for integration

• Styles of integration – Intra-application integration – Inter-application integration – External system integration

29


Enterprise Integration Patterns (EIP)

30

http://enterpriseintegrationpatterns.com/




Spring Integration

• Provides both concurrency and messaging– Message Endpoints

• Connections between services – Channel Adapters

• Adapter between application and message broker – Messaging Gateways

• Provides uni-directional or bi-directional messaging – Service Activators

• Invokes a services based on an incoming message – Routers

• Determines where to dispatch a message – Splitters and Aggregators

• Breaks up a message and reassembles it after processing

31


Spring Integration

• Supports – AMQP– Email– File system – Gemfire– JMS– JMX– MongoDB– Redis– Spring Batch – Testing – Web Services

32


Spring Integration

33


Spring Integration

34


Spring Integration

35


Spring Integration

36


Spring Integration

37


Spring Integration

38


Spring Integration

39


Spring Integration

40


Spring Integration

41


Spring Integration

42


Your Coffee Shop Does Not Use 2PC

• Order is accepted • Cup is labeled and placed in the queue • Money is exchanged• Coffee drink is processed

• Multiple baristas = competing consumers • Drinks are processed out of order = correlation id (label)• Cannot pay for drink = discard (write-off) • Erroneous drink = retry • Drink machine fails = compensating action (refund, etc.)

43


Conversation Pattern

• Interaction between two parties – Short synchronous interaction – Longer asynchronous interaction

• Other examples – Amazon.com

44


Cafe Demo

45


Q&A

Thank You!


beyond horizontal scalability: concurrency and messaging using spring

Technology

spring integration

messagelistener

typical assumptions

sync send

interaction

style

jmstemplate

execution