messaging is not just for investment banks!

53
Messaging is not just for investment banks! (+ web is not the only way) Gojko Adzic http://gojko.net [email protected] twitter.com/gojkoadzic

Upload: elliando-dias

Post on 14-Dec-2014

591 views

Category:

Technology


2 download

DESCRIPTION

 

TRANSCRIPT

Page 1: Messaging is not just for investment banks!

Messaging is not just for investment banks!

(+ web is not the only way)

Gojko Adzichttp://gojko.net

[email protected]/gojkoadzic

Page 2: Messaging is not just for investment banks!

Web is now ubiquitous

... we use it to display content, to share data, for remote procedure calls, to integrate systems...

Page 3: Messaging is not just for investment banks!

It started like this...

Page 4: Messaging is not just for investment banks!

And simply exploded...

Page 5: Messaging is not just for investment banks!

But why this???

Page 6: Messaging is not just for investment banks!

HTTP distribution/integration Easy to use Cross-platform Clustering and load balancing Almost never blocked by firewalls Even then works with proxying Stateless Synchronous Unreliable/Non transactional High latency

Page 7: Messaging is not just for investment banks!

Web technologies are not for everything...

... sometimes other stuff can save a lot of efforthttp://www.flickr.com/photos/33453508@N02/

Page 8: Messaging is not just for investment banks!

HTTP distribution/integration Easy to use Cross-platform Clustering and load balancing Almost never a security problem Even then works with proxying Stateless Synchronous Unreliable/Non transactional High Latency

Page 9: Messaging is not just for investment banks!

Messaging

Application integration pattern Data transformation, routing, resilience, high

performance, high throughput Message oriented middleware(MOM)

Page 10: Messaging is not just for investment banks!

http://www.flickr.com/photos/wirenine/

Page 11: Messaging is not just for investment banks!

http://www.flickr.com/photos/17675967@N02/

Page 12: Messaging is not just for investment banks!
Page 13: Messaging is not just for investment banks!

http://www.flickr.com/photos/stewf/

Page 14: Messaging is not just for investment banks!

Messaging

Application integration pattern Data transformation, routing, resilience, high

performance, high throughput Message oriented middleware(MOM) Event driven processing Split workflows into several asynchronous parts Share data instead of functionallity

− But use data to invoke actions!

Page 15: Messaging is not just for investment banks!

Not just for multi-billion enterprises

Some really lightweight solutions out there

Can even run everything on a single machine

Typical web sites can benefit from this approach as well

Page 16: Messaging is not just for investment banks!

Benefits

Better isolation− Easier scaling− Better performance

Resilience Responsiveness Better resource usage

Page 17: Messaging is not just for investment banks!

Case Study #1: E-mail after registration/order

Page 18: Messaging is not just for investment banks!

Case Study #1: E-mail after registration/order

Page 19: Messaging is not just for investment banks!

Case Study #1: E-mail after registration/order

Page 20: Messaging is not just for investment banks!

Case Study #1: E-mail after registration/order

Page 21: Messaging is not just for investment banks!

Problems

DNS/Networking issues SMTP rules (external verification, spam filters) How do we test this? How do we guarantee that the message gets

through?− What if it doesn't

What if the DB rolls back?

Page 22: Messaging is not just for investment banks!

An alternative approach

Page 23: Messaging is not just for investment banks!

An alternative approach

Page 24: Messaging is not just for investment banks!

An alternative approach

Page 25: Messaging is not just for investment banks!

An alternative approach

Page 26: Messaging is not just for investment banks!

An alternative approach

Page 27: Messaging is not just for investment banks!

Why is this better?

External system problems don't affect user registration

SMTP rules don't slow it down If e-mail sending fails, we can easily re-send

later

Page 28: Messaging is not just for investment banks!

Key Difference:

The first part of the process succeeds without waiting for the second

- but the second is guaranteed to happen

Page 29: Messaging is not just for investment banks!

Transactional guarantee

Page 30: Messaging is not just for investment banks!

How do we test this?

Mock queue/In memory implementation Process registration and look at the queue

contents Easily unit testable Focuses the test on what is really important

Page 31: Messaging is not just for investment banks!

Publish/Subscribe (Fire & Forget)

http://www.sxc.hu/photo/1084274

Page 32: Messaging is not just for investment banks!

Publish/Subscribe (Fire & Forget)

When you need to talk to external systems but don't really need to wait for them to finish

Batch/overnight processing Decouple processes so that they can be

performed asynchronously Effectively observer over messaging Option to have multiple subscribers (observers)

Page 33: Messaging is not just for investment banks!

Case study #2: Credit card processing

Page 34: Messaging is not just for investment banks!

Case study #2: Credit card processing

Page 35: Messaging is not just for investment banks!

Case study #2: Credit card processing

Page 36: Messaging is not just for investment banks!

Case study #2: Credit card processing

Page 37: Messaging is not just for investment banks!

What could possibly go wrong?

User closes the window mid-way User clicks on refresh Web call times out CC channels too busy/RPC times out Order processing fails after authorisation

Page 38: Messaging is not just for investment banks!

On top of that, we're wasting web/db resources!

Page 39: Messaging is not just for investment banks!

Alternative solution

Page 40: Messaging is not just for investment banks!

Alternative solution

Page 41: Messaging is not just for investment banks!

Alternative solution

Page 42: Messaging is not just for investment banks!

Alternative solution

Page 43: Messaging is not just for investment banks!

Alternative solution

Page 44: Messaging is not just for investment banks!

Alternative solution

Page 45: Messaging is not just for investment banks!

Alternative solution

Page 46: Messaging is not just for investment banks!

Alternative solution

Page 47: Messaging is not just for investment banks!

Alternative solution

Page 48: Messaging is not just for investment banks!

Alternative solution

Page 49: Messaging is not just for investment banks!

What's improved?

Closing the window makes no difference Refresh makes no difference Web call will not time out We can wait for CC channels Web and DB resources used much better

Page 50: Messaging is not just for investment banks!

Some ways to improve this...

Enqueue operation result, authorise order asynchronously (increase resilience)

Scale to more servers Process cards using dedicated servers (VLAN) Avoid polling, send a message to the client

Page 51: Messaging is not just for investment banks!

Some other situations where messaging might come in handy

Distributing work across multiple machines Performing a number of actions when

something happens (eg notify admin, notify customer)

Pushing frequent updates to clients

Page 52: Messaging is not just for investment banks!

Tools

ActiveMQ: http://activemq.apache.org/ NServiceBus: http://www.nservicebus.com/ AMQP: http://amqp.org

Page 53: Messaging is not just for investment banks!

Bridging the Communication Gap

learn how to improve communication between business people and software implementation teams

find out how to build a shared and consistent understanding of the domain in your team

learn how to apply agile acceptance testing to produce software genuinely fit for purpose

discover how agile acceptance testing affects your work whether you are a programmer, business analyst or a tester

learn how to build in quality into software projects from the start, rather than control it later

http://www.acceptancetesting.info