awssummit berlin day1 froehner bridging-message-brokers...summit © 2019, amazon web services, inc....

S U MM I TBe r l i n

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.SUMM IT

Bridging Message Brokers to Cloud-Native Messaging Services

Dirk FröhnerSr. Solutions ArchitectAmazon Web Services

S e s s i o n I D

Christian MüllerSr. Solutions ArchitectAmazon Web Services

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.SUMM ITSUMM IT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

“If your application is cloud-native, or large-scale, or distributed, and doesn’t include a messaging component, that’s probably a bug.”

Tim BraySenior Principal Engineer, AWS

Messaging, Eventing & Orchestration

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.SUMM ITSUMM IT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Synchronous systems are very tightly coupled

A problem in a synchronous downstream dependency has immediate impact on the upstream callers

Retries from upstream callers can all-too-easily fan-out and amplify problems.

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.SUMM IT

Agenda

Integration patterns

Integration scenarios

Whiteboarding

Photo by Camylla Battani on Unsplash

SUMM IT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T

Message exchangeOne-way Request-response

Sender Amazon MQqueue

Receiver Requester Responder

No response expected

Synchronous vs. fire-and-forget

Response expected

Return address

Correlation ID

Amazon MQqueue

Amazon MQqueue

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T

Message channelsPoint-to-point (queue) Publish-subscribe (topic)

Consumed by one receiver

Easy to scale

Flatten peak loads

Consumed by all subscribers

Durable subscriber

Sender Amazon MQqueue

Receivers

PublisherAmazon MQ

topic

Subscribers

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T

Message channelsVirtual topics

Amazon MQtopic

Amazon MQqueue

Sender

Receivers

Scale receivers per receiving application

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T

Message channelsDead letter queue (DLQ)

Transient failure mitigation

Poison pill handling

Amazon MQDLQ

XSender Amazon MQ

queueReceiver

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T

Additional patterns & conceptsFIFO

Sender ReceiverAmazon MQqueue

Guaranteed ordering

Message groups

Amazon MQqueue

Receivers

Sender

Message belongs to a group

Processed by one receiver at a time

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T

Additional patterns & conceptsMessage delivery QoS

Sender ReceiverAmazon MQqueue

At least once

At most once

Exactly once

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.SUMM IT

Exactly once? Really? Well…

“Tunable/configurable” messaging systems – choose between at least once or at most once

Offers deduplication capabilities to detect and eliminate messages which are sent more than once

to the messaging system

But how to deal with a situation, where the message was consumed but never acknowledged?

Your system has to handle duplicate messages!

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T

Additional patterns & concepts

Time

visibility timeout

receive message

receive message

receive message

receive message

Time

receive message

receive message

delete / acknowledge message

Inflight messages

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.SUMM IT

Additional patterns & conceptsProxy

Initiator PartnerProxy Initiator Partner

Proxy

Proxy

SUMM IT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.SUMM IT

Why does it matter?

• Recommended for born-in-the-cloud apps

• Simple APIs• Unlimited throughput• Pay per-request

• Recommended for application migration

• Standard APIs• Feature-rich• Low latency• Pay per-hour and per-GB

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.SUMM IT

Wait, there are more choices!

• Customer engagement platform

• Targeted email, SMS, push notifications, and voice messages.

• Pub/sub broker in AWS IoT Core

• Send messages to/fromdevices and AWS IoT appsin a secure fashion

• MQTT, HTTP, WebSockets

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.SUMM IT

… and even more!

• NRT data stream• Core entity: stream,

not events• Messages not automatically

deleted, consumer keepscursor on stream

• Not really a messagingservices, but caching solution (non persistent)

• Offers queue semantics

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T

Cloud-native to traditional / one-way

Cloud-nativeapplication

Traditionalapplication

Amazon MQqueue

ProxyAmazon SQSqueue

Cloud-nativeapplication

Traditionalapplications

Amazon MQtopic

ProxyAmazon SNStopic

Point-to-point Publish-subscribe

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T

Cloud-native to traditional / request-response

Cloud-nativeapplication

Traditionalapplication

ProxyAmazon

MQqueue

Amazon SQS

queue

Cloud-nativeapplication

Proxy

Amazon MQTopic

Amazon MQqueue

Amazon SNSTopic

Amazon SQSqueue

Traditionalapplications

Point-to-point Publish-subscribe

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T

Traditional to cloud-native / one-way

Point-to-point Publish-subscribe

Traditionalapplication

Cloud-nativeapplication

Amazon SQSqueueProxy

Amazon MQqueue Traditional

applicationCloud-nativeapplications

Amazon SNStopicProxy

Amazon MQtopic

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T

Traditional to cloud-native / request-response

Point-to-point Publish-subscribe

Traditionalapplication

Cloud-nativeapplication

Amazon SQSqueue

ProxyAmazon MQqueue

Amazon MQTopic

Amazon MQqueue

Traditionalapplication

Amazon SNSTopic

Amazon SQSqueue

Proxy

Cloud-nativeapplications

SUMM IT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.SUMM IT

Thank you!

SUMM IT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Dirk Fröhnerfroehner@amazon.de

Christian Müllercmr@amazon.de

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.SUMM ITSUMM IT © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.