Download - SQS ingress for AWS Lambda
for
AWS LAMBDAAWS LAMBDA
SQS ingressSQS ingress
hi, I’m Yan Cui
AWS user since 2009
can I haz SQS as ingress source?
DO ITwe canDO IT
ourselves!
retried 3 times
dead letter queue
configurable
dead letter queue
retried 3 times
dead letter queue
one invocation per message
configurable
dead letter queue
no. of pollers
retried 3 times
dead letter queue
one invocation per message
highest parallelism!
configurable
dead letter queue
no. of pollers
retried 3 times
dead letter queue
one invocation per message
consider impact on downstream systems consider Lambda throttling
which is per region, so otherfunctions can get throttled too
configurable
dead letter queue
no. of pollers
retried until success/expiry
NO dead letter queue
configurable
dead letter queue
retried until success/expiry
NO dead letter queue
configurable
dead letter queue
retried until success/expiry
NO dead letter queue
one invocation per shard
configurable
dead letter queue
no. of pollers
retried until success/expiry
NO dead letter queue
one invocation per shard
charged by shard hour + reqs
configurable
dead letter queue
no. of pollers
charged by reqs
retried until success/expiry configurable
NO dead letter queue dead letter queue
one invocation per shard no. of pollers
charged by shard hour + reqs charged by reqs
high baseline cost (1 shard) butorder of magnitude cheaper at scale
1 msg/s for a month (1K msgs)
1 x 60s x 60m x 24hr x 30days @ $0.4 per mil
= $1.0368
1 x 60s x 60m x 24hr x 30days @ $0.014 per mil+ 24hrs x 30days x $0.015 per hr
= $10.836
1000 msg/s for a month (1K msgs)
1000 x 60s x 60m x 24hr x 30days @ $0.4 per mil
= $1036.80
1000 x 60s x 60m x 24hr x 30days @ $0.014 per mil+ 24hrs x 30days x $0.015 per hr
= $47.088
“none of them is the right choice
for every situation”
SQS is right choice sometimes, no?
“none of them is the right choice
for every situation”
cron jobslevel 1
every X mins
simple cheap
simple cheap
good for predictable traffic
unpredictable traffic leads to unpredictable latency
unpredictable traffic leads to unpredictable latency
manual scaling
recursive functions
level 2
poll (long polling)
process
recurse
POLL
long polling
PROCESS
RECURSE
predictable latency even when traffic is unpredictable
unhandled exception kills loop
unhandled exception kills loop
cost
autoscalinglevel 3
poll (long polling)
process
continue?
recurseyes
stopno
token last_heartbeat
my_loop_1 2017-05-17 20:47:32.142
my_loop_2 2017-05-17 20:47:29.923
… …
passed to a recursive function at the start
every X mins
look for dead loops
every X mins
look for dead loops
recurse
start
too many messages
too many messages
add token
too many messagesrecurse
start
add token
DE moDE mo
show me the
@theburningmonktheburningmonk.comgithub.com/theburningmonk
Process SQS with Lambda
part 1 : http://bit.ly/2rvXYdW
part 2 : http://bit.ly/2pY0Grj
QUESTIONS?
