© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Announcing AWS Step Functions
Andy Katz, Senior Product Manager
September 2016Tim Bray, Senior Principal Engineer
Enterprise Resource Planning
Finance
Inventory
Manu-facturing
��
��
��
��
Sales
From Monoliths to Microservices to Functions
λλλ
DBMS
λ λλ λλ
λ λλ
λ
Queue
Modern Serverless app
Modern Serverless app
“I want to sequence functions”
“I want to select functions based on data”“I want to retry functions”“I want try/catch/finally”
“I have code that runs for hours”
“I want to run functions in parallel”
Functions into apps
Scales out Doesn’t lose state Deals with errors/timeouts Easy to build & operate Auditable
Coordination must-haves
AWS Step Functions…
…makes it easy to coordinate the components of distributed applications
using visual workflows.
Benefits of AWS Step Functions
Diagnose and debug problems faster
Adapt to change
Easy to connect and coordinate distributed components and
microservices to quickly create apps
Manages the operations and infrastructure of service coordination to ensure
availability at scale, and under failure
Productivity Agility Resilience
How Does AWS Step Functions Work?
Application Lifecycle in AWS Step Functions
Visualize in the Console
Define in JSON Monitor Executions
Amazon Confidential
Define in JSON and Then Visualize in the Console
{ ”Comment”: “Hello World Example", "StartAt” : "HelloWorld”, "States” : { "HelloWorld” : { "Type” : "Task", "Resource” :
"arn:aws:lambda:REGION:ACCOUNT_ID:function:FUNCTION_NAME”,
"End” : true } }
}
Execute One or One Million
Start
End
HelloWorld
Start
End
HelloWorld
Start
End
HelloWorld
Start
End
HelloWorld
Start
End
HelloWorld
Start
End
HelloWorld
Start
End
HelloWorld
Start
End
HelloWorld
Monitor Executions from the Console
Monitor Executions from Amazon CloudWatch
“I want to sequence functions”
AWS Step Functions, we can easily change and iterate on the application workflow of our food delivery service in order to optimize operations and continually improve delivery times. AWS Step Functions lets us dynamically scale the steps in our food delivery algorithm so we can manage spikes in customer orders and meet demand. Mathias Nitzsche, CTO, foodpanda
“
var Twit = require('twit'); var T = new Twit(require('botfiles/config.js')); exports.handler = function myBot(event, context, callback) { var list = event.inputList; var textToTweet = list.shift(); var output = { inputList: list } T.post('statuses/update', { status: textToTweet }, function(err, reply) { if (err) { console.log('error:', err); context.fail(); } else { console.log('tweet:', reply); callback(null, output); } }); };
Demo
“I want to select functions based on data”
“With AWS Step Functions, it was easy to build a multi-step product updating system to ensure our database and website always have the latest price and availability information.
AWS Step Functions let us replace a manual updating process with an automated series of steps, including built-in retry conditions and error handling, so we can reliably scale before a big show, and keep pace with rapidly changing fashions.”
Jared Browarnik, CTO, TheTake
“
"ChoiceState": { "Type" : "Choice", "Choices": [ { "Variable": "$.productSource", "StringEquals": "screen-scrape", "Next": "ScreenScrapeState" },{ "Variable": "$.productSource", "StringEquals": "vendor-a", "Next": "VendorA" },{ "Variable": "$.productSource", "StringEquals": "vendor-b", "Next": "VendorB" },
{ "Variable": "$.productSource", "StringEquals": "vendor-c", "Next":"VendorC" },{ "Variable": "$.productSource", "StringEquals": "updateProduct", "Next":"UpdateProduct" } ], "Default": "ScreenScrapeState” }
“I want to retry functions”
We get transient errors from a RESTful service we depend on, once every four or five times we call it. But if we keep retrying, it eventually works.
“
{ "Comment": "Call out to a RESTful service", "StartAt": "Call out", "States": { "Call out": { "Type": "Task", "Resource": "arn:aws:lambda:eu-central-1:123456789012:function:RestCallout", "Retry": [ { "ErrorEquals": [ "HandledError" ], "MaxAttempts": 10 } ], "End": true } } }
“I want try/catch/finally”
AWS Step Functions makes it simple to coordinate information from many different infrastructure systems using easy to design workflows and create a more intelligent monitoring system for our Platform as a Service (PaaS).
With AWS Step Functions, we can reliably automate monitoring decisions and actions in order to reduce human intervention by over 60%, which improves infrastructure operation productivity and customer application availability on our platform.
Pedro Pimenta, VP R&D, OutSystems
“
13 AWS Lambda Task States
6 Choice States
1 Fail State
“I want try/catch/finally”
Try/Catch/Finally for Media Management
"Access Media": { "Type": "Task", "Resource": "arn:aws:lambda:eu-central-1:123456789012:function:FindMedia", "TimeoutSeconds": 2, "Next": "Graceful Exit", "Retry": [ { "ErrorEquals": [ "States.Timeout" ], "IntervalSeconds": 2, "MaxAttempts": 2, "BackoffRate": 1.5 } ], "Catch": [ { "ErrorEquals": [ "States.ALL" ], "Next": "Clean Up" } ] },
Demo
“I have code that runs for hours”
We need to gather data from our production line, in units of 8-hour shifts.“
Task States Poll or Push
AWS Lambda…On-premises
AWS container, EC2 instance, or…
Long poll
Push
Task State to Dispatch Work with Push
{ "StartAt": "HelloWorld", "States": { "HelloWorld": { "Type": "Task", "Resource":"arn:aws:lambda:REGION:ACCOUNT_ID:function:FUNCTION_NAME", "End": true } }
}
Task State to Dispatch Work by Polling
{ "StartAt": "HelloWorld", "States": { "HelloWorld": { "Type": "Task", "Resource":"arn:aws:states:REGION:ACCOUNT_ID:activity:FUNCTION_NAME", "End": true } }
}
"NextShift": { "Type": "Wait", "TimestampPath": "$.ShiftStart", "Next": "Gather Plant Data" }, "Gather Plant Data": { "Type": "Task", "Resource": "arn:aws:states:ap-northeast-1:123456789012:activity:PlWatch", "TimeoutSeconds": 30000, "HeartBeatSeconds": 120, "Next": "Clean up" }
“I want to run functions in parallel”
We want to send the captured image to three OCR providers and take the result with the highest confidence value.“
"Send for OCR": { "Type": "Parallel", "Next": "Pick result", "Branches": [ { "StartAt": "Prep1", "States": { "Prep1": { "Type": "Pass", "Result": { "inputList": [ "OCR Provider 1" ] }, "Next": "Go1" }, "Go1": { "Type": "Task", "Resource": "arn:aws:lambda:eu-central-1:123456789012:function:StatesBot", "End": true } }
Where does transient application state live?
In the machine, in JSON texts passing from state to state.A:
Q:
Input processing{ "title": "Numbers to add", "numbers": [ 3, 4 ] }
{ "Type": "Task", "InputPath": "$.numbers", "Resource": "arn:aws:lambda…" …
[ 3, 4 ]
Raw input:
State spec:
Task input:
Input processing
Q: InputPath not provided?A: State gets raw input as-is.
Q: InputPath is null?A: State gets an empty JSON object: {}
Q: InputPath produces plural output?A: State gets it wrapped in a JSON array.
Result placement{ "title": "Numbers to add", "numbers": [ 3, 4 ] }
{ "Type": "Task", "InputPath": "$.numbers", "ResultPath": "$.sum”, …
{ "title": "Numbers to add", "numbers": [ 3, 4 ], ”sum": 7}
Raw input:
State spec:
Task input:
Result placement
Q: ResultPath is null?A: State input is state output.
Q: ResultPath produces plural output?A: Not allowed, validator won’t accept.
Q: ResultPath not provided?A: Input discarded, raw output used.
“I want to sequence functions”
“I want to select functions based on data”“I want to retry functions”“I want try/catch/finally”
Is this you?
“I have code that runs for hours”
“I want to run functions in parallel”
Where is AWS Step Functions Available?
Region Region Code
US East (N. Virginia) us-east-1
US East (Ohio) us-east-2
US West (Oregon) us-west-2
EU (Dublin) eu-west-1
Asia Pacific (Tokyo) ap-northeast-1
aws.amazon.com/step-functions
Getting Started
aws.amazon.com/step-functions
Thank you!