building aws lambda applications with the aws serverless application model (aws sam) - june 2017 aws...

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Chris Munns – Senior Developer Advocate - Serverless

Building AWS Lambda

Applications with the AWS

Serverless Application Model

(AWS SAM)

About me:

Chris Munns - [email protected], @chrismunns

• Senior Developer Advocate - Serverless

• New Yorker

• Previously:

• Business Development Manager – DevOps, July ’15 - Feb ‘17

• AWS Solutions Architect Nov, 2011- Dec 2014

• Formerly on operations teams @Etsy and @Meetup

• Little time at a hedge fund, Xerox and a few other startups

• Rochester Institute of Technology: Applied Networking and

Systems Administration ’05

• Internet infrastructure geek

https://secure.flickr.com/photos/mgifford/4525333972

Why are we

here today?

No servers to provision

or manage

Scales with usage

Never pay for idle Availability and fault

tolerance built in

Serverless means…

Serverless application

SERVICES (ANYTHING)

Changes in

data state

Requests to

endpoints

Changes in

resource state

EVENT SOURCE FUNCTION

Node.js

Python

Java

C#

Common Lambda use cases

Web

Applications

• Static

websites

• Complex web

apps

• Packages for

Flask and

Express

Data

Processing

• Real time

• MapReduce

• Batch

Chatbots

• Powering

chatbot logic

Backends

• Apps &

services

• Mobile

• IoT

</></>

Amazon

Alexa

• Powering

voice-enabled

apps

• Alexa Skills

Kit

IT

Automation

• Policy engines

• Extending

AWS services

• Infrastructure

management

Amazon S3 Amazon

DynamoDB

Amazon

Kinesis

AWS

CloudFormation

AWS CloudTrail Amazon

CloudWatch

Amazon

Cognito

Amazon SNSAmazon

SESCron events

DATA STORES ENDPOINTS

DEVELOPMENT AND MANAGEMENT TOOLS EVENT/MESSAGE SERVICES

Event sources that trigger AWS Lambda

… and more!

AWS

CodeCommit

Amazon

API Gateway

Amazon

AlexaAWS IoT AWS Step

Functions

Meet

SAM!

AWS Serverless Application Model (SAM)

CloudFormation extension optimized for

serverless

New serverless resource types: functions, APIs,

and tables

Supports anything CloudFormation supports

Open specification (Apache 2.0)

https://github.com/awslabs/serverless-application-model

Create templates of your infrastructure

CloudFormation provisions AWS resources

based on dependency needs

Version control/replicate/update templates like

code

Integrates with development, CI/CD,

management tools

JSON and YAML supported

AWS CloudFormation

AWSTemplateFormatVersion: '2010-09-09'

Resources:

GetHtmlFunctionGetHtmlPermissionProd:

Type: AWS::Lambda::Permission

Properties:

Action: lambda:invokeFunction

Principal: apigateway.amazonaws.com

FunctionName:

Ref: GetHtmlFunction

SourceArn:

Fn::Sub: arn:aws:execute-api:${AWS::Region}:${AWS::AccountId}:${ServerlessRestApi}/Prod/ANY/*

ServerlessRestApiProdStage:

Type: AWS::ApiGateway::Stage

Properties:

DeploymentId:

Ref: ServerlessRestApiDeployment

RestApiId:

Ref: ServerlessRestApi

StageName: Prod

ListTable:

Type: AWS::DynamoDB::Table

Properties:

ProvisionedThroughput:

WriteCapacityUnits: 5

ReadCapacityUnits: 5

AttributeDefinitions:

- AttributeName: id

AttributeType: S

KeySchema:

- KeyType: HASH

AttributeName: id

GetHtmlFunction:

Type: AWS::Lambda::Function

Properties:

Handler: index.gethtml

Code:

S3Bucket: flourish-demo-bucket

S3Key: todo_list.zip

Role:

Fn::GetAtt:

- GetHtmlFunctionRole

- Arn

Runtime: nodejs4.3

GetHtmlFunctionRole:

Type: AWS::IAM::Role

Properties:

ManagedPolicyArns:

- arn:aws:iam::aws:policy/AmazonDynamoDBReadOnlyAccess

- arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole

AssumeRolePolicyDocument:

Version: '2012-10-17'

Statement:

- Action:

- sts:AssumeRole

Effect: Allow

Principal:

Service:

- lambda.amazonaws.com

ServerlessRestApiDeployment:

Type: AWS::ApiGateway::Deployment

Properties:

RestApiId:

Ref: ServerlessRestApi

Description: 'RestApi deployment id: 127e3fb91142ab1ddc5f5446adb094442581a90d'

StageName: Stage

GetHtmlFunctionGetHtmlPermissionTest:

Type: AWS::Lambda::Permission

Properties:

Action: lambda:invokeFunction

Principal: apigateway.amazonaws.com

FunctionName:

Ref: GetHtmlFunction

SourceArn:

Fn::Sub: arn:aws:execute-api:${AWS::Region}:${AWS::AccountId}:${ServerlessRestApi}/*/ANY/*

ServerlessRestApi:

Type: AWS::ApiGateway::RestApi

Properties:

Body:

info:

version: '1.0'

title:

Ref: AWS::StackName

paths:

"/{proxy+}":

x-amazon-apigateway-any-method:

x-amazon-apigateway-integration:

httpMethod: ANY

type: aws_proxy

uri:

Fn::Sub: arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${GetHtmlFunction.Arn}/invocations

responses: {}

swagger: '2.0'

CloudFormation template

SAM template

AWSTemplateFormatVersion: '2010-09-09’

Transform: AWS::Serverless-2016-10-31

Resources:

GetHtmlFunction:

Type: AWS::Serverless::Function

Properties:

CodeUri: s3://sam-demo-bucket/todo_list.zip


Runtime: nodejs4.3

Policies: AmazonDynamoDBReadOnlyAccess

Events:

GetHtml:

Type: Api

Properties:

Path: /{proxy+}

Method: ANY

ListTable:

Type: AWS::Serverless::SimpleTable

SAM template

AWSTemplateFormatVersion: '2010-09-09’

Transform: AWS::Serverless-2016-10-31

Resources:

GetHtmlFunction:

Type: AWS::Serverless::Function

Properties:

CodeUri: s3://sam-demo-bucket/todo_list.zip


Runtime: nodejs4.3

Policies: AmazonDynamoDBReadOnlyAccess

Events:

GetHtml:

Type: Api

Properties:

Path: /{proxy+}

Method: ANY

ListTable:

Type: AWS::Serverless::SimpleTable

Tells CloudFormation this is a SAM

template it needs to “transform”

Creates a Lambda function with the

referenced managed IAM policy,

runtime, code at the referenced zip

location, and handler as defined.

Also creates an API Gateway and

takes care of all

mapping/permissions necessary

Creates a DynamoDB table with 5

Read & Write units

SAM template

From: https://github.com/awslabs/aws-serverless-samfarm/blob/master/api/saml.yaml

<-THIS

BECOMES THIS->

SAM Template Capabilities

• Can mix in other non-SAM CloudFormation

resources in the same template• i.e. S3, Kinesis, Step Functions

• Supports use of Parameters, Mappings,

Outputs, etc

• Supports Intrinsic Functions

• Can use ImportValue(exceptions for RestApiId, Policies, StageName attributes)

• YAML or JSON

SAM Template Properties

AWS::Serverless::Function

AWS::Serverless::Api

AWS::Serverless::SimpleTable

From SAM Version 2016-10-31





Handler: index.js

Runtime: nodejs4.3

CodeUri: 's3://my-code-bucket/my-function.zip'

Description: Creates thumbnails of uploaded images

MemorySize: 1024

Timeout: 15

Policies: AmazonS3FullAccess

Environment:

Variables:

TABLE_NAME: my-table

Events:

PhotoUpload:

Type: S3

Properties:

Bucket: my-photo-bucket

Tracing: Active|PassThrough

Tags:

AppNameTag: ThumbnailApp

DepartmentNameTag: ThumbnailDepartmentFrom SAM Version 2016-10-31





StageName: prod

DefinitionUri: swagger.yml

CacheClusterEnabled: true

CacheClusterSize: 28.4

Variables:

VarName: VarValue






PrimaryKey:

Name: id

Type: String

ProvisionedThroughput:

ReadCapacityUnits: 5

WriteCapacityUnits: 5


AWS::Serverless::Function Event source types

S3

SNS

Kinesis | DynamoDB

Schedule

CloudWatchEvent

AlexaSkill

Note: Events are a map of string to Event Source

Object

Event Source Objects have the following structure:

Type:

Properties:

For Example:

Events:

MyEventName:

Type: S3

Properties:

Bucket: my-photo-bucket



S3

SNS

Kinesis | DynamoDB

Schedule

CloudWatchEvent

AlexaSkill

Type: S3

Properties:

Bucket: bucket-name*

Events: S3:Supported events**

Filter:

S3Key:

Rules:

-

Name: prefix|suffix

Value: String

-

Name: prefix|suffix

Value: String

*Bucket must be declared in same template today

**https://docs.aws.amazon.com/AmazonS3/latest/dev/NotificationHowTo.html#supported-notification-event-typesFrom SAM Version 2016-10-31


S3

SNS

Kinesis | DynamoDB

Schedule

CloudWatchEvent

AlexaSkill

Type: SNS

Properties:

Topic: arn:aws:sns:<region>:<account-id>:topic_name



S3

SNS

Kinesis | DynamoDB

Schedule

CloudWatchEvent

AlexaSkill

Type: Kinesis

Properties:

Stream: arn:aws:kinesis:<region>:<account-id>:stream/stream_name

StartingPosition: TRIM_HORIZON|LATEST

BatchSize: <integer>

--------------------------------

Type: DynamoDB

Properties:

Stream: arn:aws:dynamodb:<region>:<account-id>:table/table_name/stream/<time stamp>

StartingPosition: TRIM_HORIZON|LATEST

BatchSize: <integer>From SAM Version 2016-10-31


S3

SNS

Kinesis | DynamoDB

Schedule

CloudWatchEvent

AlexaSkill


Type: Schedule

Properties:

Schedule: Cron|Rate Expression

Input: JSON formatted string

Cron examples:cron(* * * * * *) (every minute)

cron(1/5 8-17 * * 2-6 *) (every five minutes, between 8am and 5pm, Monday-Friday)

Rate Expression examples:rate(10 minutes)

rate(1 hour)


S3

SNS

Kinesis | DynamoDB

Schedule

CloudWatchEvent

AlexaSkill


Type: CloudWatchEvent

Properties:

Pattern: CWE Pattern*

Input: JSON formatted string that overrides the matched event

Inputpath: JSONPath describing part of the event to pass forward

*https://docs.aws.amazon.com/AmazonCloudWatch/latest/event

s/CloudWatchEventsandEventPatterns.html

Currently 14 Event types with many events for each!!


S3

SNS

Kinesis | DynamoDB

Schedule

CloudWatchEvent

AlexaSkill


Type: AlexaSkill*

* creates a resource policy that allows the Amazon

Alexa service to call your Lambda function

powers:

Lambda Environment Variables

• Key-value pairs that you can dynamically pass to your function

• Available via standard environment variable APIs such as process.env for Node.js or os.environ for Python

• Can optionally be encrypted via AWS Key Management Service (KMS)• Allows you to specify in IAM what roles have access to

the keys to decrypt the information

• Useful for creating environments per stage (i.e. dev, testing, production)

API Gateway Stage Variables

• Stage variables act like environment variables

• Use stage variables to store configuration values

• Stage variables are available in the $context object

• Values are accessible from most fields in API Gateway

• Lambda function ARN

• HTTP endpoint

• Custom authorizer function name

• Parameter mappings

Lambda and API Gateway Variables + SAM

Parameters:

MyEnvironment:

Type: String

Default: testing

AllowedValues:

- testing

- staging

- prod

Description: Environment of this stack of resources

SpecialFeature1:

Type: String

Default: false

AllowedValues:

- true

- false

Description: Enable new SpecialFeature1

…

#Lambda

MyFunction:

Type: 'AWS::Serverless::Function'

Properties:

…

Environment:

Variables:

ENVIRONMENT: !Ref: MyEnvironment

Spec_Feature1: !Ref: SpecialFeature1

…

#API Gateway

MyApiGatewayApi:

Type: AWS::Serverless::Api

Properties:

…

Variables:

ENVIRONMENT: !Ref: MyEnvironment

SPEC_Feature1: !Ref: SpecialFeature1

…

SAM Best Practices

• Unless function handlers share code, split them into their

own independent Lambda functions files or binaries

• Another option is to use language specific packages to share

common code between functions

• Unless independent Lambda functions share event

sources, split them into their own code repositories with

their own SAM templates

• Locally lint your YAML or JSON SAM files before

committing them. Then do it again in your CI/CD process

Create multiple environments from one template:

• Use Parameters and Mappings when possible to

build dynamic templates based on user inputs and

pseudo parameters such as AWS::Region

• Use ExportValue & ImportValue to share resource

information across stacks

• Build out multiple environments, such as for

Development, Test, Production and even DR using

the same template, even across accounts

SAM Template

Source

Control

Dev

Test

Prod

SAM Best Practices

AWS commands – Package & Deploy

Package

•Creates a deployment package (.zip file)

•Uploads deployment package to an Amazon S3 Bucket

•Adds a CodeUri property with S3 URI

Deploy

•Calls CloudFormation ‘CreateChangeSet’ API

•Calls CloudFormation ‘ExecuteChangeSet’ API

Fully managed build service that compiles source code,

runs tests, and produces software packages

Scales continuously and processes multiple builds

concurrently

You can provide custom build environments suited to

your needs via Docker images

Only pay by the minute for the compute resources you

use

Launched with CodePipeline and Jenkins integration

New: Can be used as a “Test” action in CodePipeline

AWS CodeBuild

version: 0.1

environment_variables:plaintext:

"INPUT_FILE": "saml.yaml”"S3_BUCKET": ""

phases:install:commands:- npm install

pre_build:commands:- eslint *.js

build:commands:- npm test

post_build:commands:- aws cloudformation package --template $INPUT_FILE --s3-

bucket $S3_BUCKET --output-template post-saml.yamlartifacts:type: zipfiles:

- post-saml.yaml- beta.json

buildspec.yml Example

version: 0.1

environment_variables:plaintext:

"INPUT_FILE": "saml.yaml”"S3_BUCKET": ""

phases:install:commands:- npm install

pre_build:commands:- eslint *.js

build:commands:- npm test

post_build:commands:- aws cloudformation package --template $INPUT_FILE --s3-

bucket $S3_BUCKET --output-template post-saml.yamlartifacts:type: zipfiles:

- post-saml.yaml- beta.json

• Variables to be used by phases of

build

• Examples for what you can do in

the phases of a build: • You can install packages or run

commands to prepare your

environment in ”install”.

• Run syntax checking,

commands in “pre_build”.

• Execute your build

tool/command in “build”

• Test your app further or ship a

container image to a repository

in post_build

• Create and store an artifact in S3

buildspec.yml Example

Continuous delivery service for fast and

reliable application updates

Model and visualize your software release

process

Builds, tests, and deploys your code every time

there is a code change

Integrates with third-party tools and AWS

AWS CodePipeline

Delivery via CodePipeline

Pipeline flow:1. Commit your code to a source code repository

2. Package/Test in CodeBuild

3. Use CloudFormation actions in CodePipeline to create or update stacks via SAM templates

Optional: Make use of ChangeSets

4. Make use of specific stage/environment parameter files to pass in Lambda variables

5. Test our application between stages/environmentsOptional: Make use of Manual Approvals

An example minimal Developer’s pipeline:

MyBranch-Source

Source

CodeCommit

MyApplication

Build

test-build-source

CodeBuild

MyDev-Deploy

create-changeset

AWS CloudFormation

execute-changeset

AWS CloudFormation

Run-stubs

AWS Lambda

This pipeline:

• Three Stages

• Builds code artifact

• One Development environment

• Uses SAM/CloudFormation to

deploy artifact and other AWS

resources

• Has Lambda custom actions for

running my own testing functions

Via referenced parameter file:

CodePipeline + CloudFormation Parameters

Via Parameter Overrides:



Via Parameter Overrides:Pros:

• Allows Developers to update and

provide parameters via file in the code

repository

• Easier to change and iterate via

deployment

Cons:

• Potentially harder to control

security/confidential information passed

in



Via Parameter Overrides:Pros:

• Tighter control over parameters

passed in

• Can restrict access to information

based on visibility to CodePipeline

and CloudFormation

Cons:

• Modification requires a change to the

pipeline and a re-execution

• Harder to track the changes to these

values unless you are tracking them

via CloudFormation to manage the

pipeline(as an example)

Source

Source

CodeCommit

MyApplication

An example minimal production pipeline:

Build

test-build-source

CodeBuild

Deploy Testingcreate-changeset

AWS

CloudFormation

execute-changeset

AWS

CloudFormation

Run-stubs

AWS Lambda

Deploy Stagingcreate-changeset

AWS

CloudFormation

execute-changeset

AWS

CloudFormation

Run-API-test

Runscope

QA-Sign-off

Manual Approval

Review

Deploy Prodcreate-changeset

AWS

CloudFormation

execute-changeset

AWS

CloudFormation

Post-Deploy-Slack

AWS Lambda

This pipeline:

• Five Stages

• Builds code artifact

• Three deployed to “Environments”

• Uses SAM/CloudFormation to

deploy artifact and other AWS

resources

• Has Lambda custom actions for

running my own testing functions

• Integrates with a 3rd party

tool/service

• Has a manual approval before

deploying to production

Introducing AWS CodeStar

aws.amazon.com/serverless

Additional Resources

Serverless Application Model (SAM) - https://github.com/awslabs/serverless-

application-model

Learn more:

AWS Lambda: https://aws.amazon.com/lambda

Amazon API Gateway: https://aws.amazon.com/api-gateway

Products that helped us today:

CloudFormation: https://aws.amazon.com/cloudformation

CodePipeline: https://aws.amazon.com/codepipeline

CodeBuild: https://aws.amaz.com/codebuild

https://github.com/awslabs/serverless-application-model

https://aws.amazon.com/lambda

https://aws.amazon.com/api-gateway

https://aws.amazon.com/cloudformation

https://aws.amazon.com/codepipeline

https://aws.amaz.com/codebuild

?https://secure.flickr.com/photos/dullhunk/202872717/