clojure applications in building serverless · brief introduction to serverless architecture aws...

Building Serverless Applications in ClojureStuart Robinson - Principal Engineer @ Funding Circle

Overview

Serverless IntroBrief introduction to Serverless Architecture

AWS LambdaWorking with AWS Lambda, API Gateway and Clojure

Full Stack ArchitectureDeploying a full stack Clojure application to AWS

DemoA simple chat application

Lessons LearnedGood practices and things to avoid.

2

What does “serverless” mean?

What does “serverless” mean?

No more servers?

What does “serverless” mean?

There are still servers but you can’t see them..

“Build and run applications without thinking

about servers - Amazon

Comparison with server based model

Server based application deployment

● Servers need to be provisioned and configured

● Usually at least one instance running

● Pay for running instances

● Instances are generally long lived

Serverless application deployment

● No provisioning required

● May have zero instances running

● Pay when your code executes

● Instances are short lived

Serverless / FaaS Providers

Provider Language Support Pricing Model

AWS Lambda JavascriptPythonJavaC#Go

Per GB per second

Azure Functions JavascriptPython JavaC#PHPBash

Per GB per second

Google Cloud Functions Javascript Per invocation

Amazon Serverless Components

Kinesis Cognito

Building a Clojure API on AWS Lambda

AWS Lambda

• AWS lambda is a serverless compute service

• Lambda functions are executed inside containers (VMs)

• Lambda containers are stateless*

• Containers are started and stopped on demand (freeze / thaw cycle)

• Containers that are no longer needed are destroyed

• Lambda automatically scales to meet demand (up to 1000 concurrent requests)

Writing a simple Lambda function in Clojure

Packaging and deployment

Using AWS command line tools:

lein uberjar

aws lambda create-function

--function-name clojure-serverless-demo

--handler clojure_serverless_demo.SimpleHander

--runtime java8

--memory 512

--timeout 10

--role arn:aws:iam::231321312:role/lambda_exec_role

--zip-file fileb://./target/clojure-serverless-demo-standalone.jar

Serverless Framework

14

• Command line tool for packaging and deploying serverless applications

• Support for many providers (Azure, Google, AWS)

• Allows you to codify all your serverless resources in a single serverless.yml file

• Extensible with plugins and templates

Serverless.yml

HTTP with API Gateway

• Serverless API proxy

• Support for authentication, rate limiting and caching

• An event source for Lambda functions

• Can act as proxy for other APIs

• Support for multiple versions and environments as well as canary deployments

Configuring API gateway

An API Gateway Request Event

Converting API Gateway request to a Ring request

A RESTful API on Lambda

Back-end Architecture

Full stack Application Architecture

Demo

Links • Demo Address: http://chat.stuart.cloud

• Github: https://github.com/stuart-robinson/clojure-serverless-demo

Lessons Learned

Testing

• Separate your application core from the AWS plumbing code

• Test your core logic locally with unit tests

• Some of the AWS services have versions that can be run locally (Kinesis, Dynamodb etc) for integration style tests

• Test environments on AWS cost (next to) nothing while they are not in use

Monitoring and Maintenance

• Lots of monitoring provided by AWS out of the box (Cloudwatch, X-ray)

• Log as much as possible

• Setup alerts so you can determine when you’re being throttled

• Monitor your costs very carefully

• Watch for circular calls

Performance

• Time to serve request from cold start:

- Lambda Java Runtime: ~ 600ms

- Lambda Java Runtime + Clojure ~ 3000ms

- Lambda Java Runtime + Clojure + API Gateway ~ 6000ms

• Request time once warm: < 100ms

• Reduce colds start times by:

- Pruning unused libraries and excluding dev dependencies

- Allocating more memory (CPU is allocated proportionally to memory)

- Keep function warm by regular polling (every 15mins)

- Using ClojureScript on NodeJS runtime

Costs - 25 Million requests per month (~10/sec)

• AWS Lambda

- Cost = duration * memory usage (+ 0.20 per 1M invocations)

- Minimum billable duration = 100ms

- 512 Mb memory is sufficient for a small Clojure project

- £25.85 (excludes free tier allowance)

• DynamoDB

- 40 Reads/sec + 10 Writes/sec ~ £10

• API Gateway

- ~ £2.50/million requests

- £75 for 25 million requests

• Total Cost ~ £110

Summary

• Deploying Clojure apps to Lambda is pretty straightforward

• Working with Lambda enables you to focus on the application code without worrying about infrastructure

• Cold start times for Clojure are woeful compared to other languages

• Need to be monitor costs and optimize accordingly

30

Questions?