(cmp407) lambda as cron: scheduling invocations in aws lambda

Download (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Post on 11-Apr-2017

11.228 views

Category:

Technology

10 download

Embed Size (px)

TRANSCRIPT

  • 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

    Guy Davies, Sophos Ltd.

    October 2015

    CMP407

    Lambda as CronScheduling Invocations in AWS Lambda

  • About me

    Guy Davies, Senior Systems Engineer, Sophos Ltd

    Sophos use AWS extensively in our operations to support our anti-

    malware, anti-spam, and threat-detection software and hardware

    devices.

    Also provide security products for AWS customers to complement their

    cloud security profiles:

    UTM [ free trial available! ]

    Secure Server for Linux [ AMI available in Marketplace ]

    www.sophos.com/aws

  • What to expect from the session

    How to schedule tasks in AWS Lambda

    Overview of the various options available

    Building up a pure Lambda scheduling infrastructure

    Resources and templates to implement this yourself

  • Events? Scheduling?

  • Why would I want to schedule Lambda?

    Lambda designed as event-driven computing

    Event-driven computing is awesome

    Most of the time you want to trigger because something

    happens

    HOWEVER

  • Why would I want to schedule Lambda?

    Sometimes you just plain need to do something on a

    schedule.

    Examples:

    Log cleanup

    Batching up statistics

    Alarm clock

    Infrastructure automation

  • Scheduling options on traditional infrastructure

    Unix

    cron: recurring tasks

    */2 * * * * do_something

    at: run a task at a specific timeat 1615 oct 7

  • Options for scheduling Lambda functions (1)

    1. Spin up an Amazon EC2 instance, and use crontab to

    invoke Lambda

    Why bother running Lambda at all?

    Many folks want a pure-Lambda deployment

    Running an instance means more to manage

    Not hugely financially efficient

  • Options for scheduling Lambda functions (2)

    2. Unreliable Town Clock (townclock.io)

    Awesome public Amazon SNS topic

    Chimes every fifteen minutes

    Community supported

    Has a 15 minute granularity

  • Options for scheduling Lambda functions (3)

    3. Others

    Trigger from Amazon SWF

    Trigger from an instance in AWS Data Pipeline

    Trigger from an AWS CloudTrail upload into an Amazon S3

    bucket

    All of these could be your solution! But what if we want a

    pure Lambda implementation thats managed by AWS?

  • A pure Lambda scheduler

  • How do we generate a timing signal in AWS?

    0

    0.2

    0.4

    0.6

    0.8

    1

    Photo: Signetics NE5555N, by Stefan506 is

    licenced by CC BY-SA 3.0. Source:

    https://en.wikipedia.org/wiki/555_timer_IC

    Amazon

    CloudWatch

  • CloudWatch as a time signal

    1. Set an alarm on a CloudWatch metric

    2. Alarm goes into ALARM state:

    Triggers Amazon SNS which triggers Lambda

    Lambda inverts the state of the metric

    3. Alarm goes into OK state:

    Triggers SNS which triggers Lambda

    Lambda inverts the state of the metric

  • Configuring the CloudWatch alarm

  • Configuring the CloudWatch alarm

    Once the metric is

    inverted, the alarm will

    trigger at the top of the

    next minute:

    1-minute resolution

  • Configuring the CloudWatch alarm

    All three states trigger the

    same SNS notification. The

    Lambda function figures

    which was the trigger and

    sets the CloudWatch

    metric to the opposite state

  • Putting it together

    Lambda

    cron

    function

    CloudWatch

    metric

    CloudWatch alarm

    triggers

    SNS topic10

    Invoke further Lambda

    functions (if scheduled)

  • The Lambda function

  • Lambda function

    1. The event is the SNS notification that is sent by

    CloudWatch.

    2. Invert the value of the CloudWatch metric.

    3. Jobs and schedule are managed as a separate JSON

    file in the bundle.

    4. Invoke any Lambda functions in the schedule that are

    scheduled to be run this minute.

    5. Done.

  • Main Lambda function

    exports.handler = function(event, context) {

    async.waterfall([

    function (callback) { flip_cloudwatch(event,callback); },

    read_crontab,

    execute_lambdas

    ], function (err) {

    if (err) context.fail(err);

    else context.succeed(); });

    };

  • Flipping CloudWatch

    The event is an SNS message from CloudWatch:

    var snsmessage = JSON.parse(event.Records[0].Sns.Message);

    If its just gone into alarm, we want to reset to zero. Likewise if its just gone into

    OK, reset to 1.

    if (snsmessage.NewStateValue == 'ALARM') { value = 0.0 }

    else if (snsmessage.NewStateValue == 'OK' || snsmessage.NewStateValue ==

    'INSUFFICIENT_DATA') { value = 1.0 };

    Push the new value to CloudWatch:

    var params = { MetricData: [ { MetricName: 'LambdaCron', Timestamp: new Date,

    Unit: 'None', Value: value } ], Namespace: 'LambdaCron' };

    cloudwatch.putMetricData(params, function(err, data) { . . . });

  • Crontab-like Lambda configuration

    {

    "jobs": [ {

    "schedule": "*/3 * * * *",

    "function": "testfunction",

    "args": {

    "key1": "test1",

    "key3": "test3",

    "key2": "test2"

    }

    } ]

    }

  • Checking the schedule (1)

    Use the fantastic cron-parser library (https://github.com/harrisiirak/cron-parser/

    MIT licenced)

    var parser = require('cron-parser');

    Create a Date object which refers to the top of the current minute to compare

    the schedule to.

    var d = new Date();

    d.setSeconds(0);

    d.setMilliseconds(0);

    Parse the crontab to find the next runtime for the job (= now if it needs to run)

    var interval =

    parser.parseExpression(job["schedule"],{currentDate: d});

    var runtime = interval.next();

    https://github.com/harrisiirak/cron-parser/

  • Run each job that needs running

    if (datestring == runtimestring) {

    var lambda = new AWS.Lambda();

    var params = {

    FunctionName: job["function"],

    InvocationType: "Event",

    Payload: JSON.stringify(job["args"])

    };

    lambda.invoke(params, function(err,data) {

    if (err) iteratorcallback(err);

    else iteratorcallback(null);

    });

    }

    }

  • Demo

  • Use cases

    Trialling this for intelligent scaling on a schedule

    AWS scheduled scaling has limitations:

    Absolute number of desired instances

    Limit of 120 tasks per month

    Lambda function triggered by lambda-cron could do for

    example:

    Every morning at 08:00 UTC, add 20% of capacity unless we

    have > 30 instances already running

  • Reliability and monitoring

    Empirically pretty reliable.

    Running since April without (much!) intervention.

    Even comes back after outages!

  • Reliability and monitoring

    Monitoring: CloudWatch!

    Lambda invocation metrics will tell you that its running

    Application-level monitoring on the jobs you are

    triggering

    60 invocations / hr

  • Summary

    1. Use CloudWatch alarms as two states to provide a

    timing signal to Lambda.

    2. Trigger off all three states to enhance reliability.

    3. Allows us to schedule tasks purely within Lambda.

    4. The cron function invokes once a minute.

  • Resources

    Github:

    github.com/g-a-d/lambda-cron

    Email:

    guy.davies@sophos.com

    AWS

    CloudFormation

    stack

    Lambda

    function

  • Resources

    Libraries used:

    async: https://github.com/caolan/async

    avoid nested-callback-hell (great for folks more used to

    procedural programming)

    MIT license

    cron-parser: https://github.com/harrisiirak/cron-parser

    parse crontabs

    MIT license

    https://github.com/caolan/asynchttps://github.com/harrisiirak/cron-parser

  • Resources

    Remember to check www.sophos.com/aws

    UTM for AWS (free trial available)

    Secure server for AWS

    Free trials, free home use AV

  • Remember to complete

    your evaluations!

  • Thank you!