(CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Download (CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Post on 11-Apr-2017




10 download

Embed Size (px)


<ul><li><p> 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.</p><p>Guy Davies, Sophos Ltd.</p><p>October 2015</p><p>CMP407</p><p>Lambda as CronScheduling Invocations in AWS Lambda</p></li><li><p>About me</p><p> Guy Davies, Senior Systems Engineer, Sophos Ltd</p><p>Sophos use AWS extensively in our operations to support our anti-</p><p>malware, anti-spam, and threat-detection software and hardware </p><p>devices.</p><p>Also provide security products for AWS customers to complement their </p><p>cloud security profiles:</p><p> UTM [ free trial available! ]</p><p> Secure Server for Linux [ AMI available in Marketplace ]</p><p>www.sophos.com/aws</p></li><li><p>What to expect from the session</p><p> How to schedule tasks in AWS Lambda</p><p> Overview of the various options available</p><p> Building up a pure Lambda scheduling infrastructure</p><p> Resources and templates to implement this yourself</p></li><li><p>Events? Scheduling?</p></li><li><p>Why would I want to schedule Lambda?</p><p> Lambda designed as event-driven computing</p><p> Event-driven computing is awesome</p><p> Most of the time you want to trigger because something </p><p>happens</p><p>HOWEVER</p></li><li><p>Why would I want to schedule Lambda?</p><p>Sometimes you just plain need to do something on a </p><p>schedule.</p><p>Examples:</p><p> Log cleanup</p><p> Batching up statistics</p><p> Alarm clock</p><p> Infrastructure automation</p></li><li><p>Scheduling options on traditional infrastructure</p><p>Unix</p><p> cron: recurring tasks</p><p>*/2 * * * * do_something</p><p> at: run a task at a specific timeat 1615 oct 7 </p></li><li><p>Options for scheduling Lambda functions (1)</p><p>1. Spin up an Amazon EC2 instance, and use crontab to </p><p>invoke Lambda</p><p> Why bother running Lambda at all?</p><p> Many folks want a pure-Lambda deployment</p><p> Running an instance means more to manage</p><p> Not hugely financially efficient</p></li><li><p>Options for scheduling Lambda functions (2)</p><p>2. Unreliable Town Clock (townclock.io)</p><p> Awesome public Amazon SNS topic </p><p> Chimes every fifteen minutes</p><p> Community supported</p><p> Has a 15 minute granularity</p></li><li><p>Options for scheduling Lambda functions (3)</p><p>3. Others</p><p> Trigger from Amazon SWF</p><p> Trigger from an instance in AWS Data Pipeline</p><p> Trigger from an AWS CloudTrail upload into an Amazon S3 </p><p>bucket</p><p>All of these could be your solution! But what if we want a </p><p>pure Lambda implementation thats managed by AWS?</p></li><li><p>A pure Lambda scheduler</p></li><li><p>How do we generate a timing signal in AWS?</p><p>0</p><p>0.2</p><p>0.4</p><p>0.6</p><p>0.8</p><p>1</p><p>Photo: Signetics NE5555N, by Stefan506 is </p><p>licenced by CC BY-SA 3.0. Source: </p><p>https://en.wikipedia.org/wiki/555_timer_IC</p><p>Amazon </p><p>CloudWatch</p></li><li><p>CloudWatch as a time signal</p><p>1. Set an alarm on a CloudWatch metric</p><p>2. Alarm goes into ALARM state:</p><p> Triggers Amazon SNS which triggers Lambda</p><p> Lambda inverts the state of the metric</p><p>3. Alarm goes into OK state:</p><p> Triggers SNS which triggers Lambda</p><p> Lambda inverts the state of the metric</p></li><li><p>Configuring the CloudWatch alarm</p></li><li><p>Configuring the CloudWatch alarm</p><p>Once the metric is </p><p>inverted, the alarm will </p><p>trigger at the top of the </p><p>next minute:</p><p>1-minute resolution</p></li><li><p>Configuring the CloudWatch alarm</p><p>All three states trigger the </p><p>same SNS notification. The </p><p>Lambda function figures </p><p>which was the trigger and </p><p>sets the CloudWatch</p><p>metric to the opposite state</p></li><li><p>Putting it together</p><p>Lambda </p><p>cron</p><p>function</p><p>CloudWatch</p><p>metric</p><p>CloudWatch alarm</p><p>triggers</p><p>SNS topic10</p><p>Invoke further Lambda </p><p>functions (if scheduled)</p></li><li><p>The Lambda function</p></li><li><p>Lambda function</p><p>1. The event is the SNS notification that is sent by </p><p>CloudWatch.</p><p>2. Invert the value of the CloudWatch metric.</p><p>3. Jobs and schedule are managed as a separate JSON </p><p>file in the bundle.</p><p>4. Invoke any Lambda functions in the schedule that are </p><p>scheduled to be run this minute.</p><p>5. Done.</p></li><li><p>Main Lambda function</p><p>exports.handler = function(event, context) {</p><p>async.waterfall([</p><p>function (callback) { flip_cloudwatch(event,callback); },</p><p>read_crontab,</p><p>execute_lambdas</p><p>], function (err) {</p><p>if (err) context.fail(err);</p><p>else context.succeed(); });</p><p>};</p></li><li><p>Flipping CloudWatch</p><p>The event is an SNS message from CloudWatch:</p><p>var snsmessage = JSON.parse(event.Records[0].Sns.Message);</p><p>If its just gone into alarm, we want to reset to zero. Likewise if its just gone into </p><p>OK, reset to 1.</p><p>if (snsmessage.NewStateValue == 'ALARM') { value = 0.0 }</p><p>else if (snsmessage.NewStateValue == 'OK' || snsmessage.NewStateValue == </p><p>'INSUFFICIENT_DATA') { value = 1.0 };</p><p>Push the new value to CloudWatch:</p><p>var params = { MetricData: [ { MetricName: 'LambdaCron', Timestamp: new Date, </p><p>Unit: 'None', Value: value } ], Namespace: 'LambdaCron' };</p><p>cloudwatch.putMetricData(params, function(err, data) { . . . });</p></li><li><p>Crontab-like Lambda configuration</p><p>{</p><p>"jobs": [ {</p><p>"schedule": "*/3 * * * *",</p><p>"function": "testfunction",</p><p>"args": {</p><p>"key1": "test1",</p><p>"key3": "test3",</p><p>"key2": "test2"</p><p>}</p><p>} ]</p><p>}</p></li><li><p>Checking the schedule (1)</p><p>Use the fantastic cron-parser library (https://github.com/harrisiirak/cron-parser/</p><p>MIT licenced)</p><p>var parser = require('cron-parser');</p><p>Create a Date object which refers to the top of the current minute to compare </p><p>the schedule to.</p><p>var d = new Date();</p><p>d.setSeconds(0);</p><p>d.setMilliseconds(0);</p><p>Parse the crontab to find the next runtime for the job (= now if it needs to run)</p><p>var interval =</p><p>parser.parseExpression(job["schedule"],{currentDate: d});</p><p>var runtime = interval.next();</p><p>https://github.com/harrisiirak/cron-parser/</p></li><li><p>Run each job that needs running</p><p>if (datestring == runtimestring) {</p><p>var lambda = new AWS.Lambda();</p><p>var params = {</p><p>FunctionName: job["function"],</p><p>InvocationType: "Event",</p><p>Payload: JSON.stringify(job["args"])</p><p>};</p><p>lambda.invoke(params, function(err,data) {</p><p>if (err) iteratorcallback(err);</p><p>else iteratorcallback(null);</p><p>});</p><p>}</p><p>}</p></li><li><p>Demo</p></li><li><p>Use cases</p><p>Trialling this for intelligent scaling on a schedule</p><p> AWS scheduled scaling has limitations:</p><p> Absolute number of desired instances</p><p> Limit of 120 tasks per month</p><p> Lambda function triggered by lambda-cron could do for </p><p>example:</p><p> Every morning at 08:00 UTC, add 20% of capacity unless we </p><p>have &gt; 30 instances already running</p></li><li><p>Reliability and monitoring</p><p>Empirically pretty reliable.</p><p>Running since April without (much!) intervention.</p><p>Even comes back after outages!</p></li><li><p>Reliability and monitoring</p><p>Monitoring: CloudWatch!</p><p> Lambda invocation metrics will tell you that its running</p><p> Application-level monitoring on the jobs you are </p><p>triggering</p><p>60 invocations / hr</p></li><li><p>Summary</p><p>1. Use CloudWatch alarms as two states to provide a </p><p>timing signal to Lambda.</p><p>2. Trigger off all three states to enhance reliability.</p><p>3. Allows us to schedule tasks purely within Lambda.</p><p>4. The cron function invokes once a minute.</p></li><li><p>Resources</p><p>Github:</p><p>github.com/g-a-d/lambda-cron</p><p>Email:</p><p>guy.davies@sophos.com</p><p>AWS </p><p>CloudFormation</p><p>stack</p><p>Lambda</p><p>function</p></li><li><p>Resources</p><p>Libraries used:</p><p> async: https://github.com/caolan/async</p><p> avoid nested-callback-hell (great for folks more used to </p><p>procedural programming)</p><p> MIT license</p><p> cron-parser: https://github.com/harrisiirak/cron-parser</p><p> parse crontabs</p><p> MIT license</p><p>https://github.com/caolan/asynchttps://github.com/harrisiirak/cron-parser</p></li><li><p>Resources</p><p>Remember to check www.sophos.com/aws</p><p> UTM for AWS (free trial available)</p><p> Secure server for AWS</p><p> Free trials, free home use AV</p></li><li><p>Remember to complete </p><p>your evaluations!</p></li><li><p>Thank you!</p></li></ul>