Serverless Logging with AWS Lambda and the Elastic Stack

Download Serverless Logging with AWS Lambda and the Elastic Stack

Post on 15-Apr-2017




1 download

Embed Size (px)


<p>SERVERLESS LOGGING WITH ELKEdoardo Paolo Scalafiotti - Software Engineer @ Sainsburys</p> <p>This talk is divided in 2:Introduction on What is Serverless, Serverless logging with ELK, demo in NodeJS and GOlang</p> <p>To summerize, this talk is about why I think that ELK is a good solution - not the only one - for monitoring logs and metrics for Serverless</p> <p>About meGo, NodeJSSPAs, WebComponentsMaterial DesignServerless ArchitectureMicroservicesMachine LearningFabLab, Meetups</p> <p>AgendaOVERVIEWWHY SERVERLESS?CHALLENGESUSE CASESDEMOQ &amp; A</p> <p>WHAT IS SERVERLESS?</p> <p>When we talk about serverless we are mainly talking about AWS Lambda, which is the more mature solution out there, released in 2014. Other vendors are coming with similar solutions like:Google Cloud Functions (Alpha) Azure Functions</p> <p>PL support:Lambda: Python, Java (slow) and NodeJSGoogle Cloud: NodeJSAzure: Javascript, C#, Python, PHP</p> <p>Well talk about Lambda</p> <p>TL;DRFaaS is about running back end code without managing your own server systems or your own server applications</p> <p>applications that run in stateless compute containers that are event-triggered, ephemeral and fully managed by a 3rd party</p> <p>2 overlapping areas: BaaS and PaaSrun code without provisioning or managing servershorizontal scaling is completely automatic, elastic, and managed by the providerresponds to triggers</p> <p>WHY SHOULD WE CARE?</p> <p>serverless isHOT!</p> <p>Typical servers in business and enterprise data centers deliver between 5 and 15 percent of their maximum computing output on average over the course of the year</p> <p>Reduced operational costGood with irregular, high spike traffic that cannot be handled by auto-scalingPromotes best coding practises</p> <p>WHAT ARE THE CHALLENGES?</p> <p>TL;DRnot for the faint-hearted... at the moment</p> <p>Ugly hacks for keeping the function aliveVendor lock-inUnit testing is ok, integration... another storyRepetition of logicConcurrent limitLogging, Monitoring and Debugging</p> <p>No offline testingConcurrent limit is 100 functions, then queuing / throttling happens</p> <p>USE CASES</p> <p>USE WISELYFaaS, containers and VMs are not enemies</p> <p>There is the perception that VMs, Containers and Serverless are equivalent alternatives</p> <p>Although they represent higher layers of abstractionVM abstracts the hardwareContainer the OSServerless the language runtime</p> <p>They are suited for different use cases. This is about having more choices, not the next hot thing</p> <p>FAAS is seen as a better choice for event-driven apps with few event types per application component,while CONTAINERS are seen as a better choice for synchronous-request driven components with many entry points</p> <p>VMContainersServerlessConfigure machines, storage, networking and OSRun servers, configure applications and control scalingRun my code when its needed</p> <p>Use cases:Real-time file processing (ingestion)Real-time stream processing (mkt. data)Extract, Transform, LoadIoT BackendsMobile BackendsRESTful APIs</p> <p>Open Source alternative: IBMs OpenWhiskAPI Orchestration: and claudia.jsMultiple PL support: ApexLogging and monitoring: ELK</p> <p>LETS GET TO THE CODE...</p> <p>THE to monitor a set of Lambda-based functions that generates Game Of Thrones plot points</p> <p>DEMO</p> <p></p>


View more >