Build Web Applications using Microservices on Node.js and Serverless AWS

Download Build Web Applications using Microservices on Node.js and Serverless AWS

Post on 16-Apr-2017

2.761 views

Category:

Software

4 download

Embed Size (px)

TRANSCRIPT

<p>Deck Guidelines</p> <p>Eugene Istrati, Technology PartnerBuild Web Applicationsusing Microservices on Node.js and Serverless AWSeugene@mitocgroup.comwww.mitocgroup.com</p> <p>http://www.meetup.com/nodejs/events/226713968</p> <p>Hello everybody and welcome! Thank you for taking the time to attend this session. I feel very humble and honored to be here today to talk about Building Web Applications...The more I dive into microservices, the more it reminds me of the joke: That any software program can be reduced to one line of code ... that has a bug. I hope my presentation is better than my joke :)</p> <p>1</p> <p>Eugene Istrati, Partner @ Mitoc GroupMicroservices Architecturefor Digital Platforms with AWS Lambda, Amazon CloudFront, and Amazon DynamoDBeugene@mitocgroup.comOctober 2015ARC201</p> <p> 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.</p> <p>This talk is an evolution from my last month presentation at AWS re:Invent conference.</p> <p>2</p> <p>Digital Platform ChallengesNote: Credits and thanks are listed at the end of the presentation</p> <p>The fundamental role of every web application and digital platform is to be up and running 24/7. But if you are an employee of BuzzFeed, or New York Media, or Flipkart, you have most probably experienced recently a high level of stress and pressure. In BuzzFeeds case it was caused by traffic peak generated by the famous article what is the color of this dress?. In case of New York Magazine, it was an attack led by some extremist individuals. In case of Flipkart, it was an exclusive launch of a low-cost smartphone that lots of customers wanted. Please raise your hand if you have been in similar situation before? PAUSE. Me too, I have personally experienced a similar situation when Michael Jackson died in 2009 and the breaking news brought down our entire digital platform for a couple of hours. Yes, it was painful.</p> <p>3</p> <p>Average cost of downtime$500K - $1M / hour (IDC, Dec 2014)$140K - $540K / hour (Garner, July 2014)$474K / hour (Ponemon Inst., Dec 2013)</p> <p>Most commonly reported consequencesDamage to reputation (38%)Increase in customer churn (37%)Damage to credit rating (28%)Increase to insurance premiums (26%)Digital Platform ChallengesNote: Credits and thanks are listed at the end of the presentation</p> <p>It is a big concern for us that 87% of attacks affect our digital platforms. The average downtime costs us hundreds of thousands and millions of dollars per hour, not to mention damages in reputation and credit rating, customer churn and insurance increases. So is there something, that we can easily improve in our architecture, that will solve these problems fundamentally? Well yes, otherwise I wouldnt be here :)</p> <p>4</p> <p>Digital Enterprise End-to-end Platform on AWS</p> <p>Weve been doing it for a while, constantly helping customers on AWS, business owners or decisions makers, architects or developers, technical or none technical, we help everybody improve their digital platforms. That is how we ended up using abstracted services from AWS and building Platform-as-a-Service that we call Digital Enterprise End-to-end Platform.</p> <p>5</p> <p>AboutEugene Istrati</p> <p>eugene@mitocgroup.com</p> <p>Partner @ Mitoc Group Inc</p> <p>15+ years in IT; 7+ years on AWS</p> <p>AWS Certified Solutions Architect (re-certified at re:Invent 2015)</p> <p>Companies: Hearst, Amazon, GrubHub, Tenaris (Europe)Mitoc Group Inc</p> <p>www.mitocgroup.com</p> <p>Web Development Studio</p> <p>AWS Technology Partner</p> <p>Focusing on enterprise applications and platforms</p> <p>Working with customers from media and entertainment industry</p> <p>My name is Eugene Istrati. Im the Technology Partner at Mitoc Group. I have been in IT for over 15 years, with last 7 years working on AWS. I am certified solution architect who worked at companies like Hearst, Amazon, GrubHub and Tenaris. Mitoc Group is a web development studio that focuses on enterprise applications and platforms. We are official AWS Technology Partner and most of our customers are media and entertainment companies.</p> <p>6</p> <p>AgendaWeb Apps Hosting on AWSReference ArchitectureServerless ArchitectureDemo: Setup Serverless AWS</p> <p>Microservices ArchitecturePowered by AWS LambdaTips and TricksDemo: todo.deep.mg</p> <p>Q&amp;A + Next Steps</p> <p>So lets get started with web applications hosting on AWS.</p> <p>7</p> <p>Demo: todo.deep.mgGo to the GitHub repositorygithub.com/MitocGroup/deep-microservices-todo-app</p> <p>Follow the steps from Getting Started to build and deploytodo.deep.com</p> <p>My goal today is to show you hands on the value of microservices. At the end of this talk, I will demo some steps from our development process, based on the code that runs on todo.deep.mg. Because the initial provisioning takes some time, Ill fire it up now, at the beginning of the presentation and spend the rest of the time explaining and showing.</p> <p>8</p> <p>AgendaWeb Apps Hosting on AWSReference ArchitectureServerless ArchitectureDemo: Setup Serverless AWS</p> <p>Microservices ArchitecturePowered by AWS LambdaTips and TricksDemo: todo.deep.mg</p> <p>Q&amp;A + Next Steps</p> <p>What is the reference architecture for web applications hosting on AWS?</p> <p>9</p> <p>Web Apps Hosting / Reference Architecture</p> <p>Availability Zone A</p> <p>Availability Zone B</p> <p>Auto Scaling Group</p> <p>Auto Scaling Group</p> <p>www.example.comstatic.example.com</p> <p>webservers</p> <p>webservers</p> <p>appservers</p> <p>appservers</p> <p>Scales in minutes</p> <p>By the show of hands, lets see how many of us are using this 3 tier architecture? PAUSE. Awesome! In a nutshell, the infrastructure spreads across multiple availability zones, which means it is running in separate physical datacenters that are millisecond latency apart from each other. Therefore it is no surprise that this architecture scales in minutes.</p> <p>10</p> <p>Web Apps Hosting / Reference Architecture</p> <p>Availability Zone A</p> <p>Availability Zone B</p> <p>Auto Scaling Group</p> <p>Auto Scaling Group</p> <p>www.example.comstatic.example.com</p> <p>webservers</p> <p>webservers</p> <p>appservers</p> <p>appservers</p> <p>Scales in minutesHuge challenge for breaking news, viral content, or attacks</p> <p>But if you have experienced before breaking news, or viral content, or various attacks on your digital platforms, you know that scaling in minutes is just not enough. We had to build by ourselves additional complexity to scale the infrastructure faster and meet the spiking demands.</p> <p>11</p> <p>Web Apps Hosting / Reference Architecture</p> <p>Availability Zone A</p> <p>Availability Zone B</p> <p>Auto Scaling Group</p> <p>Auto Scaling Group</p> <p>www.example.comstatic.example.com</p> <p>webservers</p> <p>webservers</p> <p>appservers</p> <p>appservers</p> <p>Scales in minutesHuge challenge for breaking news, viral content, or attacks</p> <p>Reduced operational complexity</p> <p>Using this architecture on AWS makes it easier for us to maintain and support. Less operations makes the platform less complex.</p> <p>12</p> <p>Web Apps Hosting / Reference Architecture</p> <p>Availability Zone A</p> <p>Availability Zone B</p> <p>Auto Scaling Group</p> <p>Auto Scaling Group</p> <p>www.example.comstatic.example.com</p> <p>webservers</p> <p>webservers</p> <p>appservers</p> <p>appservers</p> <p>Scales in minutesHuge challenge for breaking news, viral content, or attacks</p> <p>Reduced operational complexityRequires DevOps with experience</p> <p>But we still needed experienced devops engineers to do so.</p> <p>13</p> <p>Web Apps Hosting / Reference Architecture</p> <p>Availability Zone A</p> <p>Availability Zone B</p> <p>Auto Scaling Group</p> <p>Auto Scaling Group</p> <p>www.example.comstatic.example.com</p> <p>webservers</p> <p>webservers</p> <p>appservers</p> <p>appservers</p> <p>Scales in minutesHuge challenge for breaking news, viral content, or attacks</p> <p>Reduced operational complexityRequires DevOps with experience</p> <p>Flexible choice of technology</p> <p>As developers, we can choose whatever technology stack we would like to use: Java or C#, Python or Ruby, Scala or Go, JavaScript or JavaScript.</p> <p>14</p> <p>Web Apps Hosting / Reference Architecture</p> <p>Availability Zone A</p> <p>Availability Zone B</p> <p>Auto Scaling Group</p> <p>Auto Scaling Group</p> <p>www.example.comstatic.example.com</p> <p>webservers</p> <p>webservers</p> <p>appservers</p> <p>appservers</p> <p>Scales in minutesHuge challenge for breaking news, viral content, or attacks</p> <p>Reduced operational complexityRequires DevOps with experience</p> <p>Flexible choice of technologyRequires devs with rich skill set</p> <p>But we had to recruit and hire developers with rich skillset, who are able to build and support the entire technology stack.</p> <p>15</p> <p>Web Apps Hosting / Reference Architecture</p> <p>Availability Zone A</p> <p>Availability Zone B</p> <p>Auto Scaling Group</p> <p>Auto Scaling Group</p> <p>www.example.comstatic.example.com</p> <p>webservers</p> <p>webservers</p> <p>appservers</p> <p>appservers</p> <p>Scales in minutesHuge challenge for breaking news, viral content, or attacks</p> <p>Reduced operational complexityRequires DevOps with experience</p> <p>Flexible choice of technologyRequires devs with rich skill set</p> <p>Cost-effective</p> <p>And, of course, this architecture is cost effective, if we implement it properly. We are paying only for resources that we are using.</p> <p>16</p> <p>Web Apps Hosting / Reference Architecture</p> <p>Availability Zone A</p> <p>Availability Zone B</p> <p>Auto Scaling Group</p> <p>Auto Scaling Group</p> <p>www.example.comstatic.example.com</p> <p>webservers</p> <p>webservers</p> <p>appservers</p> <p>appservers</p> <p>Scales in minutesHuge challenge for breaking news, viral content, or attacks</p> <p>Reduced operational complexityRequires DevOps with experience</p> <p>Flexible choice of technologyRequires devs with rich skill set</p> <p>Cost-effectiveOver-provisioning and over-paying</p> <p>But when the infrastructure doesnt scale fast enough to meet the demand, our engineering teams are over-provisioning to solve short-term problems and buy time until they figure out long-term solutions. Please raise your hand if you have done it before. PAUSE. Dont tell my boss, but I did it as well.</p> <p>17</p> <p>Web Apps Hosting / Reference Architecture</p> <p>Availability Zone A</p> <p>Availability Zone B</p> <p>Auto Scaling Group</p> <p>Auto Scaling Group</p> <p>www.example.comstatic.example.com</p> <p>webservers</p> <p>webservers</p> <p>appservers</p> <p>appservers</p> <p>Scales in minutesHuge challenge for breaking news, viral content, or attacks</p> <p>Reduced operational complexityRequires DevOps with experience</p> <p>Flexible choice of technologyRequires devs with rich skill set</p> <p>Cost-effectiveOver-provisioning and over-paying</p> <p>While we were trying to solve these problems, two major events happened that changed everything.</p> <p>18</p> <p>AWS re:Invent 2014</p> <p>Note: Credits and thanks are listed at the end of the presentation</p> <p>1. Last year, at the re:Invent, Amazon launched AWS Lambda, an event-driven computing service for dynamic applications.</p> <p>19</p> <p>AWS Summit NY 2015</p> <p>Note: Credits and thanks are listed at the end of the presentation</p> <p>And 2. This year, at NY Summit, AWS launched Amazon API Gateway, a fully managed service for scalable API endpoints.</p> <p>20</p> <p>Web Apps Hosting Reinvented</p> <p>Availability Zone A</p> <p>Availability Zone B</p> <p>Auto Scaling Group</p> <p>Auto Scaling Group</p> <p>www.example.comstatic.example.com</p> <p>webservers</p> <p>webservers</p> <p>appservers</p> <p>appservers</p> <p>S3 bucketCloudFrontdistribution</p> <p>Web Tier</p> <p>CognitoIdentityDB Tier</p> <p>SQS</p> <p>DynamoDB</p> <p>Lambda</p> <p>CloudFrontlogs</p> <p>API Gateway</p> <p>www.example.comstatic.example.comApp TierAWS Region</p> <p>RDS Aurora</p> <p>These two new services enabled us to reinvent the reference architecture in a completely serverless approach.</p> <p>21</p> <p>AgendaWeb Apps Hosting on AWSReference ArchitectureServerless ArchitectureDemo: Setup Serverless AWS</p> <p>Microservices ArchitecturePowered by AWS LambdaTips and TricksDemo: todo.deep.mg</p> <p>Q&amp;A + Next Steps</p> <p>So lets dive into serverless architecture next.</p> <p>22</p> <p>What does serverless mean?Not involving a server; composed only of clients.http://www.wordsense.eu/serverless</p> <p>Serverless doesnt mean servers are no longer involved. It simply means that developers no longer have to think "that much" about them. Computing resources get used as services without having to manage around physical capacities or limits.https://www.quora.com/What-is-Serverless-Computing</p> <p>What does serverless mean? Intuitively, there should be something related to no servers. And it is. Developers dont need to deal with servers and all associated operations to keep them up and running at scale. Instead, developers get abstracted services that are highly secure and highly available, pre-scaled and pre-provisioned, so there is no need to worry about under-provisioning or over-provisioning.</p> <p>23</p> <p>Serverless vs. Reference</p> <p>Availability Zone A</p> <p>Availability Zone B</p> <p>Auto Scaling Group</p> <p>Auto Scaling Group</p> <p>www.example.comstatic.example.com</p> <p>webservers</p> <p>webservers</p> <p>appservers</p> <p>appservers</p> <p>S3 bucketCloudFrontdistribution</p> <p>Web Tier</p> <p>CognitoIdentityDB Tier</p> <p>SQS</p> <p>DynamoDB</p> <p>Lambda</p> <p>CloudFrontlogs</p> <p>API Gateway</p> <p>www.example.comstatic.example.comApp TierAWS Region</p> <p>RDS Aurora</p> <p>vs</p> <p>So, the main question is: How can we get to the serverless architecture from the reference one? Let me show you how we did it, layer by layer.</p> <p>24</p> <p>Serverless Architecture Web TierS3 bucketCloudFrontdistribution</p> <p>Web Tier</p> <p>CognitoIdentity</p> <p>CloudFrontlogs</p> <p>www.example.comstatic.example.com</p> <p>Availability Zone A</p> <p>Availability Zone B</p> <p>Auto Scaling Group</p> <p>www.example.comstatic.example.com</p> <p>webservers</p> <p>webservers</p> <p>First question, how can we transform the web tier into a serverless one? Most of us think of S3 as a storage service available over the Internet. We think of S3 as a cluster of web servers behind load balancers that have turned off server side scripting modules. It is secured through IAM and there is no need to worry about underlying infrastructure.</p> <p>25</p> <p>Serverless Architecture Web TierS3 bucketCloudFrontdistribution</p> <p>Web Tier</p> <p>CognitoIdentity</p> <p>CloudFrontlogs</p> <p>www.example.comstatic.example.comStatic AssetsSame as in reference architecturecss, js, docs, images, videos + htmlDynamic FunctionalityUse JS framework (e.g. Angular)SEO-friendly (Custom Error Response + HTML5 History API)Completely ServerlessPre-scaledLow-costLow-maintenance</p> <p>As we are doing this transformation, the static component stays exactly the same as in reference architecture. We load everything into S3: css, javascript, documents, images, videos. And even html, which usually is served by EC2.</p> <p>26</p> <p>Serverless Architecture Web TierS3 bucketCloudFrontdistribution</p> <p>Web Tier</p> <p>CognitoIdentity</p> <p>CloudFrontlogs</p> <p>www.example.comstatic.example.comStatic AssetsSame as in reference architecturecss, js, docs, images, videos + htmlDynamic FunctionalityUse JS framework (e.g. Angular)SEO-friendly (Custom Error Response + HTML5 History API)Completely ServerlessPre-scaledLow-costLow-maintenance</p> <p>Because S3 doesnt allow server side scripting, we use client side languages like JavaScript to add dynamic functionality. Modern JavaScript frameworks like AngularJS caught up a lot lately to other popular web frameworks. They provide similar patterns and best practices like Symfony, or Django, or Rails...</p>

Recommended

View more >