grails & devops: continuous integration and delivery in the cloud
DESCRIPTION
Nowadays, companies require very short release cycles, especially in lean startup environments. But to release often: deployments should be routine, not terrifying. configuration should require a few clicks, not a thousand-line shell script. problems should be easy to spot, not buried in a log file. You are a developer that need to release every week or every day with a single git commit and zero-downtime? Easily spot release performance or bugs issues? If required, roll back to previous version in few seconds and one click? And you don't want to manage any dedicated repository, monitoring, build, staging, production servers? So this talk is for you! We will explore Lean startup and DevOps concepts and share our experience on how to create a simple and fully automated build pipeline for Grails apps with a live demo, based on SaaS/cloud services: GitHub, Travis CI, NewRelic, AWS (ElasticBeanstalk, CloudFront), etc.TRANSCRIPT
Benoit Hediard AgoraPulse
Grails & DevOps CI & CD in the Cloud
RUNNING A LEAN STARTUP DEVOPS & GRAILS & AWS
Gr8Conf Europe 2014
agora pulse
2
Benoit Hediard @benorama
LEAN STARTUP Part 1
3
DEPLOYMENT PIPELINE WITH GRAILS & AWS
Demo
DEVOPS CULTURE Part 2
#lean #custdev #startup
#agile #devops
#groovylang #grailsfw #aws
BUZZWORD BINGO
About Me
4
A Social Media Management Platform for Brands
Benoit Hédiard Co-founder & CTO
Grails Plugins Author: AWS SDK, Facebook SDK, CDN Asset Pipeline, Coveralls, Raven, SegmentIO…
Passionate about • Lean/DevOps • Cloud/AWS • Java/Grails • UX/UI • Facebook/Twitter Platforms
@benorama
Client Framework
Server Framework
Cloud provider
SaaS services
Our leanstack
5
Elastic Beanstalk
RDS & DynamoDB
SES & SQS
S3 & CloudFront
Route 53
Check http://leanstack.io !
LEAN STARTUP Part 1
6
What is a Lean Startup?
7
A Startup is a search for a repeatable and scalable business model. The Customer Development process is the way startups quickly iterate and test each element of their business model. Agile Development is the way startups quickly iterate their product as they learn. A Lean Startup is Eric Ries’s description of the intersection of Customer Development, Agile Development and if available, open platforms and open source. – Steve Blank, author of The Startup Owner's Handbook
Feedback Loop
8
Ideas
BUILD
Code
MEASURE
Data
LEARN
BUILD – MEASURE – LEARN The fundamental activity of a startup is to turn ideas into products, measure how customers respond, and then learn whether to pivot or persevere. All successful startup processes should be geared to accelerate that feedback loop. – Eric Ries, author of The Lean Startup
Paul Graham’s Startup Curve
9
Minimum Viable Product Product Market Fit
TROUGH OF SORROW
J
L From search mode
Move fast, break things To execution mode
Grow fast, scale things
Scale Initial enthusiasm
Reality sets in
Experimenting & pivoting
Hockey stick
Starts working
Minimum Viable Product
10
Manual Pipeline
✔ Collaboration ✗ Manual Build ✗ Manual Testing
✗ Manual Delivery ✗ Manual Provisioning ✗ Manual Deployment ✗ Manual Monitoring
Demo – Part 1
11
Code Build Test Release Deploy Monitor
Code repo Team communication
Agile Project Management
EC2
Hosting
Idea Customer
Semi-automated Pipeline
✔ Collaboration ✔ Continuous Build ✔ Continuous Testing
✔ Continuous Delivery ✗ Manual Provisioning ✗ Manual Deployment ✗ Manual Monitoring
Demo – Part 2
12
Code Build Test Release Deploy Monitor
Continuous integration and delivery
Idea Customer
Test coverage history & stats
Coveralls Plugin
DEVOPS CULTURE Part 2
13
What is DevOps?
14
DevOps is a software development method that stresses communication, collaboration and integration between software developers and information technology (IT) operations professionals.
✔ Collaboration
✔ Automation & Infrastructure as Code ✔ Continuous Integration / Testing ✔ Continuous Delivery / Deployment ✔ Continuous Monitoring
GOALS: reduce friction and increase velocity
Pre-2000’s Developer…
15
The Game of Thrones era Developers
Requirements
Design
Development
Test
Release
Maintenance
Pre-2000’s Waterfall Development
16
Dev
Test / QA
Ops
Business
Idea
Customer
Silo
Silo
Silo
Friction
Friction
Friction
Cycle time: Months or Years
2000’s Developer…
17
The Rocky Balboa era
Ops Dev
2000’s Iterative Agile Development
18
Develop
Test Release
Design
Dev
Ops
Business
Idea
Customer
Cycle time: Weeks
or Months
Stability
Change
2010’s Developer…
19
The X-Men era Business Dev Ops
One team approach
2010’s Lean & DevOps
20
Develop
Test Release
Design
Dev
Ops
Business
Idea
Customer
Cycle time: Hours
or Days DevOps
Automated Pipeline Demo – Part 3
21
Code Build Test Release Deploy Monitor
✔ Collaboration ✔ Continuous Build ✔ Continuous Testing
✔ Continuous Delivery ✔ Automated Provisioning ✔ Automated Deployment ✗ Manual Monitoring
Elastic Beanstalk
S3 & CloudFront
RDS
PaaS CDN DB
Managed Services
Idea Customer
Elastic Beanstalk Plugin CDN Asset Pipeline Plugin
Git Branching Model
22
develop beta master (PROD)
Snapshot merge deploy Elastic
Beanstalk BETA
feature
Feature merge
test
test
test
Snapshot merge Release merge Version upgrade deploy
Elastic Beanstalk
PROD
test
test
test
test
test
test
test
Cultural shift
23
Developers must own application-layer code wherever it lives, while ops must own the infrastructure wherever that is. – Donnie Berkholz, analyst at RedMonk
Dev/Test Production
Dev Ops ApplicaMons ApplicaMons
Infrastructure Infrastructure
Dev/Test Production
Dev ApplicaMons ApplicaMons
Ops Infrastructure Infrastructure
Responsability
Responsability
Confuse of Dev or Ops? Simple rule: if you are praise for Web site success, you are Dev; if you are blame when Web site down, you are Ops – DevOps Borat
Cultural shift
IT Automation + IaaS Enterprise DevOps
24
3. Deploy your App
1. Describe your “Infrastructure as Code”
2. Build and manage your “Automated Infrastructure”
PROs Portability
Control Flexibility
CONs Complexity
Maintenance Amazon
EC2
Google Compute Engine
IaaS (Infrastructure as a Service)
Configuration Management Tools
Portable Containers
Lean DevOps / NoOps
25
PaaS + SaaS 1. Deploy your App and use Managed Services
Cloud SQL Amazon RDS
Relational DB
Cloud Datastore DynamoDB
NoSQL
Cloud Storage Amazon S3
Storage
+ Cloud EndPoints + Elasticache, AppStream, SWF, SQS, SES, FPS, Elastic Transcoder…
Amazon RedShift
Big Data BigQuery
Google App Engine
PaaS (Platform as a Service)
Cloud DNS
Elastic Beanstalk
PaaS (Platform as a Service)
Amazon Route 53
PROs Focus on building applications instead of managing Infrastructure
Automated Pipeline Demo – Part 4
26
Code Build Test Release Deploy Monitor
✔ Collaboration ✔ Continuous Build ✔ Continuous Testing
✔ Continuous Delivery ✔ Automated Provisioning ✔ Automated Deployment ✔ Continuous Monitoring
Application and server monitoring
Business monitoring
Exception monitoring
Idea Customer
Sentry Plugin NewRelic Plugin SegmentIO Plugin
Demo – Part 5
27
Zero Downtime Deployment (blue/green)
40 minutes ago…
28
MVP running on locale dev machine (manual build pipeline)
2 Travis files & few Grails Plugins later…
29
“Unlimited” Horizontal Auto-Scaling (hundred of web servers)
Push-Button Scalable Database with Multi-AZ (up to 32vCPU and 244Go of RAM)
100% Resilient (no single point of failure)
Deployment in < 10 minutes (deployment pipeline)
100% Cloud-based (fully managed/no SSH)
CONCLUSION
30
Conclusion
31
Such an exciting time to be a developer AND an entrepreneur today…
Embrace Lean Embrace DevOps
Become an entrepreneur!
#lean #custdev
#agile #devops
#groovylang #grailsfw #aws
#PaaS #SaaS
LEAN STARTUP Part 1
32
DEPLOYMENT PIPELINE WITH GRAILS & AWS
Demo
DEVOPS CULTURE Part 2
#lean #custdev #startup
#agile #devops
#groovylang #grailsfw #aws
BUZZWORD BINGO
About Me
33
A Social Media Management Platform for Brands
Benoit Hédiard Co-founder & CTO
Grails Plugins Author: AWS SDK, Facebook SDK, CDN Asset Pipeline, Coveralls, Sentry, SegmentIO…
Passionate about • Lean/DevOps • Cloud/AWS • Java/Grails • UX/UI • Facebook/Twitter Platforms
@benorama
Client Framework
Server Framework
Cloud provider
SaaS services
Our leanstack
34
Elastic Beanstalk
RDS & DynamoDB
SES & SQS
S3 & CloudFront
Route 53
Check out http://leanstack.io !
LEAN STARTUP Part 1
35
What is a Lean Startup?
36
A Startup is a search for a repeatable and scalable business model. The Customer Development process is the way startups quickly iterate and test each element of their business model. Agile Development is the way startups quickly iterate their product as they learn. A Lean Startup is Eric Ries’s description of the intersection of Customer Development, Agile Development and if available, open platforms and open source. – Steve Blank, author of The Startup Owner's Manual
Iridium woes (Fail fast)
• Conceived in 1987 by Motorala • Bought a fleet of 15 rockets from Russia,
the U.S. and China • Built and launch 72 satellites • Nine months after the first call was made
in 1998, Iridium was in Chapter 11 bankruptcy.
37
5.2 billion dollar business plan mistake
No Business Plan Survives First Contact With A Customer – Steve Blank, author of The Startup Owner's Handbook
Feedback Loop
38
Ideas
BUILD
Code
MEASURE
Data
LEARN
BUILD – MEASURE – LEARN The fundamental activity of a startup is to turn ideas into products, measure how customers respond, and then learn whether to pivot or persevere. All successful startup processes should be geared to accelerate that feedback loop. – Eric Ries, author of The Lean Startup
Paul Graham’s Startup Curve
39
Minimum Viable Product Product Market Fit
TROUGH OF SORROW
J
L From search mode
Move fast, break things To execution mode
Grow fast, scale things
Scale Initial enthusiasm
Reality sets in
Experimenting & pivoting
Hockey stick
Starts working
Minimum Viable Product
40
Manual Pipeline
✔ Collaboration ✗ Manual Build ✗ Manual Testing
✗ Manual Delivery ✗ Manual Provisioning ✗ Manual Deployment ✗ Manual Monitoring
Demo – Part 1
41
Code Build Test Release Deploy Monitor
Code repo Team communication
Agile Project Management
EC2
Hosting
Idea Customer
Semi-automated Pipeline
✔ Collaboration ✔ Continuous Build ✔ Continuous Testing
✔ Continuous Delivery ✗ Manual Provisioning ✗ Manual Deployment ✗ Manual Monitoring
Demo – Part 2
42
Code Build Test Release Deploy Monitor
Continuous integration and delivery
Idea Customer
Test coverage history & stats
Coveralls Plugin
DEVOPS CULTURE Part 2
43
What is DevOps?
44
DevOps is a software development method that stresses communication, collaboration and integration between software developers and information technology (IT) operations professionals.
✔ Collaboration
✔ Automation & Infrastructure as Code ✔ Continuous Integration / Testing ✔ Continuous Delivery / Deployment ✔ Continuous Monitoring
GOALS: reduce friction and increase velocity
Pre-2000’s Developer…
45
The Game of Thrones era Developers
Requirements
Design
Development
Test
Release
Maintenance
Pre-2000’s Waterfall Development
46
Dev
Test / QA
Ops
Business
Idea
Customer
Silo
Silo
Silo
Friction
Friction
Friction
Cycle time: Months or Years
2000’s Developer…
47
The Rocky Balboa era
Ops Dev
2000’s Iterative Agile Development
48
Develop
Test Release
Design
Dev
Ops
Business
Idea
Customer
Cycle time: Weeks
or Months
Stability
Change
Agile
2010’s Developer…
49
The X-Men era Business Dev Ops
One team approach
2010’s Lean & DevOps
50
Develop
Test Release
Design
Dev
Ops
Business
Idea
Customer
Cycle time: Hours
or Days DevOps
Lean
Automated Pipeline Demo – Part 3
51
Code Build Test Release Deploy Monitor
✔ Collaboration ✔ Continuous Build ✔ Continuous Testing
✔ Continuous Delivery ✔ Automated Provisioning ✔ Automated Deployment ✗ Manual Monitoring
Elastic Beanstalk
S3 & CloudFront
RDS
PaaS CDN DB
Managed Services
Idea Customer
Elastic Beanstalk Plugin CDN Asset Pipeline Plugin
Git Branching Model
52
develop beta master (PROD)
Snapshot merge deploy Elastic
Beanstalk BETA
feature
Feature merge
test
test
test
Snapshot merge Release merge Version upgrade deploy
Elastic Beanstalk
PROD
test
test
test
test
test
test
test
Cultural shift
53
Developers must own application-layer code wherever it lives, while ops must own the infrastructure wherever that is. – Donnie Berkholz, analyst at RedMonk
Dev/Test Production
Dev Ops ApplicaMons ApplicaMons
Infrastructure Infrastructure
Dev/Test Production
Dev ApplicaMons ApplicaMons
Ops Infrastructure Infrastructure
Responsability
Responsability
Confuse of Dev or Ops? Simple rule: if you are praise for Web site success, you are Dev; if you are blame when Web site down, you are Ops – DevOps Borat
Cultural shift
IT Automation + IaaS Enterprise DevOps
54
3. Deploy your App
1. Describe your “Infrastructure as Code”
2. Build and manage your “Automated Infrastructure”
PROs Portability
Control Flexibility
CONs Complexity
Maintenance Amazon
EC2
Google Compute Engine
IaaS (Infrastructure as a Service)
Configuration Management Tools
Portable Containers
Lean DevOps / NoOps
55
PaaS + SaaS 1. Deploy your App and use Managed Services
Cloud SQL Amazon RDS
Relational DB
Cloud Datastore DynamoDB
NoSQL
Cloud Storage Amazon S3
Storage
+ Cloud EndPoints + Elasticache, AppStream, SWF, SQS, SES, FPS, Elastic Transcoder…
Amazon RedShift
Big Data BigQuery
Google App Engine
PaaS (Platform as a Service)
Cloud DNS
Elastic Beanstalk
PaaS (Platform as a Service)
Amazon Route 53
PROs Focus on building applications instead of managing Infrastructure
Automated Pipeline Demo – Part 4
56
Code Build Test Release Deploy Monitor
✔ Collaboration ✔ Continuous Build ✔ Continuous Testing
✔ Continuous Delivery ✔ Automated Provisioning ✔ Automated Deployment ✔ Continuous Monitoring
Application and server monitoring
Business monitoring
Exception monitoring
Idea Customer
Sentry Plugin NewRelic Plugin SegmentIO Plugin
Demo – Part 5
57
Zero Downtime Deployment (blue/green)
40 minutes ago…
58
MVP running on locale dev machine (manual build pipeline)
2 Travis files & few Grails Plugins later…
59
“Unlimited” Horizontal Auto-Scaling (hundred of web servers)
Push-Button Scalable Database with Multi-AZ (up to 32vCPU and 244Go of RAM)
100% Resilient (no single point of failure)
Deployment in < 10 minutes (deployment pipeline)
100% Cloud-based (fully managed/no SSH)
CONCLUSION
60
Conclusion
61
Such an exciting time to be a developer AND an entrepreneur today…
Embrace Lean Embrace DevOps
Become an entrepreneur!
#lean #custdev
#agile #devops
#groovylang #grailsfw #aws
#PaaS #SaaS
Reading recommendations
62