devops in the public sector: how the democratic party implemented devops to migrate democrats.org to...

AWS Government, Education, and Nonprofit Symposium Washington, DC I June 25-26, 2015


DevOps in the Public SectorHow the Democratic Party Implemented DevOps to Migrate democrats.org to AWS

Leo ZhadanovskyNick Gaw

Sunil ChopraAlberto Manrique

©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved.



DevOps


What is DevOps?• « DevOps is the practice of operations and

development engineers participating together in the entire service lifecycle, from design through the development process to production support. »

- theagileadmin.com



Continuous Integration


What is continuous integration?• Changes to code automatically deployed to mainline

branch– After passing unit and mock tests

• Makes deployments and changes to code iterative, not monolithic

• Bugs are detected quickly• Helps automate deployments• Allows rapid development and deployment


SOURCE CODEREPOSITORY

PROJECT MANAGEMENT SERVER

CONTINUOUS INTEGRATION SERVER

DEVELOPER

PICKTASKS

SUBMITCODE

SCHEDULEBUILD

RECURRENTBUILDS

CODEFETCHCODE QUALITY

TESTS

TEST RESULTS

BUILD OUTPUT

DOCS

BINARIES& PACKAGES

DEV FACING NOTIFICATIONS

AWS CLOUDFORMATION

AMIS or CONTAINERS

Imm

utable In

frastru

cture

SOURCE CODE REPOSITORY

DNS

CONTINUOUS INTEGRATION SERVER


BUILDS


AWS code services

AWS CodeCommitComing soon

AWS CodePipelineComing soon

AWS CodeDeployLaunched Nov. 2014


Cloud software development lifecycle

10/13/14

MonitorProvisionDeployTestBuildCode

AWS Elastic Beanstalk

AWS OpsWorks

Amazon CloudWatch

AWS CloudFormation

?


Source controlPrivate Git repositories hosted on Amazon S3

• Full Git compatibility (use with existing tools)• All the benefits of the cloud

(scalable, durable, reliable, low pay-as-you-go pricing)

• No size limits on repositories (store binary files)• Online code tools with browse, edit, diff

10/13/14

Same Git experience$ git clone https://git-codecommit.us-east-1.amazonaws.com/v1/repos/aws-cliCloning into 'aws-cli'...Receiving objects: 100% (16032/16032), 5.55 MiB | 1.25 MiB/s, done.Resolving deltas: 100% (9900/9900), done.Checking connectivity... done.$ nano README.rst $ git commit -am 'updated README'[master 4fa0318] updated README 1 file changed, 1 insertion(+)$ git pushCounting objects: 3, done.Delta compression using up to 4 threads.Compressing objects: 100% (3/3), done.Writing objects: 100% (3/3), 297 bytes | 0 bytes/s, done.Total 3 (delta 2), reused 0 (delta 0)remote: To https://git-codecommit.us-east-1.amazonaws.com/v1/repos/aws-cli 4dacd6d..4fa0318 master -> master


Continuous deliveryCustomizable release automation, with integrated build and test

• Model and visualize custom release workflow(source build beta gamma prod)

• Automate builds, tests, and deployments• Enforce custom rules, approvals, and gates• Integrate with third-party and custom tools

10/13/14


DeploymentCoordinate software updates to fleets of EC2 instances

• Rolling updates for no downtime• Deployment health checks and easy rollback• Auto Scaling integration• Works with any application• Reuse existing setup tools

(Bash, Windows PowerShell, Chef, Puppet…)

10/13/14

9/25/14 Slides not intended for redistribution.

16


Cloud software development lifecycle

10/13/14

MonitorProvisionDeployTestBuildCode

Elastic Beanstalk

OpsWorks

CloudWatchCloudFormationCodeDeploy

CodeCommit CodePipeline

CODECOMMIT

DNS

CODEPIPELINE


BUILDS


Pain points• Unit tests incomplete• Mock tests maintenance• Expensive test environment• Test environment ≠ production• Deployment cycles


On-demand

Pay as you go

Elastic


=Programmable platform


If you can program ityou can automate it


A lot of options…• Configuration management systems

– Puppet– Chef– Saltstack

• Deployment frameworks– CodeDeploy– Elastic Beanstalk– OpsWorks– Ansible– Fabric– Capistrano

• Infrastructure management– CloudFormation

• Containers– Amazon EC2 Container Service


Bake an image Configure dynamically

Time-consuming configuration (startup time)

Static configurations (less change management)

Bootstrapping


Bake an image Configure dynamically

Continuous deployment (latest code)

Environment specific (dev-test-prod)

Bootstrapping



Stack-based deployment serviceAWS CloudFormation

CloudFormationtemplate

Headers

Parameters

Mappings

Resources

Outputs

{ "Description" : "Create RDS with username and password", "Resources" : {

"MyDB" : { "Type" : "AWS::RDS::DBInstance", "Properties" : { "AllocatedStorage" : "500", "DBInstanceClass" : "db.m1.small", "Engine" : "MySQL", "EngineVersion" : "5.5", "MasterUsername" : "MyName", "MasterUserPassword" : "MyPassword" } } }}

"AWS::CloudFormation::Init" : { "config" : {

"packages" : { "yum" : { "mysql" : [], "mysql-server" : [], "httpd" : [], "php" : [], "php-mysql" : [] } }, "sources" : { "/var/www/html" : "https://s3.amazonaws.com/my-builds/build-v4.zip" } }

{ "Parameters" : { "KeyName" : { "Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instance", "Type" : "String" } },}

CLOUDFORMATIONTEMPLATE

Proceduraldefinition

Create it programmatically

Known configurationStore stack configuration in source control

Parameter drivenDynamic and user-driven

templates

CollaborationShare templates with ease as just files


Application versions

+Infrastructure versions

CLOUDFORMATION TEMPLATE


Continuous deploymentSmall, frequent changes constantly

integrating into production


Key = iteration


Iteration=

Modify the system to better meet the expectations of your users


11.6 sec.Mean time between

deployments (weekday)

1,079Maximum number of

deployments in a single hour

10,000Mean number of

hosts simultaneously receiving a

deployment

30,000Maximum number of hosts simultaneously

receiving a deployment

Deployments at Amazon.com

(in 2011)


Software deploy

≠Product launch


Data-drivenarchitectures

Metrics @ Etsy

Metrics @ Obama for America


Metrics and monitoring options

CloudWatch

… and many more


Continuous integration

Continuous deployment



=

Continuous experimentation



=

Continuous improvement


INNOVATE


Speed and agility

Experiment often

Fail quickly at a low cost

More innovation

Experiment infrequently

Failure is expensive

Less innovation

On-premises



Democrats.org in AWS

Nick Gaw

Alberto Manrique

Sunil Chopra


Thank You.This presentation will be loaded to SlideShare the week following the Symposium.

http://www.slideshare.net/AmazonWebServices


devops in the public sector: how the democratic party implemented devops to migrate democrats.org to...

Technology

aws government

nonprofit symposium

aws codepipeline

aws codedeploy

code iterative

git experience

git compatibility use

git clone https