aws re:invent 2016: datapipe open source: image development pipeline (arc319)
TRANSCRIPT
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Patrick McClory, VP Public Cloud & Professional Services - Datapipe
Thursday, December 1st, 2016
Machines At The Scale Of…Machines
Datapipe Open Source:
Amazon Machine Image Builder Toolset
What to Expect from the Session
• Overview of Amazon Machine Image (AMI) application
• Deep-dive into hands-on learnings from working at scale
• Walkthrough of Bakery toolset
• Simple Demo (install to first build)
• Complex Scenario Walkthrough
Requirements Convergence
MANAGED SERVICES
Consistency
Volume Over Time
DEV OPS
Delivery Speed
Volume in Service
PROGRAMMATIC & SCALABLE
SOLUTION
No-bake
• Flexibility achieved via library
of user data and configuration
management scripts
• ‘Pay-per-install’ Bandwidth & time
• ‘Pretty consistent deploys
over time’ Package versions, etc.
• Always the latest version
Comparing Approaches – Per Instance Launch:
Fully-baked
• Flexibility achieved via a library
of AMI’s.
• Pay for engineering & storage Pre-work, EBS Snapshots
• Absolutely consistent deploys
time after time
• Always the version you built
BASE
AMI
OS
CONFIGURATION
APPLICATION
INSTALLATION
TESTING/
VALIDATION
IMAGE
CAPTURE
Packer
Provided
by AWS
Configuration
Management
Testing
What does it take to build an image?
• Our toolset intends to solving a problem
in a specific context.
• We very much want to get feedback and involvement
from the community, but want to be straightforward:
Pull Requests for technical issues are VERY welcome
Feedback on the design via Pull Requests to the design
documentation is welcome and we want to engage
in a conversation from there.
Feedback is Welcome!
Aligning to CI/CD Pipeline
SOURCE
CONTROL
CONTINUOUS
INTEGRATIONTEST/QA
CONTINUOUS
DELIVERY
RUNTIME
OPERATIONS
Bakery Workflow
Start
Ready
to go
NAMING AND
DESCRIPTIVE
DETAILS
CONFIGURATION
MANAGEMENTCI TOOLING CHOICE
AWS TARGET(S)
CONFIGURATION(S)
INITIALIZE REPO
LOCALLY