devops in practice: when does "practice" become "doing"?
Post on 14-Dec-2014
2.612 Views
Preview:
DESCRIPTION
TRANSCRIPT
© 2013 IBM Corporation
Accelerating Product and Service Innovation
DevOps in Prac.ce: When does "Prac.ce" Become "Doing"?
Michael Elder, IBM Senior Technical Staff Member mdelder@us.ibm.com hFp://linkedin.com/in/mdelder @mdelder
© 2013 IBM Corporation
© 2013 IBM Corporation
"Success is not delivering a feature; success is learning how to solve the customer’s problem.” - Mark Cook
© 2013 IBM Corporation
Examples of DevOps and Con.nuous Delivery
4
http://nflx.it/1dAJEBs http://slidesha.re/1mXJ6Mo
© 2013 IBM Corporation
Accelerate software delivery
Balance speed, cost, quality and risk
Reduce time to customer feedback
People Process Technology
Develop/Test
Deploy
Operate
Steer
IBM DevOps Enterprise capability for con.nuous soMware delivery that enables clients to seize market opportuni.es and reduce .me to customer feedback
© 2013 IBM Corporation
SoMware delivery challenges
Failures due to inconsistent dev and produc.on environments
Bo+lenecks trying to deliver more
frequent releases to meet market demands
Complex, manual, processes for release lack
repeatability and speed
Poor visibility into dependencies across releases, resources,
and teams
Daily Build
Monthly Delivery
Who did this last time?
Dave…
Dave’s not here man…
Dev
Prod
© 2013 IBM Corporation
High performing teams adopt DevOps
Reference: 2013 State of DevOps Report by PuppetLabs
© 2013 IBM Corporation
About your philosophy
Culture of con.nual experimenta.on and learning
• Produc.on like environments • Fully automated deployments • Accelerated delivery cycles
Build – Measure -‐ Learn “Success is not delivering a feature; success is learning how to solve the customer’s problem.” http://bit.ly/KM4JlQ
© 2013 IBM Corporation
Increase the opera&onal awareness of your so/ware earlier in the development process.
What does DevOps mean to me?
How do you do that? § Architecture
§ Automated Tes6ng
§ Automated Deployments
§ Produc6on-‐Like Environments
§ Automated Release Promo6on Process
§ Version control of all soBware, automa6on, and configura6on Why do this?
Because faster feedback loops enable rapid evolu&on of ideas and therefore faster itera6ons of your soBware
© 2013 IBM Corporation
“Cool factor” .. seen as pushing to the “next big thing” in our industry
BeLer communica6on between those who create and those who operate (same people in some cases)
Reduced fear of breaking the build/deployment/environment
Fail small before you “fail all”
Heavy focus on experimenta6on and learning
Improve the speed of your feedback loop to enable rapid evolu6on of ideas
What are the characteristics of teams who practice DevOps?
© 2013 IBM Corporation
§ Architecture § Automated Tes.ng
§ Automated Deployments
§ Produc.on-‐Like Environments
§ Automated Release Promo.on Process
§ Version control of all soMware, automa.on, and configura.on
About our next topics
© 2013 IBM Corporation
About your architecture
§ Architecture should support DevOps principles such as staged roll out, opera.onal insights, and scriptability
§ Each resource provides some very prac.cal advice for building systems which are focused on reliability and feedback loops
Experiment!: Website conversion rate
op.miza.on with A/B and mul.variate tes.ng
Release It!: Design and Deploy Produc.on-‐Ready SoMware
http://netflix.github.io/#repo http://slidesha.re/1mXJ6Mo
© 2013 IBM Corporation
Design for availability, con.nuous deployment
¡ Change your perspec.ve: as we move into hosted offerings, scalability and availability are cri.cal to our business
¡ Ne`lix Run.me Libraries provide capabili.es you can adopt in your products today – regardless of whether you’re building a hosted service
¡ The Libraries promote good development prac.ces and architectural paFerns
See the talk Andrew Spyker and I gave on this: hFp://slidesha.re/1mXJ6Mo
© 2013 IBM Corporation
Moving from monolithic applica.ons to micro-‐services
14
Monolithic app Micro services
Scaling Scaling
© 2013 IBM Corporation
¡ Compartmentalized business capability
¡ Cross-‐func.onal teams ¡ Communica.on via API ONLY!! ¡ Use messaging to remove peer-‐to-‐
peer dependencies ¡ REST communica.on ¡ Decentralized data ¡ Design for failure ¡ Pluggable architecture ¡ Enables con.nuous delivery
Proper.es of a micro-‐service architecture
15
© 2013 IBM Corporation
Leveraging Hystrix’s built-‐in event stream
© 2013 IBM Corporation
About automated tes.ng
¡ This is THE HARDEST part of the whole experience ¡ NO ONE is great at this, so don’t get discouraged ¡ Facebook’s process for code review include stats on how many tests are consistently broken to decide whether the test failures are noteworthy
¡ Con.nuously improve your coverage, flexibility, and discipline
¡ Consider all aspects of tes.ng: ¡ Programma.c API (JUnit) ¡ REST API (HTTPUnit, REST Fuse) ¡ Web UI (Selenium)
¡ Behavior driven tes.ng (Cucumber) ¡ Performance tes.ng (Apache JMeter) ¡ Security tes.ng (Ra.onal AppScan)
© 2013 IBM Corporation
Embedding unit tests in your classes directly
¡ Promoted approach by Ne`lix to reduce fric.on and introduces limited addi.onal bytes rela.ve to third party libraries
¡ Makes it easy to write commands, the UnitTest becomes the test harness and verifica.on
¡ Always testSuccess() and testFailure() use cases to ensure expected behavior
import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertNotNull; public class OSDeployEnvironmentCommand extends OSAbstractOrchestrationCommand<Response> { public static class UnitTest { @Test public void testSuccess() throws InterruptedException, ExecutionException, JSONException { … Future<Response> request = new OSDeployEnvironmentCommand(…) .queue(); Response response = request.get(); assertNotNull(response); assertEquals(HttpStatus.SC_OK, response.getStatus()); } }
© 2013 IBM Corporation
Tes.ng Your Deployed Applica.on – REST API
¡ We’re experimen.ng with REST Fuse which is an Eclipse project
¡ Captures REST calls as annota.ons, freeing up your test method to just verify behavior
¡ Requires a liFle “hoop jumping” for tes.ng a series of calls (e.g. POST .. PUT .. GET)
import static com.eclipsesource.restfuse.Assert.assertCreated; import ...restfuse.annotation.Authentication; import ...restfuse.annotation.HttpTest; import ...restfuse.MediaType; @HttpTest (method = com.eclipsesource.restfuse.Method.POST, contentMethod="getBlueprintRequestBody", authentications = { @Authentication( type = BASIC, user = "{keystoneUser}", password = "{keystonePassword}") }, path = "/rest/blueprint", type = MediaType.APPLICATION_JSON ) public void testCreateBlueprintOK() throws Exception { assertCreated(response); }
© 2013 IBM Corporation
Tes.ng Your Deployed Applica.on – Web UI
¡ Selenium is the only “apparent” choice that we’ve come across that’s effec.ve
¡ Couples automated test to some details of your web UI and must be kept in sync
¡ We’ve developed a few best prac.ces around loca.on elements for maximum flexibility, could publish this if there’s interest
public class EditorsPage extends UCDPPage { … @FindBy(id = "new-blueprint-button") private WebElement newButton; … public void createFile(String name, String id, String[] values) { newButton.click(); util().waitForElementPresentAndVisible( By.className("new-file-dialog")); WebElement widget = util().findElement( By.className("new-file-dialog")); Dialog dialog = Widget.init(this, widget, Dialog.class); dialog.setValues(values, true, true); dialog.clickPrimaryButton(); … } … }
© 2013 IBM Corporation 21
About automated deployments
Visibility and automated control of your applica6on deployment process
• Manage applica.on components and versions
• Manage configura.ons across all environments
• Offer secure ‘self-‐service’ capabili.es
• Increase transparency • Ensure governance and compliancy
© 2013 IBM Corporation
Applica.on model Reliably capture applica6on content and configura6on seRngs with versioning and traceability
© 2013 IBM Corporation
Applica.on environments
23
Define where components are deployed and capture configura6on seRngs per deployment environment for an applica6on
© 2013 IBM Corporation
Define automated processes for each applica.on component
24
Each component defines a series of steps, backed by plugin tasks or custom scripts, to deploy it
© 2013 IBM Corporation
About produc.on-‐like environments
¡ Use any and all means at your disposal
¡ Clouds can be helpful to provide “infinite” resources and drive the crea.on of new environments through automa.on ¡ See UrbanCode Deploy with PaFerns (Landscaper): hFps://ibm.biz/BdFc2X
¡ Or UrbanCode Deploy integra.on with Pure/SCO: hFps://ibm.biz/BdFc2E
¡ Vagrant (hFp://www.vagrantup.com) and Docker (hFp://www.docker.com) are good alterna.ves to clouds or to be used in combina.on with clouds
© 2013 IBM Corporation
DevOps and Cloud adop.on Automating for faster delivery with DevOps and Cloud
Networking
Storage
Servers
Virtualization
O/S
Middleware
Mid Config
Data
Applications
Infrastructure as a Service
UrbanCode Deploy
Networking
Storage
Servers
Virtualization
O/S
Middleware
Mid Config
Data
Applications
Pla@orm as a Service
IBM Bluemix
PureApplicaCon System
SmartCloud Orchestrator
SoFLayer
IBM DevOps Services
New
UrbanCode Deploy with Pa<erns
26
IBM Pa+erns
© 2013 IBM Corporation
About using clouds
¡ We’ve used the “FIT” environment based on the IBM PaFern Engine ¡ Long history of usage ¡ Lots of work to stabilize ¡ S.ll impacted by performance of transferring images, etc
¡ We’re building soMware to work with OpenStack, but there isn’t (currently) a good stable OpenStack cloud internally ¡ Cloud availability limited and behind “jump boxes”
¡ Longer term, this will be our approach along with Heat Orchestra.on Templates (HOT) for our applica.ons
SmartCloud Orchestrator
IBM PureApplicaCon System
IBM UrbanCode Deploy with Patterns
© 2013 IBM Corporation IBM CONFIDENTIAL
OpenStack is a global collabora.on of developers & cloud compu.ng technologists working to produce an ubiquitous Infrastructure as a Service (IaaS) open source cloud compu.ng pla`orm for public & private clouds.
OpenStack is our strategic direc.on for cloud
Platinum Sponsors Gold Sponsors
Compute (core)Provision and manage large networks of virtual machines"
Dashboard (core)Enables administrators and users to access & provision cloud-based resources through a self-service portal."
Heat (core)orchestrates multiple composite cloud applications using templates"
Ceilometer (shared service)Collect monitoring, metering, and other measurements
Storage (core)Provision and manage block-based and object storage"
Network (core)Provision and manage network connectivity"
Identity (shared service)Unified authentication across all OpenStack projects and integrates with existing authentication systems."
Identity (shared service)Unified authentication across all OpenStack projects and integrates with existing authentication systems."
© 2013 IBM Corporation
Our internal cloud: “FIT”
¡ The Federated Integrated Tes.ng (FIT) cloud service provides Infrastructure as a Service capabili.es from IBM
¡ Very stable, reasonably fast (has improved drama.cally over .me)
¡ Resources are “leased” so some zones aren’t meant for long running usage
¡ Great for automated deployment and tes.ng of builds ¡ CLM has been doing this against FIT
for a long .me
© 2013 IBM Corporation
About using Vagrant
¡ Some challenges with networking across pla`orm, par.cularly if you’re trying to use it to “run a cloud” such as OpenStack *within* the VM
¡ But for most applica.ons, provide a simple way to share a common infrastructure configura.on within a team or community
http://www.vagrantup.com
© 2013 IBM Corporation
About using Docker
¡ Create one or more “Dockerfiles” capturing the set of steps to setup your applica.on, middleware, and configura.on
¡ Build the “container” for each build
¡ Ship the “container” along your delivery pipeline
¡ IBM is ac.ve in this community – recently giving a keynote at Dockercon
http://www.docker.com
© 2013 IBM Corporation
Faster, repeatable, reliable full-‐stack deployment processes
SCM Build Automation
QA PROD
Execute application deployment and manage settings across environments
Publish build
Pull changes
SmartCloud Orchestrator
Environments DEV DEV DEV DEV
IBM PureApplicaCon System
IBM UrbanCode Deploy
Cloud Provisioning
Deploy Automation
IBM UrbanCode Deploy with Patterns
32
Learn more at hFps://ibm.biz/BdFcmV
© 2013 IBM Corporation
Introducing IBM UrbanCode Deploy with Pa+erns
ü PaFern designer – Design open, full stack applica.on environments in a diagram or textual editor
ü Design once, deploy anywhere – Deploy full stack environments to mul.ple clouds
ü Environment lifecycle management
– Manage infrastructure change and easily apply changes to exis.ng environments
ü Delivery process automa.on
– Automated delivery process with integrated full stack environments
Design and deploy full stack applica6on environments for mul6ple clouds
IBM UrbanCode Deploy with Patterns
Application
Middleware Config
Middleware
OS Config
Hardware
Envi
ronm
ent
Blu
eprin
t
Learn more at hFps://ibm.biz/BdFc2X
© 2013 IBM Corporation
Introducing UrbanCode Deploy v6.1 with PaFerns
Version the blueprint using your source control repo (na.ve support for git)
Apply the new versions of the template to an exis.ng environment or provision an en.re new stack
Validate changes with a "canary" paFern to ensure correctness and detect problems earlier
Verify en.re stack before produc.on
Portable across clouds
New
© 2013 IBM Corporation 35
Run Your Apps The developer can chose any language runtime or bring their own. Just upload your code and go.
DevOps Development, monitoring, deployment and logging tools allow the developer to run the entire application
APIs and Services A catalog of open source, IBM and third party APIs services allow a developer to stitch together an application in minutes.
Cloud Integration
Build hybrid environments. Connect to on-premises systems of record plus other public and private clouds. Expose your own APIs to your developers.
Extend SaaS Apps
Drop in SaaS App SDKs and extend to new use cases (e.g,. Mobile, Analytics, Web) !
IBM Bluemix
35
© 2013 IBM Corporation
About your promo.on process
UrbanCode Deploy with Patterns
© 2013 IBM Corporation 37
Release & Deploy Automa.on Deployment of Applica6ons across Environments Applications are
managed for deployment
Components from the app
Specific versions of the Components’ artifacts are deployed
Compliancy keeps track of whether the actual versions matches the expected
Each Component knows its relevant deployment processes
Compare versions across environments and define Snapshots of known working sets of Components
View deployment requests against the environment
hFps://developer.ibm.com/urbancode/
© 2013 IBM Corporation
Configura.on Management -‐ Snapshots
Creating a Snapshot
Component Versions
1
1
2
2
3
3
321Web
Mid. Code
DB
Snapshot
3
2
1
Mid. Config 1 2 3 3
Snapshot Deployment
ENVSnapshot
3
2
1
WEB HOST
MID HOST
DB HOST
2
1
1
3
2
X
3 3 1
Contents of environments that pass tests • contain deployable version and configuration
Deployments are based on deltas They help with:
ü Automation, Audit, and Visibility
uDeploy
© 2013 IBM Corporation
Version applica.ons with Snapshots
39
Capture tested component versions and configura6on seRngs as a snapshot that is promoted to other environments for beLer visibility and control
© 2013 IBM Corporation
About version control
¡ All of your source code is likely already version controlled
¡ All of your automa.on scripts, configura.on files, tests, etc should also version controlled
¡ UrbanCode Deploy automa.cally understands versions of components from your build process, but also versions all changes to configura.on proper.es and automa.on processes
https://hub.jazz.net/create
© 2013 IBM Corporation
IBM DevOps Services for Bluemix Tools in the cloud for the cloud
Web IDE
Agile Planning
Delivery Pipeline
http://jazzhub.com
© 2013 IBM Corporation
Code review should be part of your versioning process
¡ Gerrit is a popular code review integra.on for git ¡ Used for all UrbanCode products
¡ Enables you to create “pre-‐review” verifica.ons which automate running tests before a human reviews the change
¡ Provides immediate, automated feedback about the quality of the change based on your test suites
© 2013 IBM Corporation
Every aspect of our product delivery should be driven from the principles of validated learning; no more anecdotes, only meaningful results from our fast cycle prototypes. - Michael D. Elder
© 2013 IBM Corporation
§ Architecture § Automated Tes.ng
§ Automated Deployments
§ Produc.on-‐Like Environments
§ Automated Release Promo.on Process
§ Version control of all soMware, automa.on, and configura.on
About this deck
mdelder@us.ibm.com hFp://linkedin.com/in/mdelder @mdelder
Download from hFps://ibm.biz/BdFc2b
© 2013 IBM Corporation 45
© Copyright IBM Corporation 2013. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, Rational, the Rational logo, Telelogic, the Telelogic logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.
www.ibm.com/software
top related