devops for ibm commerce
TRANSCRIPT
© 2016 IBM Corporation
DevOps for IBM CommerceMichaelD.ElderIBMDistinguished EngineerChiefDeveloper, UrbanCode
[email protected]@mdelder
2© 2016 IBM Corporation
PleaseNote
2
• IBM’sstatementsregardingitsplans,directions,andintentaresubjecttochangeorwithdrawalwithoutnoticeatIBM’ssolediscretion.
• Informationregardingpotentialfutureproductsisintendedtooutlineourgeneralproductdirectionanditshouldnotbereliedoninmakingapurchasingdecision.
• Theinformationmentionedregardingpotentialfutureproductsisnotacommitment,promise,orlegalobligationtodeliveranymaterial,codeorfunctionality.Informationaboutpotentialfutureproductsmaynotbeincorporatedintoanycontract.
• Thedevelopment, release, andtimingofanyfuturefeaturesorfunctionalitydescribed forourproductsremainsatoursolediscretion.
• Performance isbasedonmeasurementsandprojectionsusingstandardIBMbenchmarksinacontrolledenvironment.The actualthroughputorperformance thatanyuserwillexperience willvarydependinguponmanyfactors,includingconsiderationssuchastheamountofmultiprogrammingintheuser’sjobstream,theI/Oconfiguration,thestorageconfiguration,andtheworkloadprocessed. Therefore, noassurancecanbegiventhatanindividualuserwillachieveresultssimilartothosestatedhere.
3© 2016 IBM Corporation
• WhatproblemsdoDevOpsreleasemethodssolve?
• HowcanyoushiftyourculturetoanEnterpriseviewofDevOps?
• PrinciplesandtechniquesofDevOpsfromArchitecturetoAutomatedDeploymentstoVersionControl
Agenda
IBMConfidential
4© 2016 IBM Corporation
Failures duetoinconsistentdev and
productionenvironments
Bottlenecks tryingtodelivermorefrequentreleases tomeetmarketdemands
Complex,manual,processesforreleaselackrepeatabilityand
speed
Poorvisibilityintodependencies across
releases, resources,andteams
Softwaredeliverychallenges
DailyBuild
MonthlyDelivery
Whodidthis lasttime?
Dave…
Dave’snothereman…
Dev
Prod
5© 2016 IBM Corporation
HighperformingteamsadoptDevOps
Reference: 2013 State of DevOps Report by PuppetLabs
6© 2016 IBM Corporation
Aboutphilosophy
Cultureofcontinualexperimentationand
learning
• Productionlikeenvironments• Fullyautomateddeployments• Accelerateddelivery cycles
Build– Measure- Learn
“Successisnotdeliveringafeature; successislearninghowtosolvethecustomer’sproblem.”
7© 2016 IBM Corporation 7
IBMBluemix GarageMethod
PracticesProcesses or actions you & your team perform to achieve an outcome.
TracksA selected set of practices, tools, and toolchains to create a cloud native or cloud enabled solution
ToolchainComplementary & compatible tools that when used together create a flexible solution
Experts & ExperiencesListen to our Experts to learn how we transformed our business
Combines industry bestpracticesforDesignThinking, LeanStartup,AgileDevelopment,DevOps,and Cloudtobuildanddeliverinnovative solutions.
https://www.ibm.com/devops/method
9© 2016 IBM Corporation © IBM Corporation
Linkage to Bluemix Garage and Lab Services
Method Website – global deploymentUsage analyticsLinkage to Online tools and tutorialsLinkage to Tool Chains for tracksUser Login and profile management
- Tailored views by userMethod personalization & customizationeLearning & certificationsInnovative & new “Method-as-a-Service”
With consultingservices
Self service
Common/Shared:Method Content
Practices & Tracks
Asalargeenterprise, howdoIget2,000developers doingDevOpsonCloud?
GarageMethodforDevOpsTransformation@EnterpriseScale
10© 2016 IBM Corporation
Accelerate Innovation
ivDeliverCodeThink Run
Systems of engagement
Systems of Record
Enterprise Applications
Production
MostenterprisesneedacombinationofDevOpspatterns
10
API Catalog
Operational Excellence
ivDeliverCodeThink Run Production
Shared Services
Digital Applications
• Coordinatingdifferentteamsworkingatdifferentspeeds
• Balancing between agility,stability, andrisk
• DecoupleacrossAPIboundary,butmanagerelease processwithconsistent approach
11© 2016 IBM Corporation
ExpectationsofContinuousDeliveryUser Experience Today Emerging
Primary Workload Type Systems of Record (Transactional)
Systems of Engagement (+ Record)Big Data, Analytics, Mobile/Social Channels
Delivery Model Planned Incremental (DevOps)
Development and Operations Team Sizes 100s and costly 10s with built-in DevOps automation
Release Frequency Months to years Days to weeks, based on business opportunity
Integration Frequency Weeks Continuous
Infrastructure Deployment Days Minutes
Time to Value Planned Opportunistic
Operational Model Systems management Built into application, recovery-oriented, continuous availability
Service Sourcing Develop Consume and assemble (public and private)
Feedback Loop Periodic Continuous interaction with clients
12© 2016 IBM Corporation
Increasetheoperationalawarenessofyoursoftware earlierinthedevelopmentprocess.
WhatdoesDevOpsmeantome?
Howdoyoudothat?§ Architecture
§ AutomatedTesting
§ AutomatedDeployments
§ Production-LikeEnvironments
§ AutomatedReleasePromotionProcess
§ Versioncontrolofallsoftware,automation,andconfigurationWhydothis?
Becausefasterfeedbackloopsenablerapidevolution ofideasandthereforefasteriterationsofyoursoftware
ShiftLeft
13© 2016 IBM Corporation
“Coolfactor”..seenaspushingtothe“nextbigthing”inourindustryBettercommunication betweenthosewho createandthosewhooperate
(samepeopleinsomecases)Reducedfearofbreakingthebuild/deployment/environment
Failfastbeforeyou“failall”Heavyfocusonexperimentationandlearning
Improvethespeedofyourfeedbacklooptoenablerapidevolutionofideas
WhatarethecharacteristicsofteamswhopracticeDevOps?
14© 2016 IBM Corporation
§ Architecture
§ AutomatedTesting
§ AutomatedDeployments
§ Production-LikeEnvironments
§ AutomatedReleasePromotionProcess
§ Versioncontrolofallsoftware,automation,andconfiguration
Aboutournexttopics
15© 2016 IBM Corporation
Aboutyourarchitecture
• ArchitectureshouldsupportDevOps principlessuchasstagedrollout,operationalinsights,andscriptability
• Eachresourceprovidessomeverypracticaladviceforbuildingsystemswhicharefocusedonreliabilityandfeedbackloops
Experiment!:Websiteconversionrate
optimizationwithA/Bandmultivariatetesting
ReleaseIt!:DesignandDeployProduction-Ready
Software
16© 2016 IBM Corporation
CommoncomponentsinCommerceconfigurations
Management Center
IBMCommerce
WebSphere ND
Static Content
SwimLane1
SwimLane2www
Management Center
IBMCommerce
WebSphere ND
Static Content
Commerce Data
Content Mgmt System
Commerce Data
Content Mgmt System
17© 2016 IBM Corporation
CommonmanagedartifactsinCommerceconfigurations
www
Management Center
IBMCommerce
WebSphere ND
Static Content
Commerce Data
Content Mgmt System
.sql.sql
BusinessContent
.war
.earpartial.ear.xml
httpd.conf
iPhone7microsite.war
18© 2016 IBM Corporation
CommonmanagedartifactsinCommerceconfigurations
www
Management Center
IBMCommerce
WebSphere ND
Static Content
Commerce Data
Content Mgmt System
.sql.sql
BusinessContent
.war
.earpartial.ear.xml
httpd.conf
iPhone7microsite.war
SnapshotofRelatedAssetstobedeliveredasasingleunitofdeployment
19© 2016 IBM Corporation
Updates&RollbacksinProduction
Management Center
IBMCommerce
WebSphere ND
Static Content
SwimLane1
SwimLane2www
Management Center
IBMCommerce
WebSphere ND
Static Content
Commerce Data
Content Mgmt System
Commerce Data
Content Mgmt System
ApplyUpdates
20© 2016 IBM Corporation
Automatedtestingiscriticalforautomatedfeedback
• This isTHEHARDESTpartofthewholeexperience
• NOONEisgreatatthis,sodon’tgetdiscouraged
• Facebook’s processforcodereview include statsonhowmanytests areconsistently brokentodecidewhether thetestfailuresarenoteworthy
• Continuously improveyourcoverage, flexibility, anddiscipline
• Consider allaspects oftesting:
• ProgrammaticAPI(JUnit)
• RESTAPI(HTTPUnit, RESTFuse)
• WebUI(Selenium)
• Behaviordriventesting (Cucumber)
• Performancetesting (ApacheJMeter)
• Securitytesting (RationalAppScan &IBMQRadar)
21© 2016 IBM Corporation
Embeddingunittestsinyourclassesdirectly
• PromotedapproachbyNetflixtoreducefrictionandintroduces limited additionalbytes relativetothirdpartylibraries
• Makes iteasytowritecommands,theUnitTestbecomes thetestharnessandverification
• AlwaystestSuccess() andtestFailure() usecases toensureexpected behavior
importstaticorg.junit.Assert.assertEquals;importstaticorg.junit.Assert.assertNotNull;
publicclassOSDeployEnvironmentCommand extendsOSAbstractOrchestrationCommand<Response>{
publicstaticclassUnitTest {@TestpublicvoidtestSuccess() throwsInterruptedException,
ExecutionException,JSONException {…Future<Response>request=
newOSDeployEnvironmentCommand(…).queue();
Responseresponse =request.get();assertNotNull(response);assertEquals(HttpStatus.SC_OK,
response.getStatus());}
}
22© 2016 IBM Corporation
TestingYourDeployedApplication– RESTAPI
• WehaveusedRESTFusewhich isanEclipse project
• Captures RESTcallsasannotations, freeingupyourtestmethodtojustverifybehavior
• Requires alittle“hoopjumping”fortesting aseries ofcalls (e.g.POST..PUT..GET)
importstaticcom.eclipsesource.restfuse.Assert.assertCreated;import...restfuse.annotation.Authentication;import...restfuse.annotation.HttpTest;import...restfuse.MediaType;
@HttpTest (method=com.eclipsesource.restfuse.Method.POST, contentMethod="getBlueprintRequestBody",authentications={@Authentication(type=BASIC,user="{keystoneUser}",password="{keystonePassword}")},path="/rest/blueprint",type=MediaType.APPLICATION_JSON
)publicvoidtestCreateBlueprintOK()throwsException{assertCreated(response);}
23© 2016 IBM Corporation
BehaviorDrivenDevelopment&Testing
• Capturesnatural languagestatements using“Given”,“When”,“Then”clauses
• Eachline ismatched toasnippet ofcodebasedonregularexpression
• Complete codestatementsareexecuted tovalidatebehavior
• Bridgebetween businessgoalsandtechnical details
Scenario:Success– UploadclientdirectoryGivenIprovidevalidcredentialstotheapi clientAndIwanttousetheapi clienttomakerequestsWhenIuploadthedirectory”data/sample-app”withtheclientThenthefilesreturnedmatchestheuploaddirectory”data/sample-app"
24© 2016 IBM Corporation
TestingYourDeployedApplication– WebUI
• Selenium istheonly“apparent”choicethatwe’vecomeacrossthat’seffective
• Couples automated testtosomedetails ofyourwebUIandmustbekept insync
• We’vedeveloped afewbestpractices aroundlocationelements formaximumflexibility, couldpublishthisifthere’sinterest
publicclassEditorsPage extendsUCDPPage {…@FindBy(id="new-blueprint-button")privateWebElement newButton;…publicvoidcreateFile(String name,Stringid,String[]values){newButton.click();util().waitForElementPresentAndVisible(By.className("new-file-dialog"));WebElement widget=util().findElement(By.className("new-file-dialog"));
Dialog dialog=Widget.init(this, widget,Dialog.class);dialog.setValues(values, true,true);dialog.clickPrimaryButton();
…}…}
25© 2016 IBM Corporation
Automateddeployments• Visibility andautomatedcontrolofyourapplication deployment process
• Manageapplicationcomponents andversions
• Manageconfigurationsacrossallenvironments
• Offersecure‘self-service’capabilities
• Increasetransparency• Ensuregovernance andcompliancy
26© 2016 IBM Corporation
IBMUrbanCode forContinuousDelivery
Drive down costReduce amount of manual labor, resource wait-time, and rework by eliminating errors & providing self-service environments
Speed time to marketIncrease frequency of software delivery through automated, repeatable deployment processes across development, test and production
Reduce riskRobust configuration management, coordinated release processes, audits, and traceability
Enabling clients to continuously deliver applications with high quality and low risk
IBM UrbanCode Build is a continuous integration and build management server optimized for the enterprise, designed to make it easy to scale the configuration and management of build infrastructure and seamlessly plug in to development, testing and release tooling.
IBM UrbanCode Deploy enables hybrid cloud by automating the deployment of applications, databases, configurations and containers into development, test and production on prem or in the cloud, helping to drive down cost, speed time to market with reduced risk. With full-stack environment management and deployment automation, users to design, deploy and update full-stack environments for multiple clouds, including Bluemix, Softlayer, AWS, Azure, and virtualized environments.
IBM UrbanCode Release is an intelligent collaboration release management solution that replaces error-prone manual spreadsheets and streamlines release activities for application and infrastructure changes.
27© 2016 IBM Corporation
Applicationmodel
Reliablycaptureapplicationcontentandconfigurationsettingswithversioningandtraceability
28© 2016 IBM Corporation
Applicationenvironments
Definewherecomponentsaredeployedandcaptureconfigurationsettingsperdeploymentenvironmentforanapplication
29© 2016 IBM Corporation
Defineautomatedprocesses foreachapplicationcomponent
Eachcomponentdefinesaseriesofsteps,backedbyplugintasksorcustomscripts,todeployit
30© 2016 IBM Corporation
Production-LikeEnvironments
PossiblyoneofthemostimportantprinciplesofDevOpsShiftleftanyoperationalconcernsthatleadto
differencesfromDEVtoPROD
https://hub.docker.com/u/ibmcom/#
31© 2016 IBM Corporation
DeploymentofApplicationsacrossEnvironmentsApplicationsaremanagedfordeployment
Componentsfromtheapp
SpecificversionsoftheComponents’artifactsaredeployed
Compliancykeepstrackofwhethertheactualversionsmatchestheexpected
EachComponentknowsitsrelevantdeploymentprocesses
CompareversionsacrossenvironmentsanddefineSnapshotsofknownworkingsetsofComponents
Viewdeploymentrequestsagainsttheenvironment
https://developer.ibm.com/urbancode/AutomatedReleasePromotionProcess
33© 2016 IBM Corporation
ConfigurationManagement- SnapshotsCreating a Snapshot
Component Versions
1
1
2
2
3
3
321Web
Mid. Code
DB
Snapshot
3
2
1
Mid. Config 1 2 3 3
Snapshot Deployment
ENVSnapshot
3
2
1
WEB HOST
MID HOST
DB HOST
2
1
1
3
2
X
3 3 1
Contents ofenvironments thatpasstests• containdeployable versionandconfiguration
Deployments arebasedondeltasTheyprovide Automation,Audit,andVisibility
34© 2016 IBM Corporation
VersionapplicationswithSnapshots
Capturetestedcomponentversionsandconfigurationsettingsasasnapshotthatispromotedtootherenvironmentsforbettervisibilityandcontrol
35© 2016 IBM Corporation
Updates&RollbacksinProduction
Management Center
IBMCommerce
WebSphere ND
Static Content
SwimLane1
SwimLane2www
Management Center
IBMCommerce
WebSphere ND
Static Content
Schemachangesmaybeharder torollback
App logicchangesjustrollback toaknown good versionorsnapshot
Commerce Data
Content Mgmt System
Commerce Data
Content Mgmt System
SnapshotofAssets
36© 2016 IBM Corporation
PromotionLifecylces
• Somechanges suchasapplication logicwill followacomplete cyclefromDEVtoPROD
• Othercontent-based changesmayonlyflowfromStagetoPROD
• Allsystem changesshouldflowthroughaconsistent,automated processwithappropriateapprovals inplace
AppLogic&Behavior(Example:Updatesperweekorpermonth)
Content-based changes toDatabase(Example:Nightlyupdates)
37© 2016 IBM Corporation
Aboutversioncontrol
§ Allapplication sourcecodeandconfigurationshouldbeversioncontrolled
§ Allofyourautomation scripts,configurationfiles, tests,etcshouldalsoversioncontrolled
§ UrbanCodeDeployautomatically understandsversions ofcomponents fromyourbuildprocess,butalsoversions allchanges toconfigurationpropertiesandautomation processes
38© 2016 IBM Corporation
Codereviewshouldbepartofyourversioningprocess
§ Defineandenforcecodingstandardsearly intheprocess
§ Automatedeployment toproduction-like systems aspartofautomatic verification
§ Eachcommitshould runautomated testingandcodecoverage,andthetoolchainshouldenforcecompliancebeforemergingorpromotingchanges
39© 2016 IBM Corporation
§ Architecture
§ AutomatedTesting
§ AutomatedDeployments
§ Production-LikeEnvironments
§ AutomatedReleasePromotionProcess
§ Versioncontrolofallsoftware,automation,andconfiguration
Aboutthisdeck
[email protected] http://linkedin.com/in/mdelder @mdelder
41© 2016 IBM Corporation41
©Copyright IBMCorporation 2013. All rights reserved. The information contained in these materials is provided for informational purposes only, and isprovided AS ISwithout warranty ofanykind, express orimplied. IBMshall notbe responsible for anydamages arising outof theuse of, orotherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have theeffectof, creatinganywarranties or representations from IBMor its suppliers or licensors, oraltering the terms andconditions of theapplicable license agreement governing theuse of IBMsoftware. References in these materialsto IBMproducts, programs, or services donot imply that theywill beavailable inall countries inwhich IBMoperates. Product release dates and/or capabilities referenced in these materials maychange atanytime atIBM’s sole discretion based onmarketopportunities orother factors, and arenot intended tobe acommitment to future product or feature availability inanyway. IBM, the IBM logo, Rational, theRational logo, Telelogic, theTelelogic logo, andother IBMproducts andservices are trademarks of the International Business Machines Corporation, in the United States, other countries or both.Other company,product, or service names maybe trademarks or service marks ofothers.
www.ibm.com/software