tech talk: the continuous delivery metrics that will make your pipeline fast and your apps better

World®’16

TechTalk:TheContinuousDeliveryMetricsThatWillMakeYourPipelineFastandYourAppsBetter

DanZentgraf – Sr.PrincipalConsultant– CATechnologies

DO5T20T

DEVOPS

2 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD

©2016CA.Allrightsreserved.Alltrademarksreferencedhereinbelongtotheirrespectivecompanies.

Thecontentprovidedinthis CAWorld2016presentationisintendedforinformationalpurposesonlyanddoesnotformanytypeofwarranty. The informationprovidedbyaCApartnerand/orCAcustomerhasnotbeenreviewedforaccuracybyCA.

ForInformationalPurposesOnlyTermsofthisPresentation


Abstract

Akeyprincipleofcontinuousdeliveryiscontinuouslyoptimizingyourapplicationdeliverypipelineforqualityandspeed.Thisreliesonadeepunderstandingofthevariedprocessesatdifferentstagesofyourpipelineandtheirpotentialtokilldeliverytimesandproductivity.

Inthissession,learnwhyyouneedbothresultsandperformancemetricstouncoverdeepinsightsthatdrivesubstantiallongtermperformanceincreases,notjustincrementalchange.Understandhowapplyingmetricsthroughoutthepipelinefundamentallyenablesfeedbackloopsacrossyourcontinuousdeliverytoolchain.ExploretheintegralroleofCAReleaseAutomationContinuousDeliveryEditionintrackingandapplyingsoftwaredevelopmentlifecyclemetricstouniquelyoptimizeyourprocessesandenvironments.

DanZentgraf

CATechnologiesSr.PrincipalConsultant,Presales


TopicsforToday

INTRODUCTION

METRICS

RELATEDCAWORLDSESSIONS

INSTRUMENTATION

VALUESTREAMMAPPINGCONCEPTS

KEYMEASUREMENTITEMS

1

2

3

4

5

6


CoreValuesofDevOpsMovement– “CAMS”

CultureCAMS

Automation

Measurement

Sharing


AutomationIsAwesome

THEREARE,HOWEVER,AFEWPRE-REQUISITESTOASUCCESSFULEFFORT

§ Wheredowestart?§ Howdoweknowwearemakingprogress?§ Howdoweknowwearegettingvaluefromthe

effort?Arewedoingtherightthings?

Let’sAutomateALLTHETHINGS!!!


WE

“IfYouCan'tMeasureIt,YouCan'tImproveIt. ”

–PeterDrucker

“Ineverguess.Itisacapitalmistaketotheorizebeforeonehasdata.Insensiblyonebeginstotwistfactstosuittheories,insteadoftheoriestosuitfacts.”

– Sir.ArthurConanDoyle

“Ifthestatisticsareboring,you’vegotthewrongnumbers.” –EdwardTufte

“Therearelies,damnlies,andstatistics.”

–MarkTwain


ResultsMetricsandPerformanceMetrics

Myth:ResultmetricswillimproveperformanceReality:Youneedboth&theymustbealigned

RESULTMETRICS

ShowtheRESULTofanActionorProcess

PERFORMANCEMETRICS

TrackandMeasuretheActionsandProcessesthatLEADTOtheResult


ContinuousDeliveryMetrics

THECDCLASSIC:“CycleTime”Actualworktimeonaniteminaprocess

DEVOPS:“LeadTime”Elapsedtimefromwhenrequestwasmadetodelivery

CycleTimeandLeadTimeareRESULTmetrics

PERFORMANCEmetricsarelesswelldefined§ Muchvariancebasedonindividualteamor

businesssituation§ Organizationalbarrierscreatechallenges


Where/WhattoInstrument

VALUESTREAMMAPPINGAleanmanufacturingorleanenterprisetechniqueusedtodocument,analyzeandimprovetheflowofinformationormaterialsrequiredtoproduceaproductorserviceforacustomer.

STEP STEP STEP STEP

CycleTime CycleTime CycleTime CycleTime

LeadTime

WaitTime

WaitTime

WaitTime


EnterpriseDeliveryProcessesAreComplex…Anygivenenvironmenthasmultipleinvolvedstreams/subprocesses

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

Validate

Deploy

Configure

Provision

PROMOTIONDECISION

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime


AndTheyMultiplyThroughthePipeline…

Dev/Int QA Prod

RequiresvisibilitytosecondleveltounderstandactualmixofCycle/Waittimetofindbottlenecks.§ Needthecoreautomationdriver+oversightinordertotrack

manualprocesseswiththeautomationacrossthewholepipeline§ THIS isthemetricsengineeringallofusmustdo!

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

Validate

Deploy

Configure

Provision

PROMOTIONDECISION

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

Validate

Deploy

Configure

Provision

PROMOTIONDECISION

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

Validate

Deploy

Configure

Provision

PROMOTIONDECISION

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime


KeyOperationalMetricsatEachPipelinePhase

Howlongtillyouhaveadeployablepackage?

TimeframesforkeyTasksperenvironment

§ Howlongtoprovision/configure/deploy/validate?

§ Failurerateateachstep

ReductioninmanualTasks

Reductioninmaintenanceofautomation

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime

Validate

Deploy

Configure

Provision

PROMOTIONDECISION

STEP STEP STEP STEP


LeadTime

WaitTime

WaitTime

WaitTime


KeyOperationalMetricsforQualityintheFlow

CodequalitymetricsFeaturecompleteness

§ Howfardoesachangegobeforebeingkickedoutofprocess§ Rateoffeedback-basedrework

Deliveryquality§ “SmokeTest”passagerateperenvironment§ Decreaseinunintendedbreakage/brokendependencyevents

ApplicationHealth/Usage§ Monitoringsystem/reliability§ Appfeaturetracking

YoucanonlyBUILDqualityin.Youcannottestqualityin.

Dev

QA

Deploy

Run


PipelineDeliversStatisticstoReleaseManagementDiscipline-specificANDOverallFlowExamples

AgileMetrics§ Velocity§ Burndown/burnup§ Leadtime

Requirements&Defects

ConcurrentTesting

Dev&Build

Pre-production

Production&Monitoring

CAReleaseAutomation+Continuous

DeliveryEdition

AgileRequirements

Designer

ServiceVirtualization

TestDataManagement

CodeQuality§ Coverage§ UnapprovedOpenSource§ AppSec Remediation§ CycleTime

Testing§ Dependencybreakagerate§ Stagetimeperphase§ Smoketestpassage§ CycleTime

ProductionMetrics§ MTTR§ Supportincidentrate§ Usagedata

UAT§ Rejection/Reworkrate§ Stagetimeperphase§ Smoketestpassage§ CycleTime


ApplyingMetrics->AnalyticsIdeasfromTheLab– KPIDashboard/ReportCard


YourPipelineMetricGoalsAre…§ UnderstandhowyourPerformanceandResultMetricsrelatetoeachother§ AnalyzeyourValueStream§ IdentifyyourInstrumentationpoints§ TrackbothEfficiencyandQualitymetrics§ Listentoyourmetricsandapplythelessons

SummaryAFewWordstoReview


RecommendedSessions

SESSION# TITLE DATE/TIME

DO5X07SJeweler’sMutualInsuranceStreamlinesComplexAppDeployw/CARA

11/17/2016at12:45pm

DO5X12SEchoStarTransformsComplexSoftwareReleasestoCustomerNetworkSetTopBoxeswithCARACDE

11/17/2016at01:45pm

DO5T19SGMFinancialBuildsaSustainable,Holistic,ContinuousDeliveryPractice

11/17/2016at04:30pm


MustSeeDemos

ModernizeAppDeliveryIntegratedCDTheater5- DOV501P

DeliverTestDataFasterTestDataManagerTheater5- DOV511P

DeliverBetterAppsServiceVirtualizationTheater5- DOV507P

OrchestrateYourReleaseReleaseAutomationTheater5- DOV513P


Stayconnectedatcommunities.ca.com

Thankyou.


DevOps– ContinuousDelivery

FormoreinformationonDevOps– ContinuousDelivery,pleasevisit:http://cainc.to/PiTFpu

tech talk: the continuous delivery metrics that will make your pipeline fast and your apps better

Technology