KMSBA22

DevelopmentofanOperationalProcessforContinuous

DeliveryAMajorQualifyingProjectReport

completedinpartialfulfillmentoftheBachelorofSciencedegreeatWorcesterPolytechnicInstitute,Worcester,MA

Submittedto:On-SiteLiaison: Mr.MichaelIrving Mr.JamesBermanProjectAdvisors:

ProfessorKevinSweeney,DepartmentofManagement ProfessorJonAbraham,DepartmentofMathematics ProfessorMichaelCiaraldi,DepartmentofComputerScience

ProfessorXinmingHuang,DepartmentofElectricalandComputerEngineering

InCooperationWith: BarclaysBankPLC

Submittedby:

KhazhismelKumykovComputerScienceandMathematicalSciences

TianLuo

ElectricalandComputerEngineering

AlexanderK.ShoopComputerScienceandActuarialMathematics

BorongZhang

MathematicalSciences

Date:January,2016

i

Abstract This project, sponsored by Barclays Bank PLC, focused on improving the continuous

deploymentprocessanddeveloping theguideline foronboardingnewapplications for thePrime

Financinggroup.WeidentifiedthatJenkinsisthebettertoolforPrimeFinancing,andexploredand

created a few plugins, which could enhance the use of Jenkins and provide management

information system details. Our recommendations aim to eliminate manual steps within the

continuousdeliveryprocessandsavetimeonadoptingnewapplications.

ii

Acknowledgement Our groupwould like to acknowledge everyone that helped to provide an opportunity to

takepartinthisprojectandthegreatsuccessthatresultedfromit.

First and foremost we would like to thankBarclays and specificallyMichael Irving for

supportinguswithourproject.

WewouldliketothankJamesBerman,TechnicalLeadVPofBarclaysEnhancedEnterprise

Reporting, for assisting us with many technical details regarding the project. We sincerely

appreciatethetimeofguidanceandsupportheprovidedusduringourtimeatBarclays.

WewouldalsoliketothankChristopherDarconte,PrimeServiceIT,forassistinguswith

manytechnicalrequestsandapprovals,andthankSubhashReddyBoreddyforassistinguswith

everythingrelatedwithJenkins.

We alsowould like to thankLouis Lu, Giridhar Manda, Ramkumar Sakthivel and Jeff

Johnston who volunteered their applications to do Jenkins implementations. We sincerely

appreciatetheirtimeandsupportduringthewholeimplementationprocesses.

Moreover,wewouldliketothankallBarclaysemployeeswe interviewedforproviding

relevantinformation.Theirknowledgeandexperienceweregreatlybeneficialtousunderstanding

CAReleaseAutomationandJenkins.

Additionally, we would like to thank our project advisors, Professor Kevin Sweeney,

ProfessorJonAbraham,ProfessorMichaelCiaraldi,ProfessorXinmingHuang,fortheirguidance,

support,advice,andassistancethroughouttheproject.

Lastly, we would like to extend our thanks to Worcester Polytechnic Institute for

providingthisprojectopportunity.

iii

AuthorshipContent PrimaryAuthorTitlePage Borong

Abstract Borong

Acknowledgements Borong

Authorship Borong

TableofContent Borong

TableofFigures Borong

ExecutiveSummary Borong

1.0Introduction Borong,Khazhy

2.0Background Borong

2.1Barclays Borong

2.2AgileSoftwareDevelopment Alex,Tian

2.3DevOps Borong,Tian

2.4ContinuousIntegration Tian

2.5ContinuousDelivery Borong

2.6ContinuousDeployment Tian

2.7CAReleaseAutomation Alex,Borong

2.8Jenkins Khazhy

3.0Methodology Borong

3.11stObjective Borong

3.22ndObjective Borong

3.33rdObjective Borong

3.4Summary Borong

4.0ResultsandAnalysis Borong

4.1ComparativeAnalysis Alex

4.2JenkinsPlugins Khazhy

4.3Reporting Alex,Khazhy

4.4Summary Borong

iv

5.0Conclusions Borong

5.1Conclusions Borong

5.2Recommendations Tian

5.3ImpactofOurProject Borong

AppendixA Tian,Alex

AppendixB Alex

AppendixC Alex

AppendixD Khazhy

AppendixE Alex

AppendixF Alex

AppendixG Alex

AppendixH Alex

AppendixI Alex,Khazhy,Ben

Allsectionswereeditedasateam,withequalcontributionsmadebyeverymember.

Appendix Iwaswritten andeditedbyAlex Shoop,KhazhyKumykov, andBenSharronas

partofour(notincludingTianandBorong)additionalMQPworkforourMathmajorrequirement.

v

TableofContentsAbstract...........................................................................................................................................................................................iAcknowledgement.....................................................................................................................................................................iiAuthorship...................................................................................................................................................................................iiiTableofContents.......................................................................................................................................................................vTableofFigures........................................................................................................................................................................viiTableofTables........................................................................................................................................................................viiiExecutiveSummary..................................................................................................................................................................11.0Introduction..........................................................................................................................................................................32.0Background...........................................................................................................................................................................52.1Barclays.............................................................................................................................................................................52.2AgileSoftwareDevelopment....................................................................................................................................62.2.1TheAgileManifesto.............................................................................................................................................72.2.2TwelvePrinciplesofAgileSoftware.............................................................................................................7

2.3DevOps...............................................................................................................................................................................82.3.1ComparisonofTraditionalITandDevOpsOrientedTeam.............................................................10

2.4ContinuousIntegration............................................................................................................................................112.5ContinuousDelivery..................................................................................................................................................122.6AutomatedDeployment...........................................................................................................................................132.7CAReleaseAutomation/Nolio..............................................................................................................................142.8Jenkins.............................................................................................................................................................................162.8.1JenkinsPromotedBuilds................................................................................................................................17

3.0Methodology......................................................................................................................................................................193.1Evaluatedadvantagesanddisadvantagesoftwodifferentcontinuousdeliverytools................193.2InvestigatedWorkflowImprovements.............................................................................................................203.3Generateddatareportsaboutcontinuousdeliveryprocesses...............................................................203.4Summary........................................................................................................................................................................20

4.0ResultsandAnalysis......................................................................................................................................................224.1ComparingTools.........................................................................................................................................................224.1.1InvestmentBank(IB)StandardToolandControlObjective..........................................................224.1.2LevelofSupport.................................................................................................................................................234.1.3OnboardingIntuitiveness...............................................................................................................................244.1.4DeploymentProcess.........................................................................................................................................244.1.5SampleImplementation..................................................................................................................................25

vi

4.2ImprovingWorkflows..............................................................................................................................................254.2.1AutomaticDeploymentWorkflow..............................................................................................................264.2.1.1PromotedBuildsPlugin..........................................................................................................................264.2.1.2BuildPipelinePlugin................................................................................................................................284.2.1.3DeliveryPipelinePlugin.........................................................................................................................29

4.2.2AutomatingJenkinsConfiguration.............................................................................................................304.2.2.1UsingJobDSLtoScriptJobCreation.................................................................................................314.2.2.2LimitationsofusingtheJobDSL/ExtendingJobDSL..............................................................314.2.2.3UsingJenkinsAPItoDuplicateJobs..................................................................................................334.2.2.4UsingGroovytoFetchSVNBranches...............................................................................................334.2.2.5Automatewhatyoucan–UsingScriptingtoautomateJobcreationforbranches......334.2.2.6JenkinsIntegrationwithPlugins........................................................................................................34

4.3GeneratingReports....................................................................................................................................................354.3.1DashboardViewPlugin...................................................................................................................................364.3.2DataExtractionPlugin.....................................................................................................................................364.3.2.1Prototype–IdentifyingDataExtractionMethod........................................................................364.3.2.2Prototype–ExtractingDatatoFile...................................................................................................384.3.2.3JavaExtractionToolPlugin...................................................................................................................384.3.2.4Outcomes......................................................................................................................................................40

4.3.3ReportingwithExcelPivotTable................................................................................................................414.4Summary........................................................................................................................................................................42

5.0ConclusionsandRecommendations.......................................................................................................................435.1Conclusions...................................................................................................................................................................435.2Recommendations.....................................................................................................................................................445.3ImpactofOurProject................................................................................................................................................44

References..................................................................................................................................................................................46AppendixA:GlossaryofTechnicalTerms...................................................................................................................49AppendixB:InstallingJenkinsPlugins..........................................................................................................................51AppendixC:NolioInstallationInstructions................................................................................................................55AppendixD:job_per_branch.groovy...............................................................................................................................57AppendixE:ExcelReportingusingSQLdatabase....................................................................................................62AppendixF:ExcelReportsusingCSVdata..................................................................................................................71AppendixG:Featuresofourpivottableconfiguration..........................................................................................72AppendixH:DashboardViewPlugin.............................................................................................................................76AppendixI:ExtensionofHighSpeedEquitiesTradingResearchStudy........................................................84

vii

TableofFiguresFigure2.1:ProcessofAgileSoftwareDevelopment(Mountain_Goat_Software,2005).............................6Figure2.2:TheIdeologyofDevOps...................................................................................................................................9Figure2.3:Hoursspenteachweekcarryingoutkeyactivities(Logan,2014)............................................10Figure2.4:BasicStructureofCISystem(Trimios,2012).....................................................................................12Figure2.5:LaunchuserinterfaceofCAReleaseAutomation.............................................................................14Figure2.6:ExampledashboardofCAReleaseAutomation’sROC....................................................................15Figure2.7:scenarioofdeployingandpromotinganapplication(CA_Technologies,2015a)...............15Figure2.8:JenkinsPromotedBuildsPlugin(Inman,2012)................................................................................17Figure4.1:TheDevOpsprocesswithapprovedtools............................................................................................23Figure4.2:(Left)Drag-and-dropactionsandflows(Right)Exampleshellscriptlog..............................25Figure4.3:JobPage(Left)andBuildPage(Right)ofPromotedBuildsPlugin...........................................27Figure4.4:PipelineUIofBuildPipelinePlugin.........................................................................................................28Figure4.5:PipelineUIofDeliveryPipelinePlugin..................................................................................................29Figure4.6:Multiplepromotionsonthesamebuild................................................................................................30Figure4.7:ConfigurationScreenforaSyncBranchesBuildStep.....................................................................35Figure4.8:ConfigurationScreenwithAdvancedOptionsandError-CheckingVisible...........................35Figure4.9:GlobalConfigurationScreen.......................................................................................................................35Figure4.25:JenkinsScriptConsole................................................................................................................................37Figure4.26:ConfigurationScreen...................................................................................................................................40Figure4.27:ErrorCheckingvalidatescredentialsandURL................................................................................40Figure4.28:DatainSQL......................................................................................................................................................41Figure4.29:PivotChartcreatedfromsampledata.................................................................................................42Figure7.1:Dashboardstatuswindow...........................................................................................................................50Figure7.2:PromotedBuildsPlugindescriptionpage............................................................................................51Figure7.3:ManageJenkins.................................................................................................................................................52Figure7.4:ManagePlugins.................................................................................................................................................52Figure7.5:Installinganavailableplugin......................................................................................................................52Figure7.6:Plugininstallationstatusscreen...............................................................................................................53Figure7.7:ChuckNorrisPlugin........................................................................................................................................53Figure7.8:Uploadingacustomplugin..........................................................................................................................54Figure7.9:ImportaSQLdatabasedata........................................................................................................................62Figure7.10:DataConnectionWizard............................................................................................................................62Figure7.11:Selecttheappropriatedatabase.............................................................................................................63Figure7.12:writeanameanddescriptionforthefile...........................................................................................63Figure7.13:Importdata(left)andpropertieswindow(right).........................................................................64Figure7.14:Importedtable...............................................................................................................................................65Figure7.15:FormatCells....................................................................................................................................................65Figure7.16:SummarizewithPivotTable.....................................................................................................................66Figure7.17:PivotTableFieldList....................................................................................................................................67Figure7.18:Customizerowlabels..................................................................................................................................68Figure7.19:Groupbutton(left)andgroupingwindow(right).........................................................................68Figure7.20:PivotChartbutton(left)andChartstyleselection(right)...........................................................69Figure7.21:Samplepivotchart.......................................................................................................................................69Figure7.22:Customizecolumnsandrows..................................................................................................................70Figure7.23:Samplechart...................................................................................................................................................70Figure7.24:raw.CSVdatasetinExcel..........................................................................................................................71

viii

Figure7.25:InsertPivotTablebutton(left)andcreatePivotTablewindow(right).................................71Figure7.26:Groupreportfilter........................................................................................................................................72Figure7.27:Promotionresultcolumnlabel...............................................................................................................72Figure7.28:Promotiontimerowlabel.........................................................................................................................73Figure7.29:Promotionnamerowlabel.......................................................................................................................74Figure7.30:Pivotchartwithtrendline........................................................................................................................75Figure7.31:AddanewviewwhenusingDashboardViewPlugin...................................................................76Figure7.32:Jenkinsjobslistunderdashboardview..............................................................................................77Figure7.33:Jobstatisticsunderdashboardview....................................................................................................77Figure7.34:BuildstatisticsunderDashboardView...............................................................................................78Figure7.35:JobsgridunderDashboardView...........................................................................................................78Figure7.36:UnstablejobsunderDashboardView..................................................................................................78Figure7.37:TeststatisticsunderDashboardView.................................................................................................79Figure7.38:TeststatisticschartunderDashboardView.....................................................................................79Figure7.39:TesttrendchartunderDashboardView............................................................................................80Figure7.40:DashboardViewwithdefaultportlets.................................................................................................80Figure7.41:Numberofbuildsplugin............................................................................................................................81Figure7.42:RMBuildTimesChart.................................................................................................................................81Figure7.43:LatestBuildswithBadges.........................................................................................................................82Figure7.44:samplecodeofMyPortlet.java(Hayes&Ambu,2015)................................................................83Figure7.45:samplecodeofportlet.jelly(Hayes&Ambu,2015)......................................................................83

TableofTablesTable7.1:WeatherStatus...................................................................................................................................................50Table7.2:Processareasandflows.................................................................................................................................55Table7.3:PorcessesandCategories...............................................................................................................................56

1

ExecutiveSummary Thetechnologiesandsoftwaredevelopmentstrategiesarevitalforthecompetitivenature

of most industries and organizations. To deliver new features and get feedback rapidly would

increase the reputation of an organization and furthermore lead the company in the specific

industry(Kim,2014).Inordertodoso,itisessentialtoreleasesoftwaremoreoftenwithlowrisk.

Tofulfillthisdemand,theconceptofAgileSoftwareDevelopmenthasbeenestablished.FromAgile,

DevOps, previously known as “Agile Operations” was born. DevOps is the collaboration of

developmentandoperationteamsthroughoutallstagesofthedevelopmentlifecycletoensurecode

qualityandtoreleasesoftwareatanygiventime(Mueller,Wickett,Gaekwad,&Karayanev,2011).

Continuous Integration toolsandContinuousDeliveryplatformshavebeencreated to implement

theDevOpsinitiative.

For Barclays Prime Financing, they have considered several initiatives to adopt an Agile

Software Development approach which would be widely pushed within the organization. Even

thoughsometeamsinPrimeFinancingareusingsomeformsofContinuousIntegrationtoolsora

Continuous Delivery platform, it would be better to standardize the toolset in order to manage

relatedapplications.Thestrengthsandweaknessesofeachtoolwerenotclear,soitwashardfor

PrimeFinancingtochooseatool.TheywereinvestigatingabetterDevOpstoolthatcouldbeused

as a standard tool in their team, and also standardized guideline for onboarding projects to the

specifictool.

Thegoalofourprojectwasto improveautomateddeployment in thecontinuousdelivery

process.Our first objectivewas to evaluate the advantages anddisadvantagesof two continuous

delivery tools: CA Release Automation and Jenkins Promoted Builds Plugin. We conducted

background research to understand concepts within DevOps and their importance. Through

2

interviewingseveralexperiencedemployees,wecollected the subjective responses regarding the

advantages and disadvantages of each tool. We also identified some objective strength and

weaknessesviasampleimplementationsforeachtool.Afterthat,wedidacomparativeanalysisto

reach our conclusion of each tool and provide recommendations. Our second objective was to

implementthreevolunteerapplicationsfromPrimeFinancing,whichenabledustocatalogdetailed

functionalities within the chosen tool.We discovered themissing functionalities of the tool and

providedoursuggestionstopromotetheuseofthetool.Theresearchandimplementationsguided

ustoformastandardizedguidelinefortheonboardingprocessforapplicationsontothetool.Our

finalobjectivewastogeneratedatareportsaboutthechosentool’scontinuousdeliveryprocesses.

WeretrievedrawdatafromJenkinsandmanipulatedtheminordertoshowusefulinformation.

We concluded that Jenkins satisfied Barclays Prime Financing’s requirements, but the

automateddeployment of its continuousdeliveryprocess canbe improved from several aspects.

WesuggestedadoptingJenkinsPromotedBuildsbasedonourcomparativeanalysisofJenkinsand

CARelease Automation.We developed an easily extensible Jenkins Plugin for automatic Jenkins’

configurationwithregardstojobcreationandcleanupfromSVNbranching.Moreover,wecreateda

management information systemplugin that can extract build and promotion information into a

SQL database and CSV files, and then import into a reusable Excel templatewe created to show

reports.Ourrecommendationscanhelptoimproveautomateddeploymentofcontinuousdelivery

processwithinPrimeFinancinggroupandtogeneratereportsofcontinuousdeliveryprocess.

3

1.0Introduction The technologies and software development strategies play essential roles in many

industriesandorganizations.Inordertoachievehigherperformanceintechnology,itisnecessary

toreleasesoftwaremoreoften.Todelivernewfeaturesandgetfeedbackfasterwouldincreasethe

competitivenatureofanorganizationand furthermore lead thecompany in thespecific industry

(Kim,2014).TheconceptofAgileSoftwareDevelopmenthasbeenestablishedtofulfillthisdemand,

andDevOpsdevelopednaturallyfromtheconceptsofAgileSoftwareDevelopment.DevOpsisthe

collaboration of development and operation teams throughout all stages of the development

lifecycle toensurecodequalityand to releasesoftwareatanygiven time Importantconcepts for

Agile and DevOps are Continuous Integration (CI) – the process of continuously merging and

“integration” the code base of different developers so as to avoid painwhen having tomanually

combinewidelydivergingcodebases,andthebugsassociatedwithcombiningchangesthathaven’t

beentestedwitheachother–andContinuousDelivery/ContinuousDeployment(CD)–theprocess

ofcontinuouslydeployingtheproducttotestand“productionlike”environments(andeventually

productionenvironments)andrunningappropriatetestsforallbuilds.InContinuousDeployment

everybuildthatpassestestswillbedeployedtoproduction,whereasinContinuousDelivery,every

build has the potential to be deployed to production, butmay only particular chosen builds are

actuallydeployed.

Barclays IB is currently adopting Agile, and as part of that initiative teams are adopting

CI/CD practices and automation tools. Alreadymost teams are using CI practices and tools, and

have processes to deploy to and test in Quality Assurance (QA) testing environments and

“productionlike”UserAcceptanceTesting(UAT)environments.However,mostteamsdonothave

automated deployment to Production environments, and often the actual deployment process

4

differs between testing and production environments which risks unknown factors impacting

deploymenttoProduction.

Fortheautomateddeploymentprocesstherearemanydifferentproductsthatmaybeused.

Within Barclays Prime Financing two specific tools were identified for evaluation – CA Release

Automation(formerlyknownasNolio),anautomateddeployment tool identifiedby thewider IB

organization as recommended, and Jenkins, a primarily continuous integration tool that has

extensionsthatenableautomateddeploymentandmanagingofacontinuousdeliveryprocess.The

strengthsandweaknessesofeachtoolwerenotmadeclear,soitwashardforPrimeFinancingto

chooseatool.BarclaysPrimeFinancingwasinvestigatingabetterDevOpstoolthatcouldbeused

asastandardtoolintheirteam,andalsodesiredtohaveastandardizedguidelinewhenonboarding

projectstothespecifictool.

The goal of our project was to standardize the guideline to adopt the selected tool on

applications. By conducting research, interviews with experienced employees and comparative

analysis,weconcludedthatadoptingtheJenkinsPromotedBuildsPluginisthebetteroptionforthe

team,whichalsotakesawaymanymanualsteps.Basedonourimplementationsofthreevolunteer

applications,weidentifiedpossibleimprovementstotheworkflowcurrentlybeingused.Moreover,

wegenerateddatareportsonJenkinsPromotedBuildsPlugin’scontinuousdeploymentprocesses

and managed them to show useful information. Our recommendations can help to improve the

automateddeploymentprocessoftheJenkinswithinPrimeFinancingandtodeveloptheguideline

forthegrouptointegratemorecandidateapplications.

5

2.0Background Nowadays,manymoreindustriesrelysignificantlyontechnologiesandsoftware.Forhigh

performing technological organizations, it is vital to release their software more frequently to

upgradenewfeaturesandreceivefeedbackfastertoplanforthenextdeployment(Kim,2014).In

ordertoreduceerrorsandspeedupdeployments,theconceptsofAgileSoftwareDevelopmentand

DevOpsaimtofulfillthetechnologydemands.ThemajorareastoimplementinregardstoDevOps

areContinuousIntegrationandContinuousDelivery,whichallowdeveloperstoensurecodequality

throughout the software life cycle and release software at anytime. Jenkins is the Continuous

IntegrationtoolbeingusedbymanyITteams,andCAReleaseAutomationistheBarclaysstandard

ContinuousDeliverytool.

2.1Barclays

Barclays Bank PLC, founded in 1690, is one of the largest multinational banking and

financial services companies in the United Kingdom, and its headquarter is in London. Barclays

providesdifferentfinancialservicesinretail,wholesale, investmentbanking,wealthmanagement,

mortgage lending and credit cards, which all operate in multiple countries across Europe, the

Americas,Asia,andAfrica.TherearefourcorebusinesssectorsofBarclays:Personal&Corporate

(Personal Banking, Corporate Banking, Wealth & Investment Management), Barclaycard,

InvestmentBankingandAfrica.BarclayshasaprimarylistingontheLondonStockExchangeanda

secondary listingon theNewYorkStockExchange.At theendof2011,Barclays’ assetsachieved

US$2.42trillion,anditrankedtheseventh-largestbankworldwide.

Their purpose is “helping people achieve their ambitions – in the right way”, and their

valuesare“respect,integrity,service,excellenceandstewardship”(Barclays,2015).Theirpurpose

andvaluesguidethemonhowtomeasureandrewardpeople,“notjustoncommercialresults,but

onhowtheyliveourValuesandbringthemtolifeeveryday.”

6

As for Barclays in New York, during the 2008 financial crisis, Barclays announced its

agreement topurchase the investmentbanking and tradingdivisionsof LehmanBrothers. In the

end,BarclaysPLCpaidUS$1.35billiontoacquirethecorebusinessofLehmanBrothers,including

Lehman’s US$960millionmidtownManhattan office skyscraper and the responsibility for 9,000

formeremployees.

2.2AgileSoftwareDevelopment

Agile softwaredevelopment is a set of softwaredevelopmentmethodsbasedon iterative

and incremental development, through software developer self-organizing, cross-functional team

communicationandcollaborationtocompletethedevelopmentwork(Highsmith,2002).

Figure2.1:ProcessofAgileSoftwareDevelopment(Mountain_Goat_Software,2005)

Softwaredevelopmentteamsnormallysettwotofourweeksperiteration,asseeninFigure

2.1(Mountain_Goat_Software,2015).Theyusuallyhavedaily scrummeetings,whicharebasically

standupmeetings,toensuretheyareonthesamepageinordertocontinuouslymakeprogress.

Agilemethodssometimesaremisunderstoodasunplannedandundisciplinedapproaches.

As a matter of fact, agile methods emphasize adaptability rather than predictability (Cohen,

Lindvall,&Costa,2003).Anadaptiveapproach focusesonquickly adapting to changing realities.

Whentherequirementsofaprojectchange,theadaptiveteamshouldchangeaswell.However, it

mightbedifficultfortheteamtopredictwhatwouldhappeninthefuture.

7

Agiledevelopment,anewdevelopmentmodeltoavoidtheshortcomingsofthetraditional

waterfall model, emphasizes on delivering features as soon as possible and doing continuous

improvementsandenhancementsthroughouttheprojectcycle,whichhelpstheteamcontinuously

adaptitsplanssoastomaximizethevalueitdelivers(Highsmith&Cockburn,2002).

Compared to iterative and incremental development methods, agile methods put more

emphasisonhighcollaborationoftheteam,whichinturncancreateashorterdevelopmentcycle.

Bothsoftwaredevelopmentmethodsemphasizeshippingsoftwareinarelativelyshortcycle.

2.2.1TheAgileManifesto

Weareuncoveringbetterwaysofdevelopingsoftwarebydoingitandhelpingothersdoit.

Throughthisworkwehavecometovalue(Becketal.,2001)1:

IndividualsandinteractionsoverProcessesandtools

WorkingsoftwareoverComprehensivedocumentation

CustomercollaborationoverContractnegotiation

Respondingtochangeoverfollowingaplan

Thatis,whilethereisvalueintheitemsontheright,wevaluetheitemsontheleftmore.

2.2.2TwelvePrinciplesofAgileSoftware

ThefollowingareprinciplesbehindtheAgileManifesto(Becketal.,2001):

Ourhighestpriorityistosatisfythecustomerthroughearlyandcontinuousdeliveryof

valuablesoftware.

Welcomechangingrequirements,evenlatein development.Agileprocessesharnesschange

for thecustomer'scompetitiveadvantage.

1 KentBeck,JamesGrenning,RobertC.Martin,MikeBeedle,JimHighsmith,SteveMellor,ArievanBennekum,AndrewHunt ,KenSchwaber,AlistairCockburn,RonJeffries,JeffSutherland,WardCunningham,JonKern,DaveThomas,MartinFowler,BrianMarick©2001,theaboveauthors.Thisdeclarationmaybefreelycopiedinanyform,butonlyinitsentiretythroughthisnotice.

8

Deliverworkingsoftwarefrequently,froma coupleofweekstoacoupleofmonths,witha

preferencetotheshortertimescale.

Businesspeopleanddevelopersmustwork togetherdailythroughouttheproject.

Buildprojectsaroundmotivatedindividuals.Givethemtheenvironmentandsupportthey

need, andtrustthemtogetthejobdone.

Themostefficientandeffectivemethodof conveyinginformationtoandwithina

development teamisface-to-faceconversation.

Workingsoftwareistheprimarymeasureofprogress.

Agileprocessespromotesustainabledevelopment. Thesponsors,developers,andusers

shouldbeable tomaintainaconstantpaceindefinitely.

Continuousattentiontotechnicalexcellence andgooddesignenhancesagility.

Simplicity--theartofmaximizingtheamount ofworknotdone--isessential.

Thebestarchitectures,requirements,anddesigns emergefromself-organizingteams.

Atregularintervals,theteamreflectsonhow tobecomemoreeffective,thentunesand

adjusts itsbehavioraccordingly.

2.3DevOps

DevOpsisasoftwaredevelopmentideologyconcentratedoncommunication,collaboration

and integration among software development, technology operations and quality assurance

(Mueller et al., 2011). Since the software industry has recognized the necessity to integrate

9

developersandoperationsengineers in theentireservice lifecycle, fromdesign toproductionas

seeninfigure2.2,DevOpsisaimedathelpingorganizationsrapidlyenhancesoftwareproductsand

services.

Figure2.2:TheIdeologyofDevOpsGenerally, releasing an application is risky and relates to multiple development teams.

However, in an organization that uses DevOps tools, the risk of application release decreases

significantlyforthefollowingreasons(InfoQ,2014):

1. Reducingthescopeofchange

Compared to the traditional waterfall development model, using agile or iterative

development means more frequent releases and each release contains fewer changes.

Thankstothehighfrequencyofdeployment,eachsingledeploymentwouldnothaveahuge

impactontheproductionsystem.Asaresult,theapplicationwillgrowsmoothly.

2. Strengthenthecoordinationofrelease

It isgoodtohaveastrongcoordinationtobridgeskillsandcommunicationgapsbetween

developmentandoperationteams;onemustensureallresponsiblepersonnelunderstand

the content changes and cooperatewell through electronic data sheets, teleconferencing,

instantmessaging,enterpriseportals(wiki,SharePoint)andothercollaborationtools

3. Automation

10

Powerful automated deployment tools ensure the repeatability of task and reduce the

possibilityofdeploymenterrors.Thisprocess takesawaythe tremendous timeandeffort

requiredformanualdeployment.

2.3.1ComparisonofTraditionalITandDevOpsOrientedTeam

Tounderstandthedifferencesbetweentraditional ITandDevOpsorientedteam, itwould

beessentialtoinstillaDevOpsorientedculturewithinanorganization(Logan,2014).Theauthorof

theArticle“FreshStatsComparingTraditionalITandDevOpsOrientedProductivity”surveyed620

engineers to accomplish this comparison. The survey asked the time spent on improving

infrastructure,settingupautomationforrepeatabletasks,fightingfires,communicationandsoon,

theresultsofwhichareshowninfigure2.3.

Figure2.3:Hoursspenteachweekcarryingoutkeyactivities(Logan,2014)Thisresultedinseveralconclusionsasfollows(Logan,2014):

• DevOpsorientedteamsspendslightlymoretimeautomatingtasks

• BothtraditionalITandDevOpsorientedteamscommunicateactively

11

• DevOpsorientedteamsfightfireslessfrequently

• DevOpsorientedteamspendlesstimeonadministrativesupport

• DevOpsorientedteamsworkfewerdaysafter-hours

2.4ContinuousIntegration

Continuous Integration (CI) is a software engineering practice, used by many software

development teams, to ensure codequality throughout the softwarebuild life cycle.Thepractice

wasfirstnamedandproposedin1991byGradyBoochandwasadoptedbyExtremeProgramming

(XP), a type of Agile Software Development (Wikipedia, 2015b). The practice was suggested to

avoidlargeintegrationissuesthatarisewhenintegratinglargesetsofchangeswitheachother.In

XPaswellCIwasintendedtobeusedalongwithautomatedunittests,whichshouldbedesignedto

ensure functionality.Themajor roleofCI is to controlandreact toproblems immediately. It isa

practicedesignedtoeaseandstabilizesoftwaredevelopmentprocesses.

CIwouldbringthefollowingbenefits(Martin,2006):

1. SoftwareBuildAutomation:Once a configuration is completed, a CI system can build the

targetsoftwareinaccordancewithapre-establishedschedule.

2. SustainableAutomatedInspection:ACIsystemcancontinuouslyobtainaddedormodified

source code.When the software development teams need to periodically check added or

modifiedcode,theCIsystemcancheckwhethertheaddedcodecoulddestroytheoriginal

softwarebuild.

3. SustainableAutomatedTesting:Oncethecodeisbuilt,pre-establishedtestswouldrun,and

theCIsystemwouldautomatically triggerreal timenotificationsviaRSS,Emailor Instant

Messagingtoappropriatedevelopers.

4. AutomatedFollow-upProcesses:Aftertheautomatedinspectionandtestscomplete,there

may be some additional required tasks in the software build cycle, including generated

12

documents,packagedsoftware,anddeployedcomponents,inordertoimprovethespeedof

releaseforproject.

Figure2.4:BasicStructureofCISystem(Trimios,2012)Thefigure2.4showstheworkflowofaCIsystem,whichwouldreactandreflectthe

build problem or integration problem. The main role of the CI system in the whole

developmentprocessiscontrol:whenthesystemdetectsachangeinthecoderepository,it

will entrust the task of running build to the build process itself. If the build fails, the CI

systemwillnotifytherelevantpersonnel,thencontinuetomonitortherepository.Whileit

seemspassive,theCIsystemdoesrevealproblemsimmediately.

2.5ContinuousDelivery

ContinuousDelivery(MacDonald,2005)isasoftwareengineeringapproachforautomating

softwarereleaseatanytime.CD isanextensionofCI; ituses theconceptsand frameworksofCI,

unit testing, and acceptance testing to streamline softwaredelivery (Wikipedia, 2015a). CD is an

importantpartoftheDevOpsideology.InCDcodeiscompiledandtestedeverytimeitischanged,

and is frequently deployed to test environments and run through automated acceptance tests.

Moreover, code may pass through manual acceptance tests as well. Only code that passes all

13

automated andmanual tests/approvals ismarked as “releasable,” ensuring that even though the

systemautomatesmuchoftheprocess,codeisonly“released”onceitisensuredtobevalid.

Continuous Delivery allows anyone to receive fast, automated feedback regarding any

changesmadetotheircode,andfurthermorepeoplecanperformpush-buttondeploymentsofany

versiontoanyenvironment(Farcic,2014).

2.6AutomatedDeployment

Automateddeploymentmeansautomaticallydeployingeverychangetoproduction,andthe

main concept of automated deployment is deployment automation (Smith, 2015). Deployment

automationisasolutiontodeliverapplicationsfasterandmoreefficiently(XebiaLabs,2015).Even

thoughtherearesomedifficulties in implementingthe fullyautomateddeploymentprocess,such

as the overhead of creating, setting up, configuring and maintaining an automated deployment

mechanism, the benefits of using automated deployment are significant in comparison to the

difficulties.

Automated deployments make the application deployment processes become much less

error-prone and much more repeatable, which eliminates manual steps and avoids delivering

incorrectversions(XebiaLabs,2015).Oncetheautomateddeploymentprocessisset,ifitworksthe

firsttime,itwillworkmanymoretimesafterwards.Theknowledgeoftheautomateddeployment

processiscapturedinthesystem,independentfromanindividual,andthusanyoneintheteamcan

deploy software. Furthermore, rather than spending time on performing and validating manual

deployments,thesoftwaredevelopmentteamscanspendtheirtimeondevelopingthenextsetof

qualityfeaturesandenhancementstothesoftware.Thetargetenvironmentsandmachinescanbe

changedeasily,whichsimplyrequiresconfiguringtheexistingsetupandthenrelyingonthenew

releaseautomation.Thesystemallowsfrequentreleasesinordertoshipvaluablefeaturestousers

moreoftenandtogetcontinuousfeedbacktoenhancethesoftware.

14

2.7CAReleaseAutomation/Nolio

CA Release Automation, formerly known as Nolio, is an enterprise-class, multi-release,

continuous delivery solution which automates complex release deployments (CA_Technologies,

2015b).Thissoftwareisaimedtoeaseandstreamlinethedeploymentprocess.

CA Release Automation enables to design, manage, and automate application-centric

operations across physical, virtual, and cloud environments. It can speed up application release

cycles, reduce errors, and achieve higher quality releases. The software also can create more

frequentreleases,reducecostsandpromotecollaborationandalignmentbetweendevelopersand

operationengineers.

Figure2.5:LaunchuserinterfaceofCAReleaseAutomationIn regards to the User Interface for every CA Release Automation system, there are two

parts in the launch interface as shown in figure 2.5: Release Operations Center (ROC) and

Automation Studios. ROC is a web application for creating release flows and managing their

executionintermsoftheoperationalfunctionality,whichisorganizedintotabsinthedashboard,

as seen in figure 2.6. Automation Studio is a legacy client that provides some administrative

features, such as administrative tasks, export and import functions, and process scheduling and

notificationsettings.ROCisthemainareaformostdevelopmentandoperationteams.

15

Figure2.6:ExampledashboardofCAReleaseAutomation’sROC

The typical and simplified scenario of deploying an application and promoting that

applicationfromintegrationtestingthroughtoproductionisinfigure2.7.

Figure2.7:scenarioofdeployingandpromotinganapplication(CA_Technologies,2015a)

1. CreatethereleaseusingtheROC.

2. Retrievethespecificartifacts(applicationfilesandcontent)fromtherepositories.

16

3. Next,stagetheartifacts.

4. Thenextsteprunsthedeployment,whichcouldhaveavarietyofsteps.Hereitdoesa

pre-deployment verification and configures a load balancer. It then deploys the

applicationanddatabaseandthendoessomepost-deploymentverification.

5. If everything was successful, the same deployment can be used (with different

environmentandreleasedata)topromotetheapplicationtothenextstage.

6. Prior to the deployment to production, the deployment can go through an approval

processwithintegrationtoaservicedesk.

7. Iftheapprovalprocesspasses,thedeploymentisthenpromotedtoproduction.

Throughouttheprocessabove,itshowshowtolinkarepeatablecontinuousdeliveryprocess.

In general, CA Release Automation enables IT operations to centrallymanage, automate and

controlapplicationserviceoperationsoveradatacenter,aswellasstandardizeapplicationservice

operation and application workflows. More terminologies about CA Release Automation are in

AppendixA.

2.8Jenkins

Jenkinsisanopensourcecontinuousintegrationtoolwithanactivecommunityandarich

ecosystemofextensions(Kawaguchi&Hayes,2015).JenkinsiswritteninJavaandworksonmost

operatingsystems.Itisalsotheworld’smostpopularopensourcecontinuousintegrationsoftware,

whichisbeinguseinmorethan65,000sitesworldwide.

Two critical features of Jenkins are monitoring the software development process and

building and testing softwareprojects continuously,whichhelpusers to obtain thenewest build

andcodequality(Kawaguchi,2015).

Jenkins provides functionality to run jobs - predefined tasks - on certain circumstances,

suchasanupdatetoasourcecontrolrepository,runningataskinasetintervaloftime,oraftera

17

manualrequesttorunajob(Kawaguchi,2015).Jobsareversatileastheycanbecustomdefinedto

doanything-buildcode,rununit tests,checkcodequality,deploycodetoaserverenvironment,

ping a web server, etc. Jenkins provides functionality through its core product and plugin

extensions for running unit tests on a source control repository following the practices of CI. It

could test integration of development andmainline branches before they aremerged in. During

testing, it can identify issuesaheadof time,automaticallyrunqualityassurance,deploysoftware,

create processes to require manual approval, and test to proceed in a step in the deployment

pipeline.MoreJenkinsterminologiesareinAppendixA,andtheinstructiononhowtouseJenkins

PluginisinAppendixB.

2.8.1JenkinsPromotedBuilds

JenkinsPromotedBuildsPlugin,asshowninfigure2.8,providesaframeworkformanual

approvalwithintheCDpipeline.

Figure2.8:JenkinsPromotedBuildsPlugin(Inman,2012)

TheplugincreatesaUserInterface(UI)whereautomatedprocessescanberundepending

on certain circumstances. For example, a build couldbe automaticallydeployed to an automated

acceptance-testingenvironment,oritcouldwaitformanualapprovaltobedeployedtoaQAtesting

environment.Afinalstepcouldalsobeaddedtoautomatedeploymenttoproductionsystemsonce

18

all other steps have been completed. Jenkins with this Promoted Builds Plugins is another

continuousdeploymentoptionbesidesCAReleaseAutomation.

19

3.0Methodology The goal of this project is to improve automated deployment in the continuous delivery

process.Ourmeasurableobjectiveswere:

1. Evaluated advantages and disadvantages of the firm standard tool (CA Release

Automation)andthetoolusedinPrimeFinancing(Jenkins)

2. Investigatedworkflowimprovementsofthechosentool

3. Generateddatareportsaboutcontinuousdeploymentprocessesofthechosentool

Weusedonlineresearch,casestudies, interviews,andsampleimplementationstoachieve

ourobjectives,whichwediscussindetailinthischapter.

3.1Evaluatedadvantagesanddisadvantagesoftwodifferentcontinuous

deliverytools

Thefirstphaseofourprojectwastocomparethetwotoolstodecidetheproperonetouse

whenimplementingapplications.Wedidresearchonlinetostudytherelatedterminologies,suchas

Agile Software Development, DevOps, Continuous Integration, and Continuous Delivery.We also

didresearchofeachtoolandcatalogedfeaturesofbothtools,JenkinsandCAReleaseAutomation.

Moreover,tounderstanddetailedinformationconcerningimplementation,wedidcasestudiesvia

implementingtestapplicationsintothetwotools.Inordertobetterunderstandthetwotools,we

interviewedseveralexperiencedemployeesinbothJenkinsandCAReleaseAutomation.Theteam

determined the advantages and disadvantages of each tool and compared feature by feature to

reachaconclusion.

20

3.2InvestigatedWorkflowImprovements

OncewehaddecidedtorecommendJenkinswiththePromotedBuildsPluginforthePrime

Financing group we investigated possible workflow improvements. We investigated plugins

recommended for usage in a continuous deliverywith automated deployment environment, and

compared the different recommended plugins in both their feature set and compatability with

Prime Financing’s workflow. As well, we investigated the possibilities for automating Jenkins

configuration, and created a proof of concept for managing configuration through Job DSL, and

throughJenkins’scripting,andfinallycreatedaJenkinsplugintoautomatetheprocessofcreating

and pruning per-release branch jobs. We also explored functionalities of three volunteer

applicationstoidentifycorestrategiesofonboardingtheapplicationstothetool.

3.3Generateddatareportsaboutcontinuousdeliveryprocesses

In order to provide a better way to evaluate continuous deployment processes, we

developedaprocesstoextractrawdatafromJenkins.Thisrawdata,whichincludesusefulstatistics

such asnumberof successful/failedbuilds andpromotions, canbe saved into a SQLdatabaseor

CSV file and thenmanaged in Excel.We used the pivot table functionality in Excel to filter and

managedata,andcreatedareusabletemplate.Todeveloptheprocessforextractingdatawefirst

created a rapid proof-of-concept using Jenkins’ groovy scripting capabilities and by reading the

relevantdocumentationforidentifyingthelocationofthedata.Bydoingsowewereabletorapidly

identifyandtestaprocessforextractingdata,andtoagreeuponacommondataformatsuchthat

the finalized extraction plugin and the reporting excel template could be developed in parallel.

ProducedwereaJenkinspluginandanExcelreportingtemplate.

3.4Summary

The methods described above were to fulfill our project goal: to improve automated

deploymentinthecontinuousdeliveryprocess.Thebackgroundresearchhelpedustounderstand

21

theoverallconceptswithinDevOpsandtheirimportance.Theexperiencedemployees’ interviews

providedussubjectiveresponsesontheadvantagesanddisadvantagesofeachtool.Weidentified

someobjectivestrengthandweaknessesviasampleimplementationsforeachtool,andcompared

with the information provided by experienced people. The investigations on workflow of

automateddeploymentimprovesthecontinuousdeliveryprocessoursponsor,PrimeFinancing,to

adopt the chosen tool on their existing application. In order to generate data reports about the

continuousdeploymentprocesses,weextractedandmanageduseful statistics toevaluateoverall

processes.

22

4.0ResultsandAnalysisIn this chapter,wediscussour results andanalysisonbackgroundresearch, comparative

analysis,andimplementations.WeevaluatedtheadvantagesanddisadvantagesofbothCARelease

Automation, also known as Nolio, and Jenkins Plugin, determined the better tools for Prime

Financing group, and improved workflows of automated deployment within Prime Financing’s

Jenkins. Based on our research and practices, we successfully achieved our goal to improve

automateddeploymentinthecontinuousdeliveryprocessforoursponsor,Barclays.

4.1ComparingTools

To compare Nolio and the Jenkins plugin, we studied extensive online research, both

through publicly available resources and Barclays’ internal resources, and interviewed several

experiencedemployeesregardingeachtool.

4.1.1InvestmentBank(IB)StandardToolandControlObjective

In order to better adopt the DevOps initiative in Barclays, there is a list of IB standard

development tools for each DevOps category, such as Continuous Integration and Continuous

Deployment.Eachapprovedtoolhasadifferent levelofBarclays’s internalsupport. Inregards to

thestandardtoolforContinuousDeployment,Nolioistheonlylistedapprovedtool.Thefigure4.1

showsotherapprovedtoolsfortheDevOpsinitiativewithinIB.

23

Figure4.1:TheDevOpsprocesswithapprovedtoolsBarclayshas setupwhat are called “control objectives”whicharemandatoryoroptional

criteria during an application’s development cycle. One specific mandatory control objective

pertaining to Continuous Deployment states that “no change can be made to any technology

configuration item without appropriate authorization.” Fortunately, both Nolio and the Jenkins

pluginsatisfythiscontrolobjective.

4.1.2LevelofSupport

ForNolio,thereisacomprehensiveamountofsupportavailablewithinBarclays.Thereisa

designated Nolio support team in Barclays providing approximately 400 pages of support

documentationaswellasemail,chatclient,andtelephonesupport.Themostusefuldocument,“my

firstNolioapplication,”providesatutorialonhowtoimplementa“helloworld”-esqueapplication

intoNolio.

In regards to Jenkins, since Jenkins and the various plugins are open-source, it is easy to

retrievemanyavailableresourcesonlinetogethelp.WhilethereisnointernalBarclayssupportfor

Jenkins,manyemployeeshavebasicunderstandingsofthetool.

24

4.1.3OnboardingIntuitiveness

The Barclays-supported Nolio requires an extensive onboarding process which involves

submittingasupportticketonboardingrequest,requestingtocreatetheNolioprojectgroupsand

then following a long detailed process to onboard an application. Through interviews, especially

with employees of the support team, we received information that the approximate time to

onboardanewapplicationisaweekortwo.Themajorreasonforthelengthyonboardingtimeis

becauseofwaitingforapprovalsorassistancefromBarclays’Noliosupportteam.

Oneexampleteamthatweinterviewedsaidthattheystartedtoonboardtheirapplications

inMay2015andeven todayarestillworkingon theNolio implementation.The teammentioned

that the complexity and heavyweight nature of Nolio is part of the reason for the long

implementationtime.Ontheotherhand, there isaseparateteamwhopraisedthesupportof the

Barclays’sNoliosupportteamwhenonboardingtheirspecificapplications.

AsforonboardingJenkins,itiseasier.IfaprojectteamwishestoonboardwiththeJenkins

PromotedBuildsPluginmanagedbythePrimeFinancingteam,theprojectteamsimplyhastofill

outadetailedquestionnaire,writtenupbyPrimeFinancing,whichasksformostofthenecessary

configuration information. Then the project teamwould have to supply appropriate deployment

scripts so that the deployments can actually run. In general, the time spent in the onboarding

processforJenkinsandthepluginisaboutonlyafewhours.

4.1.4DeploymentProcess

NolioandJenkinspluginaretwoalmostentirelydifferentecosystems.Themainpurposeof

Nolioistododeploymentsviadrag-and-dropactionsandflowsonitswebdashboard,whichison

the leftof figure4.2.Thisoffers flexibilityandrobustness fordifferent typesofprojects,whereas

the Jenkins Promoted Builds Plugin handles deployments through approval gates and the

deployment scripts written by the application team. As the right of figure 4.2 shows, the main

25

deploymentprocedureintheJenkinspluginisshellscripting,andthecomplexityleveldependson

theuser’sneed.

Figure4.2:(Left)Drag-and-dropactionsandflows(Right)Exampleshellscriptlog

4.1.5SampleImplementation

On theNolio front,we followed the tutorial suppliedbyBarclays’sNolio support team to

haveabetterunderstandingoftheNolioarchitecture.However,itwasalongandarduousprocess.

AppendixCshowsasummarizedstep-by-stepprocedureofwhatthetutorialwasaskingandhow

toaccomplishadeployment.AsforJenkins,thesampleimplementationisintuitive.Themajorityof

investigationandresultsareontheplugins,asseeninsection4.2.

4.2ImprovingWorkflows

Besides basic functionalities of Jenkins itself, there are various plugins that are able to

enhance the use of Jenkins. According to the requirements of Prime Financing group, our team

exploredsomeusefulpluginswithinJenkinsinordertofulfilltheirdemands.Themajorpartwasto

enhanceautomatedonboardinganddeploymentprocess.

26

4.2.1AutomaticDeploymentWorkflow

Jenkinscanbeusedtobuildacontinuousdeliveryordeploymentpipelineout-of-the-box.

There are numerous plugins that add additional features such as manual approval gates, visual

overviews, and custom permissions, to allow users to better fine tune and fit the automated

deployment set up to their needs. The plugins evaluated were Promoted Builds Plugin, Build

PipelinePluginandDeliveryPipelinePlugin.

PromotedBuildsPlugin introduces theconceptof“promotions”,whichhaveprerequisites

and resulting actions to allow fine tuned control over triggering actions related to an individual

build in Jenkins. Build Pipeline Plugin is an overview of the promotions, which provides an

overviewbasedonbuilt in jobrelations,aswellasprovidinganewtypeofrelation–a“manual”

build step. Delivery Pipeline Plugin also provides an overviewbased on built-in job relations, as

well as supporting the manual steps of the Build Pipeline Plugin and the promotions of the

PromotedBuildsPlugin.

Weevaluatedtheprovidedfunctionalityoftheplugin,includingsupportfor“fan-out-fan-in,”

manual approvals, and allowing for specifying build parameters; the look and usability of the

overview;andthecomplexityofsetup.

4.2.1.1PromotedBuildsPlugin

The Promoted Builds Plugin provides the ability to mark certain builds of a job as

“promoted” if the build has fulfilled a certain criteria, as well as to trigger certain actions upon

promotion (Kawaguchi&Hayes, 2015). This plugin extends the functionality of Jenkins to allow

triggeringofdownstreamjobsonmorecriteriathanjustifthebuildpassedorfailed.Italsoallows

requiring multiple previous or triggered jobs to complete successfully, manual approval from a

specificsetofapprovers,andotherpromotionsforthejobthathavealreadyoccurred,inorderto

allow one to enforce an order of promotion. Moreover, the plugin allows for specifying build

27

parameters for triggered jobs and allowing a manual approver to provide needed details for a

deployment,suchasaservicedeskticketnumberordeploymentcredentials.

Theplugindisplaysthepromotionstatusofeverybuilddirectlyonthejobandbuildpages,

asshowninfigure4.3.

Figure4.3:JobPage(Left)andBuildPage(Right)ofPromotedBuildsPlugin

Settingup thesepromotions isquick,as simpleas settingupautomatic triggersbuilt into

Jenkins.However,theplugindoesnotshowtheupstream/downstreamrelationshipsbetweenjobs,

soitcanbedifficulttoseewhatjobsexactlyaretriggeredandinwhatorderofjobsarerelatedtoa

promotion. Ordinarily one would have to click through many pages concerning builds and

promotions to understand the upstream and downstream relationships. Additionally, while the

plugincanrequireonepromotionbecompletedsuccessfullybeforeanotherone,onecan“approve”

apromotionatanytimeaccidentally.However, itcannotdisapproveapromotionevenif ithasn’t

occurredyet,whichmayresult intwosequentialmanualpromotionsoccurringimmediatelyafter

one another because of the second out-of-order accidental approval. If, for example, the first

promotionwastodeploytoamanualQAenvironmentandthesecondadeploymenttoabeta/UAT

environment,thiswouldnotbeideal.

28

4.2.1.2BuildPipelinePlugin

The build pipeline plugin provides an overview of a continuous deployment “pipeline”

basedonaninitial“seed”build.Thepipelinedisplaysallrelateddownstreamjobstriggedbyitand

theirstatus.Thepluginalsoprovidesanewtypeoftrigger/relationshiptoallowmanualapproving

fortriggeringadownstreamjob,similartothePromotedBuildsPlugin.Howeveritdoesnotallow

formore sophisticated approval requirements, such as restrictingwho can approve the build, or

requiring that multiple upstream jobs have completed successfully before a job is triggered.

Furthermore,onecannotsupplymanualparametersforamanualbuild;suchparametersmustbe

provided at the start of the build cycle, which is not feasible in many cases when builds are

deployedseveraldaysorweekslater.

The pipeline UI, in figure 4.4, itself is rather difficult to use and look at, and does not

integratewellwiththeJenkinsUI.

Figure4.4:PipelineUIofBuildPipelinePlugin

29

4.2.1.3DeliveryPipelinePlugin

DeliveryPipelinePluginfulfillsasimilarpurposetoBuildPipelinePlugin,howeverwitha

better UI execution, as in figure 4.5. Similarly to the Build Pipeline Plugin, the Delivery Pipeline

Pluginshowsanoverviewbasedonjobupstream/downstreamrelationships.Additionally,itallows

for grouping of related processes in “stages.” It can use the Build Pipeline Plugin manual build

relationship to allow for manual builds. Like the Build Pipeline Plugin, you cannot specify

parametersforamanuallyapprovedbuild.

Figure4.5:PipelineUIofDeliveryPipelinePlugin

It is also compatiblewith the PromotedBuilds Plugin; however support seems to not be

perfect, as multiple promotions on the same build will show as parallel processes, even if the

promotions are sequential as seen in figure 4.6. Also, while promotions can be shown on the

pipelinepage,youcannotpromoteabuilddirectlyfromthepage.

30

Figure4.6:Multiplepromotionsonthesamebuild

TheDeliveryPipelinePluginshowspromiseforshowingacomprehensiveoverheadviewof

thejobsrelatedtoaninitialseedbuild,andwhenusedwiththePromotedBuildsPlugincansatisfy

the requirementsof allowing restrictedmanualdeploymentwith specifiedparameters.However,

its integration with the Promoted Builds Plugins is lacking. The community in Jenkins and the

Delivery Pipeline Plugin has expressed interest in proper integration with the Promoted Builds

Plugin,anditispossiblethatinthesoonfuturetherewillbefullintegration.Evenwithoutperfect

integration the Delivery Pipeline Plugin provides a useful overview to see the status of builds,

especially as velocity increases or as more automated test steps are added, such as automated

integration tests in QA, or automated assurances in Production. The Promoted Builds Plugin

providesallthenecessarymanualapprovalrequirementsandparameters,andtheper-promotion

overviewonthebuildspageprovidessufficientoverviewformanycases.

4.2.2AutomatingJenkinsConfiguration

Groovy is a scripting language based on top of Java that runs in the Java JVM, and it can

leverageandextendexistingJavaAPIsinalooselytyped,dynamiclanguage.Jenkinsislargelybased

31

on groovy and allows for extensions based in this scripting language. Moreover, Jenkins has

multiple plugins that allow for executing scripts that can interact with Jenkins’ internal APIs to

managejobcreationandJenkinsmanagement.Additionally,thereisapluginthatdefinesaDomain

SpecificLanguage(DSL)formanagingjobcreationandmanagementwithinJenkinscalledthe“Job

DSL”Plugin,whichsimplifiestheprocessofprogrammaticallycreatingJenkinsJobs.Throughusing

Jenkins’ internalAPIs,onecanautomatevariousaspectsofmanagementof Jenkins’configuration

forcertainapplications.

4.2.2.1UsingJobDSLtoScriptJobCreation

Operations that frequently create a large number of jobs and require keeping the

configurationofmanyjobsup-to-datemaybenefitfromusingtheJobDSL.

UsingJobDSLtocreateasimpleJob:

mavenJob("my_job"){ description("""Mygreatjob""") logRotator{ daysToKeep(7)

} keepDependencies(false) scm{ svn{ location(“https://svn/project/branch”){ credentials(SVN_CREDENTIALS_ID) } checkoutStrategy(SvnCheckoutStrategy.UPDATE_WITH_CLEAN) } } triggers{ scm('H/10****'){ ignorePostCommitHooks(false) } } }

4.2.2.2LimitationsofusingtheJobDSL/ExtendingJobDSL

There are several significant issues, however, with using the Job DSL. The first, most

obvious issue isthe learningrequiredtobeabletousetheJobDSLeffectively.WhiletheJobDSL

aimsandsucceedsatmaking jobseasiertomaintainthanbydirectlyusingJenkinsAPIs, it isstill

32

much less intuitive than using the GUI interface, and some more obscure features may not be

supported at all, requiring extending the Job DSL. Furthermore, any updates to Jenkins may be

potentially breaking, as was experienced during evaluation. During evaluation, a Jenkins update

changedtheXMLformatfora“logRotator”plugin,causingtheJobDSLimplementationtobebroken.

OnecanusegroovytoextendtheJobDSLclassestoeitherfixorextendtheJobDSL.Groovy

provides“mixin” functionality thatallowsaddingmethods toexistingclasses.Duetohowthe Job

DSLis implemented,onecanjustaddmethodstotheDSLclassesandtheyareavailabletousein

theDSL. This functionality can be used to add implementation specific shorthand or to addnew

featureswithouthavingtowaitforacceptancebytheopensourceproject.However,onemustlook

atanexisting, functionalXMLconfiguration to identify the requiredstructure,whichcan thenbe

automated.

UsingGroovyMixinstoextendtheDSLtosupportanadditionalfeature(“keepbuildsforever”):

classPromotionActions{ staticdefbind(){ javaposse.jobdsl.dsl.helpers.step.StepContext.mixinPromotionActions } /** *Promotedbuildsflagtokeepbuildforever */ voidkeepBuildForever(){ stepNodes<<newNode(null,'hudson.plugins.promoted__builds.KeepBuildForeverAction') } }UsingGroovyMixinstoextendtheDSLtorepairanexisting featurethatwasbrokenduetoa

Jenkinsupdate:

classJob{ staticdefbind(){ javaposse.jobdsl.dsl.Job.mixinJob } /**Newversionofjenkinsmovedthelocationofthisconfigurationnode Oldversionusedforinspiration:

33

https://github.com/jenkinsci/job-dsl-plugin/blob/master/job-dsl-core/src/main/groovy/javaposse/jobdsl/dsl/Job.groovy#L203-L216 */ voidlogRotator(@DslContext(LogRotatorContext)Closureclosure){LogRotatorContextcontext=newLogRotatorContext()ContextHelper.executeInContext(closure,context) configure{project-> project/'properties'/'jenkins.model.BuildDiscarderProperty'/ 'strategy'(class:'hudson.tasks.LogRotator'){daysToKeep(context.daysToKeep)numToKeep(context.numToKeep)artifactDaysToKeep(context.artifactDaysToKeep)artifactNumToKeep(context.artifactNumToKeep) } } }}

4.2.2.3UsingJenkinsAPItoDuplicateJobs

Anotheroption for the creationofnew jobs is to just lookat existing jobsand copy their

configuration. Sincewearen’t concernedwith the internalsof the job structure, theAPI calls are

very simple. This method was used by Prime Financing in a manual process, where a job is

structuredsothatanyjob-specificconfigurationisreliantontheJobTitle(inthiscase,thejobtitle

matches theSVNbranch title),and jobsareduplicatedasneeded.Automationofcreatingrelease

branchjobswasstraightforward.

4.2.2.4UsingGroovytoFetchSVNBranches

Jenkinscomeswiththe“SVNKIT”librarybuiltinforitsbasefunctionality,sowewereable

to leveragethis librarytofetcha listofbranches. It isalsopossibletousethecommandlineSVN

clientandparsetheoutputitgives,whichwastheoriginalimplementation,howeverusingSVNKIT

meansweareusinga librarythat isalways installedwith Jenkins. (TheSVNcommand lineclient

mayormaynotbeinstalledontheLinuxserverclient).

4.2.2.5Automatewhatyoucan–UsingScriptingtoautomateJobcreationforbranches.

Weendedupcreatinga script,as shown inAppendixD, thatpollsanSVNrepositoryand

ensures that the Jenkins configuration matches what is expected. The script uses the newest

34

matchingbranch jobasa template forcreatingnewbranch jobs. Itcandeleteordisableold jobs,

andcreatenewjobsautomatically,removingamanualprocessthatrequireddistractingaJenkins

administrator away from their other responsibilities. The frequency of the polling process is

configurable,butgenerallyitwouldbesettocheckonaregularschedule.

Using the Job DSL was not ideal because the configurations were complicated and used

pluginsnot supportedby JobDSL.The jobswerenot createdwith such frequency thatusing Job

DSLwould speed things up, and the learning of JobDSL and requirements of extending JobDSL

wouldbe as errorprone if notmore errorprone thanbuilding/modifying Jenkins jobsmanually

through GUI. The other benefit of Job DSL over “copy paste” is to keep all jobs with the same

configuration. Since branches are generally for release, old branches are just kept for historical

purposes and do not need to be kept up to date. Copying the last used branch provides the

configurationneeded.

4.2.2.6JenkinsIntegrationwithPlugins

AfterdevelopingarapidprototypeusingGroovyscript,thedesireforJenkinsconfiguration

integrationarose,andapluginwasdeveloped.Jenkins’pluginframeworkallowsforquickcreation

of configuration menus with error checking, and the similarity of Groovy to Java (the language

Jenkinspluginsarebyinlargewrittenin)allowedforaveryquickdevelopmentcycle(themajority

of the time taken implementing new error checking features). Moreover, the integration with

Jenkinsallowedstreamliningsomeoftheprocess–insteadofdependingontheGroovyplugin,the

EnvInject Plugin, and proper configuration of both, one just installs and configures the Sync

Branchespluginwedeveloped.Thepluginisusedasabuildstepandtheconfigurationanderror

checkingcanbeseen in figures4.7and figure4.8.Aglobalconfigurationareawasalsoaddedfor

commonconfiguration,asseeninfigure4.9.

35

Figure4.7:ConfigurationScreenforaSyncBranchesBuildStep

Figure4.8:ConfigurationScreenwithAdvancedOptionsandError-CheckingVisible

Figure4.9:GlobalConfigurationScreen4.3GeneratingReports

Metrics andmeasurements of projects are important to understandwhether projects are

improving. We investigated available Jenkins reporting plugins, however they did not meet our

expectations. We decided to look at the source; Jenkins itself keeps all metrics and statistical

information, such as number of successful/failed builds and number of successful/failed

promotions. Therefore we investigated ways of extracting information from Jenkins into a

consumableformatthatcanbeusedinreportingsoftwaretogeneratereports.

36

4.3.1DashboardViewPlugin

WeexploredcertainJenkinspluginsthatmightprovidefunctionalitytoshowManagement

InformationSystemdetails.TheDashboardViewPlugin isonesuchpluginwhichcanshowsome

straightforwardinformationaboutjobsandbuilds.

DashboardViewadds “anewview implementation thatprovidesadashboard/portal-like

viewforJenkins”(Hayes&Ambu,2015).Thepluginallowsuserstocreateoneormorenew“views,”

labeledasDashboard,whichiscustomizableandcontainsvariousportletswithinformationabout

theselected job(s).Portletsarewidgetscontainingusefulchartsorstatistics.Furtherexplanation

onthisterminologycanfoundinAppendixA.MoreinformationaboutDashboardViewPlugincan

befoundinAppendixH.

EventhoughDashboardViewprovidescapabilitiestovisualizesomeinformationaboutjobs

and builds within the Jenkins web instance, it doesn’t provide any information regarding

promotions.Toovercomethislimitationwedecidedtocustomimplementtheabilitytoaccessthis

information.Whendecidinghowtodisplaytheinformation,wechosetoextractthedatatobeused

inexternalreportingsoftwareasthatwouldprovidemoreflexibilitythanwhatwasshowninthe

Jenkinsdashboardforfarlesswork.

4.3.2DataExtractionPlugin

Aspartof theprojectobjective toprovideMISreports for Jenkinsusage, itwas identified

thata toolneeded tobecreated toextract JobandPromotiondata froma Jenkins server intoan

easilyconsumableformatthatcouldbethenanalyzedandtransformedintoareport.

4.3.2.1Prototype–IdentifyingDataExtractionMethod

Duringtheinitialresearchphase,weusedgroovyscriptingintheJenkinsconsole,asseenin

figure 4.25, along with documentation to identify where the data we desired resided. The data

37

identifiedwasthebuildstatusandtimeforeverybuildandpromotioninformationwithinJenkins.

Jenkins exposes this data through a stable andwell defined, albeit internal, API. Thismethod of

extractingdataisquickandsimple,andleveragestheusabilityoftheJenkinscodebase.

Figure4.25:JenkinsScriptConsole

Alternatives

AlsoevaluatedwasthepossibilityofusingtheJenkinsRESTAPI–anHTTPAPIprovidedby

Jenkins and its plugins. However, in our research we discovered that the API does not have an

endpointformassdataextractionofthistype,butratherallowsforinvestigationandmodification

ofindividualJobs.Itdoesprovideawaytoenumeratejobsandbuilds,howeverusingthatendpoint

wouldstillrequireanewHTTPRESTAPIcallforeverysinglejob.Aswell,thereisnoAPIendpoint

toenumeratePromotionsforagivenbuild.Giventheselimitations,andgivenhowsimpleusinga

programmaticmethodis,wechosethelatter.

Another possibility is to do manual parsing of the XML files in which the Jenkins

configuration is stored, an idea that was brought up by the sponsor. However, this idea was

38

droppedforanumberofreasons:theXMLfilesmaybeoutofsyncwiththeJenkinsinstance,and

maychangewhilereading.Aswell,theoverheadofwritingthecodetoparsetheXMLmakeslittle

sensewhenJenkinsitselfalreadyhasdonethetroubleofthatforus.

4.3.2.2Prototype–ExtractingDatatoFile

Initially,ourprototypegroovyscriptwasmadetoextractthedesireddatatoaCSVFile.The

nextstepwastoextractintoaSQLdatabase.Therewerethreeideasposed:extractthedatawithin

thegroovyscripttoSQL,extractthedatawithgroovytoCSVandthencreateasecondapplicationin

JavatoreadtheCSVandexporttoSQL,orcreateasingleJenkinsplugininJavatoextracttoSQL.

Hypothetically,onecanuseafeatureoftheGroovyJenkinsPlugintoaddadditionaljarfiles

totheclasspathof thescriptVM,allowingforusingexternal jarssuchaswouldberequiredfora

databaseconnector.However,thisoptionwasdroppedfor2reasons:1)duetoabugintheplugin,

itdidn’twork,and2)having tomaintaindependenciesusingcomplexclasspathconfigurations is

errorproneandnotintuitivetosetup.

Having two separate tools to maintain was not ideal. Creating a Jenkins plugin has

additionalbenefits,sothispathwaschosen.

4.3.2.3JavaExtractionToolPlugin

JenkinspluginsareallwrittenusingtheJavaprogramminglanguage.UsingJavahasafew

keybenefits:wecanstructuretheprojectasanactualprojectratherthanascriptfileandloosely

gather dependencies; a more strongly typed and strict language allows for creation of a more

robust tool; and dependencies can bemanaged as part of the project and packaged together for

deployment.Furthermore,creatingaJenkinspluginallowsforintegrationwiththeJenkinswebGUI.

39

CreatingaPlugin

Jenkins has a wide-ranging plugin extension system that we used to develop our plugin.

Jenkinsprovidesatemplateforcreatingthebaseofapluginthatcanbecustomizedtoone’sneeds.

Thepluginwas implementedas a JenkinsBuild Step,which canbe added to and configured in a

JenkinsJob.ThepluginwasmadeinJavausingMavenfordependencies,whichallowsforbundling

therequireddependenciessuchasdatabasedriversandhelperlibrariesinthetooldistributablein

thepluginfile.

ConfigurationConsiderations

Parameters are configurable as a part of the standard Jenkins configuration console. The

target database and credentials are configurable in Jenkins in the build job. Credentials are

encryptedusingJenkinsstandardmethods.

DataTypeConsiderations

In designing the tool some special considerations were made towards data types for

interoperability.JavadatesoperateusingmillisecondssincetheUnixEpoch.MicrosoftSQLServer,

as a .NET product, operates on 100 nanosecond “ticks” since the year 1 started.While the .NET

DateTime is 10000 timesmore precise than Java’smilliseconds, in the conversion between .NET

andJavawesawerrorswherethetimerecordedwas1millisecondofffromtheactualtimedesired

to be recorded. To prevent such conversion errors, time was stored as a raw long integer

representing the Java Date, in such a way that conversion between different Date styles is not

needed.However,MSExceldoesnothaveaneasywaytoconvertMillisecondstoalocaltime,sothe

datawasdenormalizedandanadditional localdatecolumnwasaddedthat ispreciseenough for

reportingneeds,andinaformatthatExcelunderstands.

40

4.3.2.4Outcomes

A Jenkins Pluginwas created and implemented on Prime Financing’s Jenkins server. The

configuration screen is shown in figure 4.26, and it has error checking validates credentials and

URL,asseeninfigure4.27.ThepluginwillextractnewjobmetadatatothespecifiedSQLserver,as

seeninfigure4.28,ortoaspecifiedCSVfile.Thepluginisdesignedsuchthatitcanberunatany

interval,andwillonlyappendnewdata.

Figure4.26:ConfigurationScreen

Figure4.27:ErrorCheckingvalidatescredentialsandURL

41

Figure4.28:DatainSQL

4.3.3ReportingwithExcelPivotTable

TheJobandPromotioninformationfromtheSQLserverwasthenimportedintoExcel.We

decided to use pivot tables to filter and group the very large data set. A pivot table is a data

summarizationtoolthatcanautomaticallysort,count,giveaverage,anddootherfunctionsondata

stored in a spreadsheet or table. The actions and filtering are independent of the original data

(MacDonald,2005). InstructionsonhowtocreateapivottableusingaSQLdatabaseandCSVfile

sourcecanbefoundinAppendixEandAppendixFrespectively.TheversionofExcelweworkedon

wasMicrosoft Excel 2007, but the pivot table functionality is available in every version of Excel.

InformationaboutthefeaturesofourpivottableconfigurationcanbefoundinAppendixG.Theend

resultcanbeasseeninfigure4.29below.Thepivotchartisniceinthatitshowsanunderstandable

visualrepresentationofthenumberofsuccessful/failedpromotionsacrosstime.

42

Figure4.29:PivotChartcreatedfromsampledata.4.4Summary

By analyzing the strength andweakness of both CAReleaseAutomation and Jenkins and

sample implementations, we determined Jenkins is a better to bewidely implemented in Prime

Financing.WeexploredandtestedseveralJenkinsplugins,whichcouldbehelpful,suchaspipeline

view, JobDSL,Groovyscripting,andDashboardView,andweevaluatedfeasibilityofeachplugin.

WealsodemonstratedhowtocreateManagementInformationSystemdatareportswithJenkins.In

the next chapter, we will conclude our findings and provide our recommendations in order to

enhance the continuous delivery process of the chosen tool, Jenkins, and the process to adopt

candidateapplicationsintoJenkins.

43

5.0ConclusionsandRecommendations Inthischapter,weprovideourconclusionsandrecommendationstochoosethebettertool

and to develop strategies to onboard applications on the chosen tool. We recommend Prime

Financinguses Jenkinsbasedonourcomparativeanalysisof JenkinsandCAReleaseAutomation.

We determined several strategies to enhance the continuous delivery process on Jenkins via

investigating anddeveloping someplugins.We also provided recommendations onhow to show

andusemanagementinformationsystemdetails.

5.1Conclusions

Based on our comparative analysis, both CA Release Automation and Jenkins have their

respectiveadvantages.CAReleaseAutomationprovidesrobustandsharabledeploymentstrategies,

and allows users to configure environments.Moreover, there is less administrative overhead for

frequentlychangingdeploymentprocessesandcomplicateddeploymenttomanydifferenttargets.

As for Jenkins, it follows build-your-own deployment strategies, which provides convenient

integrationwith current application configurations. Furthermore, the set-up process is quick for

teamswithexistingdeploymentscriptsorsimpledeployments.

Weexploredseveralpluginsthatcouldenhancetheautomateddeploymentprocess.Jenkins

has several available pipeline view plugins, which could show upstream and downstream

relationships.Thesepluginsareuseful,butdon'tintegratewellwiththePromotedBuildsPluginor

withPrimeFinancing’sconfiguration.AsforJobDSLPlugin,whileitisapowerfultool,ithasahigh

learningcurveand isheavyweight ingeneral. ForGroovyscripting, it canaccomplish simpleand

useful actions, however it has a high learning curve for complicated jobs. In the end, however,

Jenkins plugins written in Java that leverage Jenkins’ API and tools proved themost useful and

44

stable for reliably automating these tasks, while Groovy scripting proved to be a useful tool for

quicklytestingandscriptingsmallthings.

As forManagement Information System reporting,we discovered that any specified data

could be extracted from Jenkins, such as build andpromotion information.We created a Jenkins

Plugin to extract data either into a SQL database or CSV files. Furthermore, the extracted

information can be imported into data reporting software, such as Excel, to create reports; we

createdareusableExceltemplatethatcreatesreportsfromPrimeFinancing’sJenkinsinstance.

5.2Recommendations

Basedonourresearchandimplementations,werecommendusingJenkinsoverCARelease

Automation because of minor overhead and quicker set-up. In order to eliminate some manual

steps, we recommend using our Jenkins Plugin ‘Sync Branches’ to accomplish automatic job

creationandcleanupfromSVNbranchinginsteadofmanuallycopying jobs.Moreover, toprovide

management information details. We recommend using our Jenkins plugin to extract data into

eitheraSQLdatabaseorCSVfiles.SQLdatabaseisthepreferreddestinationratherthanCSVfiles

because SQL databases are easier to manage data and has better compatibility with different

reportingsoftware.Additionally,wealsodevelopedanExceltemplatetovisuallyshowdatareports.

WehighlyrecommendusingExcelwithour template, sinceExcel iswidelyusedand intuitive for

reporting.

5.3ImpactofOurProject

Ourprojectresultsandrecommendationscanhelpoursponsor,BarclaysPrimeFinancing,

to improveautomateddeployment in thecontinuousdeliveryprocess,andtogeneratereports to

understand the continuous delivery process. Implementing strategies we developed could

eliminateafewmanualstepswithintheircontinuousdeliveryprocess.Byusingourstrategies,we

45

hope the continuousdeliveryprocess couldbemore efficient and easy, thushelping address the

challengesofautomateddeployment.

46

ReferencesAmbu, M. (2015). Project Statistics Plugin. Retrieved from https://wiki.jenkins-

ci.org/display/JENKINS/Project+Statistics+Plugin

Barclays. (2015). Purpose and Values. Retrieved from https://www.home.barclays/about-

barclays/barclays-values.html

Beck, K., Beedle, M., Bennekum, A. v., Cockburn, A., Cunningham,W., Fowler, M., . . . Thomas, D.

(2001).Agilemanifesto.Retrievedfromhttp://agilemanifesto.org/principles.html

CA_Technologies. (2015a). CA Release Automation. Retrieved from

http://www.ca.com/us/devcenter/ca-release-automation.aspx

CA_Technologies.(2015b).CAReleaseAutomation5.5.x:Overview100.

Cohen, D., Lindvall, M., & Costa, P. (2003). Agile Software Development. Retrieved from

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.201.2704&rep=rep1&type=pdf

Farcic, V. (2014). Continuous Delivery: Introduction to concepts and tools. Retrieved from

http://technologyconversations.com/2014/04/29/continuous-delivery-introduction-to-

concepts-and-tools/

Hayes, P., & Ambu, M. (2015). Dashboard View. Retrieved from https://wiki.jenkins-

ci.org/display/JENKINS/Dashboard+View

Highsmith, J. (2002). What is Agile Software Development? The Journal of Defense Software

Engineering,6.

Highsmith,J.,&Cockburn,A.(2002).Agilesoftwaredevelopment:thebusinessofinnovation.IEEE

ComputerSociety,34(9),8.

InfoQ. (2014). Introducting DevOps to the Traditional Enterprise. Retrieved from

http://www.neucloud.cn/wp-content/uploads/2015/02/Introducing-DevOps-to-the-

Traditional-Enterprise-final.pdf

47

Inman,H. (2012).Continuous Integration forMobileAppswith Jenkins:PromotedBuilds, theQA

Process and Beta Distribution. Retrieved from

https://www.cloudbees.com/blog/continuous-integration-mobile-apps-jenkins-promoted-

builds-qa-process-and-beta-distribution

Kawaguchi, K. (2015). Meet Jenkins. Retrieved from https://wiki.jenkins-

ci.org/display/JENKINS/Meet+Jenkins

Kawaguchi,K.,&Hayes,P. (2015).PromotedBuildsPlugin. Retrievedfromhttps://wiki.jenkins-

ci.org/display/JENKINS/Promoted+Builds+Plugin

Kim, G. (2014). My fifteen year journey studying high performing IT organizations. YouTube:

Docker.

Logan, M. J. (2014). Fresh Stats Comparing Traditional IT and DevOps Oriented Productivity.

Retrieved from http://devops.com/2014/01/23/fresh-stats-comparing-traditional-it-and-

devops-oriented-productivity/

MacDonald, M. (2005). What is a Pivot Table? Retrieved from

http://archive.oreilly.com/pub/a/windows/archive/whatisapivottable.html

Martin, F. (2006). Continuous Integration. Retrieved from

http://www.martinfowler.com/articles/continuousIntegration.html

Mountain_Goat_Software. (2005). Process of Agile Software Development. Retrieved from

https://www.mountaingoatsoftware.com/agile/scrum/images

Mountain_Goat_Software. (2015). Scrum Overview for Agile Software Development. Retrieved

fromhttps://www.mountaingoatsoftware.com/agile/scrum/overview

Mueller, E., Wickett, J., Gaekwad, K., & Karayanev, P. (2011).What is DevOps? Retrieved from

http://theagileadmin.com/what-is-devops/

Ramfelt, E. (2015). Terminology. Retrieved from https://wiki.jenkins-

ci.org/display/JENKINS/Terminology

48

Rhine, N. (2014). Project Build Times. Retrieved from https://wiki.jenkins-

ci.org/display/JENKINS/Project+Build+Times

Smith,C.(2015).The5BigBenefitsofAutomatedDeployment.Retrievedfromhttps://www.red-

gate.com/blog/5-big-benefits-automated-deployment

Trimios.(2012).BasicstructureofCIsystem.Retrievedfromhttps://trimios.com/ci_services.html

Wikipedia.(2015a).ContinuousDelivery.

Wikipedia.(2015b).ContinuousIntegration.

XebiaLabs. (2015). Depolyment Automation. Retrieved from

https://xebialabs.com/solutions/deployment-automation/

49

Appendix A: Glossary of TechnicalTermsGeneralApplication:ApieceofsoftwarethatisdevelopedandbuiltbyaBarclaysdevelopmentteam.

Applicationandprojectmaybeusedsynonymouslywithinthispaper.Onboard:Thisistheconfigurationprocedurenecessaryforapplicationstodowhenbeingsetupto

acontinuousdeploymentenvironment,whetherthatisNolioorJenkins.ForthecaseofJenkins,theapplicationdevelopmentteamisresponsibleforfillingoutthePrimeFinancing-suppliedquestionnaire,settingupthedeploymentscriptswhichdetailoutthebuildsequence,builddependency,andbuildfrequencyaswellasnotificationmethod.

Jenkins(Ramfelt,2015)Job/Project:Jenkinsseemstousethesetermsinterchangeably.Theyallrefertorunnabletasks

thatarecontrolledand/ormonitoredbyJenkins.Build:Theresultofonerunofaproject.Plugin:Third-partypluginscanextendJenkinsbyaddingfeaturesnotpresentinthecore.This

allowsuserstosatisfyanyneed,nomatterhownicheitiswithoutaffectingthecore.Node/Slave:Slavesarecomputersthataresetuptobuildprojectsforamaster.Jenkinsrunsa

separateprogramcalled"slaveagent"onslaves.Whenslavesareregisteredtoamaster,amasterstartsdistributingloadstoslaves.ThetermNodeisusedtorefertoallmachinesthatarepartofJenkinsgrid,slaves,andmaster.

Promotion:ThistermpertainstothePromotedBuildsPlugin.Abuildthatpassedanyadditional

criteriawouldbemarkedbypromotionicons.Thiskindofpromotionallowstriggeringofdownstreamjobsonmorecriteriathanjustifthebuildpassedorfailed.Userscandistinguishgoodbuildsfrombuildsthathadruntimeproblemsthroughcheckingpromotions.

Groovy:GroovyisascriptprogramminglanguagebasedonJava.Itisalightweightlanguagethat

canrunonaJavaserver.SinceJenkinsrunsonJava,aGroovyscriptisdirectlycompliableandcompatiblewithJenkins.

Jelly:JellyisacustomizableJavaandXMLbasedscriptingandprocessingengine.ItturnsXML

scriptintoexecutablecode.JellyisacomponentofApacheCommons.Portlet:ThistermpertainstotheDashboardViewPlugin.Generallyspeaking,aportletisaJava-

basedwebcomponentthatprocessesrequestsandgeneratesdynamiccontent.Theyaredeployedinacontainer,DashboardViewPluginforthecaseofourproject.

50

View(dashboardstatuswindow):Inthescreenshotshownbelow,therearetwoiconsthatdescribethestatusofthecurrentbuild.The“S”columnrepresentsthe“statusofthelastbuild".The“W”columnrepresentsthe“weatherreportshowingaggregatedstatusofrecentbuilds.”Jenkinsusethesetwoconceptstointroducethegeneralsituationofabuild.

Figure7.1:Dashboardstatuswindow

Status:Therearefourkindsofstatusofabuild.

Successful: Buildiscompletedandconsideredstable.

Unstable: Buildiscompleted,butisconsideredunstable.

Broken: Buildisfailedduringbuilding.

Disabled: Buildisdisabled.Weather:Jenkinswillreleasearobustvalue(from0-100)forbuildbasedonsomeprocessortasks,

whichbeimplementedthroughplugins.Thosetasksmayincludeunittest(JUnit),coverage(Cobertura)andstaticcodeanalysis(FindBugs).Higherscoreindicatesthebuildismorestable.Buildstabilityscorelessthan80pointsisconsideredasunstable.

Table7.1:WeatherStatus80+ 60-79 40-59 20-39 0-19

CAReleaseAutomation(CA_Technologies,2015b)Artifacts:theobjectsthatmakeupthesoftwareapplicationbeingdeployed.Components:acollectionoftheobjectstobuildaprocessActions:anindividualoperationinthetargetdeploymentenvironmentFlow:agroupofactionswithadefinedsequenceProcess:asetofactionsexecutedinalogicalorderEnvironment:agroupofcomputerstowhichdeploymentsaredeployedsimultaneouslyReleaseTemplate:eachtemplateiscontainedwithinCategoryandprovidestheprocessesthat

needtobeexecutedforreleasealongwiththeordertheyshouldbeexecutedPipeline:awaytoexpressasequenceofoperationsoverDevOps

51

AppendixB:InstallingJenkinsPluginsJenkins isanopen-sourceandextensiblecontinuous integrationsoftware.Therearemore

than1,000publiclyavailablepluginsthatarereadytouseinanyinstanceof Jenkins. InformationandexplanationonJenkinspluginsareontheJenkinswikiwebsite(Kawaguchi,2015).Theactualplugins,aswellas their sourcecode,areprimarilyon JenkinscommunityrepositoriesonGithub.Through this manner, it allows easier contributions from any community developer. So in thecircumstance that anoriginalplugindeveloper leavesbehindhis/herplugin,plugindevelopmentcanstillcontinuethankstothecommunity.AftermeetingprerequisitesandcreatedapluginGithubrepository,aplugindevelopercanmessagethecoreJenkinsteamtohaveadedicatedpageontheJenkinswiki.

On apluginpageon the Jenkinswiki, for example in thebelow figure7.2 that shows thePromoted Builds Plugin, one can seemany details about a plugin: plugin name, release version,releasedate, required Jenkins core version, dependencies, graphofmonthly installations, changelog,sourcecode,issuetracking,pullrequests,authors,andnumberofinstallationsbymonth.

Figure7.2:PromotedBuildsPlugindescriptionpage.Thisappendixassumesyouhaveproperlyinstalledandhavedonethebasicconfigurationsfor

Jenkins. The following are instructions showhow to find and install plugins through the Jenkinswebdashboard:

1. OnthehomepageofJenkins,click“ManageJenkins”,asseeninfigure7.3:

52

Figure7.3:ManageJenkins.

2. Fromthelistofconfigurationoptions,click“ManagePlugins”,asseeninfigure7.4:

Figure7.4:ManagePlugins.

3. Youshouldseetabslabeled“Updates,”“Available,”“Installed,”and“Advanced”.Tofindandinstallanewplugin,clickthe“Available”tab,asseeninfigure7.5.Selectthedesiredpluginandthenclick“Installwithoutrestart”(youcanalsoclickthepluginnametobetakentotheplugindescriptionpageontheJenkinswiki):

Figure7.5:Installinganavailableplugin.

4. Theplugininstallationscreenshouldlooklikefigure7.6.Oncecomplete,youcanusethenewlyinstalledpluginwherevertheplugincanbeused,asseeninfigure7.7.

53

Figure7.6:Plugininstallationstatusscreen.

Figure7.7:ChuckNorrisPlugin.

Ifyouhappentohaveacustom-madeJenkinsplugin,whichshouldbea .hpi file, thenyoucanimportitintoJenkinsinthe“Advanced”tabofManagePluginsandthenimportingsaidfile,asseeninfigure7.8.

54

Figure7.8:Uploadingacustomplugin.

55

Appendix C: Nolio InstallationInstructions

1. Request two functional groups to be created, one labeled Dev and one labeled Ops.FunctionalgroupsareBarclayssystemgroupsfordeveloperstocoordinatewitheachother.

2. Request that the twogroupsbeadded to the “sandpit”Nolio instance.An instancecanbeconsidered a Noliowebsite dashboard. Barclays has two different instances: sandpit (forlearningandfamiliarizationpurposesonly)andproduction(foractualdevelopmentofflowanddeployments).

3. Download a zip file that contains Barclays’s Nolio extended version template from thetutorialpage.

4. GototheNoliosandpitwebsitedashboardandentertheROC.5. Importthezipfilemakingsuretoimportallcomponents.6. SetyourapplicationasBarclays’sNolioextendedversionplatformtestapplication.7. NavigatetoDesigner->ProcessDesign->Components.8. Assign shared components from the imported components. For the tutorial, it stated to

assign deploy copy & uncompress, windows service start & stop, and email utilitycomponents.

9. Mapaservertype,thetutorialstatedApp_Name1,foreachsharedcomponent.10. NavigatetoDesigner->ProcessDesign->Processes.11. IntheprocessDeployment,selectNotify_Deploy_Beginandclick“AddAction/Flow”forthe

emailutilitycomponent.12. Selectthef_send_release_begin_emailflowfromthedropdownlist.Saveandpublish.Make

suretoassigntheprocesstotheUATenvironment.13. Repeatsteps11and12forthefollowingastable7.2.

Table7.2:ProcessareasandflowsProcessArea FlowPost-Deployment->Notify_Deploy_End f_send_release_end_email (from email utility

sharedcomponentDeployment->Stop_System f_stop_component (from windows service start

&stopsharedcomponent)Deployment->Start_System f_start_component(fromwindowsservicestart

&stopsharedcomponent)Deployment->Deploy_Binaries f_deploy_component (from deploy copy &

uncompresssharedcomponent)14. NavigatetoEnvironment->AgentAssignment.15. MakesurethatyourNolioagentisinstalledandrunningproperly.Forourteam’scase,we

installed theagent,which is awindowsservice running in thebackground,ononeofourlocalworkcomputers.

16. Selecttheappropriateservername(forourcaseitwasthelocalcomputerhostname)andassignittotheservertypeApp_Name1.AlsosettheenvironmentasUAT.

17. Repeatstep16fortheservertypeutil.Email.18. NavigatetoArtifact->ArtifactManagement.19. SelecttheartifactSC.CompressedandselectAddArtifactDefinition.

56

20. Be sure to set the file type of the definition as zip and ensure that the server typeApp_Name1isselectedforthedefinition.

21. SelectthenewlycreatedArtifactDefinitionandthenclickAddVersion.22. Version canbe anyversion.The artifact source shouldbe the full pathwhere the artifact

sourcefileshouldbe(steps18-20didnotworkforourcase,soweskippedthesesteps).23. NavigatetoReleases->TemplateCategories.Click“New”.24. Createanewtemplatecategorywithanyname.25. Withinthenewtemplatecategory,click“New”andcreateanewdeploymenttemplate.26. Withinthenewdeploymenttemplate,click“AddStep”.Addthestepsasshownbelow:

Table7.3:PorcessesandCategories

27. Drag anddrop all deployment steps to the center tab called “Deployment Steps” or “PostDeployment”.Thisiswhenyoucanalsosetdependencies.

28. NavigatetoEnvironments->ParameterConfiguration->Parameterstab.29. SelecttheUATenvironment.Youwillseealistofthesharedcomponentsweselected.Itis

here that a user can set up the email addresses for emails to be sent to, the specifiedwindowsservicestostopandstart,andtheartifactdefinitionpath.

30. NavigatetoReleases->DeploymentPlans.Click“New”.31. Inthenewlycreateddeploymentproject,click“NewDeploymentPlan”.32. Withinthesettingsforanewdeploymentplan,youmustsupplyabuildnumberandhave

theplanusethetemplatethatwecreated.33. Click on the newly created deployment plan to view details. There should be a “Deploy”

button,whichisthefinalstep.34. Click“Deploy”andyoumustsupplysomesettings,forthetutorial’scaseyoumustselectthe

UATenvironment.35. Alldone!

Process CategoryNotify_Deploy_Begin DeploymentStop_System DeploymentDeploy_Binaries DeploymentStart_System DeploymentNotify_Deploy_End PostDeployment

57

AppendixD:job_per_branch.groovy/* WPI DevOps MQP Khazhismel Kumykov <kkumykov@wpi.edu> <khazhismelkumykov.kumykov@barclayscapital.com> job_per_branch.groovy Overview: This script gets a list of branches from the given SVN repository, sorted by the date of the newest commit in the branch. The script then searches for jobs existing in Jenkins with the *exact same* name as the branch. The script will the matching job for the newest branch as a template for creating new jobs. The script will create new jobs for up to `BRANCHES_TO_KEEP` branches. The script will disable jobs after `BRANCHES_TO_KEEP` branches. The script will *delete* jobs after `BRANCHES_DELETE_AFTER` branches. Branch names *must be* unique between projects. Using: - Must create and configure a base branch job for one of the branches in SVN. - The job must be configured such that copying and renaming the job are the only required configuration changes to allow the job to work for a new branch. (i.e., use $JOB_NAME for SVN branch selector, etc.) - Must create a new Job for each SVN repository you wish to be managed. - Add a required Credentials Parameter named "SVN_CREDS" with the default set to the 'username with password' credentials for your SVN repository. - Check "Prepare environment for the run" - Check "Keep Jenkins Environment Variables" - Check "Keep Jenkins Build Variables" - Add Parameters as needed to "Properties Content" (Do not surround in quotes) e.g.: BRANCHES_REGEX=^WPI_Copy_.* SVN_REPO=http://svn/repos/WPI/ - Set the Job to build periodically as appropriate. - Add a "Execute system Groovy script" build step, select "Groovy script file" and provide the path to this groovy script. - Optionally, you may configure global properties in the global configuration. - "Manage Jenkins" > "Configure System" > "Global Properties" - Check "Environment variables" and add key-value pairs as desired

58

Required Plugins - Subversion Plugin (Comes with Jenkins) - Groovy Plugin (http://wiki.jenkins-ci.org/display/JENKINS/Groovy+plugin) - EnvInject Plugin (https://wiki.jenkins-ci.org/display/JENKINS/EnvInject+Plugin) - Credentials Plugin (https://wiki.jenkins-ci.org/display/JENKINS/Credentials+Plugin) Params: BRANCHES_TO_KEEP - How many branches to keep active projects for. BRANCHES_DELETE_AFTER - How many branches to keep in Jenkins. If there are more jobs than this limit, extra jobs will be deleted. BRANCHES_REGEX - Regex for branches to look at. E.g. "^WPI_Copy_.*" - All branches starting with 'WPI_Copy_' E.g. "^[^_].* - All branches not starting with '_' SVN_REPO - The __root__ of the repository. E.g. "http://svn/repos/WPI/" SVN_CREDS - Must be a username-password credentials ID for accessing the SVN Repo */ import hudson.model.* import com.cloudbees.plugins.credentials.CredentialsProvider import com.cloudbees.plugins.credentials.common.StandardUsernameCredentials import org.tmatesoft.svn.core.io.SVNRepositoryFactory import org.tmatesoft.svn.core.SVNURL import org.tmatesoft.svn.core.auth.BasicAuthenticationManager def log(l) { println ("[SVN Branch Sync] $l") } def resolve(name) { bv = build.buildVariableResolver.resolve(name) if (bv != null) { return bv } else { return build.getEnvironment(listener).get(name) } } SVN_REPO=resolve("SVN_REPO") SVN_CREDS=resolve("SVN_CREDS") BRANCHES_REGEX=resolve("BRANCHES_REGEX") BRANCHES_TO_KEEP=resolve("BRANCHES_TO_KEEP") BRANCHES_DELETE_AFTER=resolve("BRANCHES_DELETE_AFTER") log "Beginning SVN Branch Sync"

59

log "SVN_REPO: $SVN_REPO" log "SVN_CREDS: $SVN_CREDS" log "BRANCHES_REGEX: $BRANCHES_REGEX" log "BRANCHES_TO_KEEP: $BRANCHES_TO_KEEP" log "BRANCHES_DELETE_AFTER: $BRANCHES_DELETE_AFTER" log "" // Lookup the credentials by ID cred = CredentialsProvider.lookupCredentials(StandardUsernameCredentials.class, Hudson.instance).findResult({it.id == SVN_CREDS ? it : null}) if (cred == null) { throw new hudson.AbortException("Could not find credentials with ID: $SVN_CREDS") } // Depending on how they are specified, they may be Ints or Strings. // Will throw if unable to parse. if (!(BRANCHES_TO_KEEP instanceof Integer)) { BRANCHES_TO_KEEP = Integer.parseInt(BRANCHES_TO_KEEP) } if (!(BRANCHES_DELETE_AFTER instanceof Integer)) { BRANCHES_DELETE_AFTER = Integer.parseInt(BRANCHES_DELETE_AFTER) } // Throw if this is invalid if (BRANCHES_DELETE_AFTER < BRANCHES_TO_KEEP) { throw new hudson.AbortException("BRANCHES_DELETE_AFTER should be greater than or equal to BRANCHES_TO_KEEP") } if (BRANCHES_TO_KEEP <= 0) { throw new hudson.AbortException("BRANCHES_TO_KEEP must be greater than 0") } items = Hudson.instance.items.groupBy{ it.name } def repo = SVNRepositoryFactory.create(SVNURL.parseURIDecoded(SVN_REPO)) repo.setAuthenticationManager(new BasicAuthenticationManager(cred.username, cred.password.getPlainText())) def branches = new ArrayList() repo.getDir("branches", repo.getLatestRevision(), true, branches) branches.sort{a,b -> b.date <=> a.date} branches = branches.findAll { log "Found Branch: $it.name - rev. $it.revision ($it.author) - $it.date" should_manage = (it.name =~ BRANCHES_REGEX).matches() if (!should_manage) { log "Ignoring Branch: $it.name"

60

} return should_manage } log "" // Find newest branch that exists newestBranchWithJob = "" for (branch in branches) { if (items.containsKey(branch.getName())) { newestBranchWithJob = branch log "Using ${branch.getName()} as template job." break; } } log "" // Throw if we can't find a template job. if (newestBranchWithJob == "") { throw new hudson.AbortException("Could not find branch job to copy!") } templateJob = items[newestBranchWithJob.name][0] // Iterate over every branch that we aren't ignoring branches.eachWithIndex({branch, idx -> jobExists = items.containsKey(branch.name) job = null if (jobExists) { job = items[branch.name][0] } if (idx >= BRANCHES_TO_KEEP) { // If job exists, we want to disable or delete it if (jobExists) { if (idx >= BRANCHES_DELETE_AFTER) { log "${branch.name} - Deleting job ${job.name}" job.delete() } else { if (job.disabled) { log "${branch.name} - Doing nothing for job ${job.name} - it's disabled as it should be. It will be deleted soon." } else { log "${branch.name} - Disabling job ${job.name} - it will be deleted soon" job.disabled = true job.save() } } } else { log "${branch.name} - Doing nothing - it should be deleted, and

61

is." } } else { // If job doesn't exist, we want to create it. if (jobExists) { if (job.disabled) { log "${branch.name} - Enabling job ${job.name}" job.disabled = false job.save() } else { log "${branch.name} - Doing nothing for job ${job.name} - it exists as it should" } } else { log "${branch.name} - Creating new job ${branch.name} by copying ${templateJob.name}" newJob = Hudson.instance.copy(templateJob, branch.name) // This seems to be required to enable building newJob.disabled = false newJob.save() // Run a new build right after making the job. newJob.scheduleBuild(0, new Cause.UpstreamCause(build)) } } }) // build.result = Result.UNSTABLE log "" log "SVN Branch Sync Complete"

62

AppendixE:ExcelReportingusingSQLdatabase1) Within Excel, click the Data tab and then click “From Other Sources” -> “From SQL

Server”,asseeninfigure7.9.

Figure7.9:ImportaSQLdatabasedata2) In theDataConnectionWizard, fillout theSQLservernameaswellasusernameand

passwordlogoncredentials,asseeninfigure7.10.

Figure7.10:DataConnectionWizard

63

3) Nextyouneedtoselecttheappropriatedatabasethatourpluginextractedthedatato.Youalsoneedtoselecttheappropriatetablefromthelist:

Figure7.11:Selecttheappropriatedatabase4) At the next window, you’re prompted to write a name and description for the Data

Connectionfilethathasthisconnection.Whenfinished,click“Finish”:

Figure7.12:writeanameanddescriptionforthefile

* database name

64

5) Thenextwindowwillpromptyouonhowyouwanttoviewthedata.Makesure“Table”is selected and then click “Properties”, as seen in the left of figure 7.13. In theConnectionPropertieswindow,asseeninrightof figure7.13,youcanedittherefreshsettings in the “Usage” tab. Click the “Definition” tab; the important areas here are“CommandType”and“CommandText”.MakesureCommandTypeissetto“SQL”.

Figure7.13:Importdata(left)andpropertieswindow(right)6) Paste in the followingSQLcommand into theCommandText, as seenbelow , (please

notethe*YOUR_DATABASE_NAME*partsofthecommandshouldbereplacedwiththedatabasenameyouselectedfromStep3).Oncecomplete,clickOKandthenOKinthe“ImportData”window.

SELECT [jobName], [groupName], [number], [result], cast([time] as datetime) as [time], cast([finishTime] as datetime) as [finishTime], [promotionName], [promotionBuildNumber], [promotionBuildResult], cast([promotionBuildTime] as datetime) as [promotionBuildTime], cast([promotionFinishTime] as datetime) as [promotionFinishTime] FROM [*YOUR_DATABASE_NAME*].[dbo].[jenkins_mis_extract_promotions] as prom JOIN [*YOUR_DATABASE_NAME*].[dbo].[jenkins_mis_extract_builds] as build ON (build.id = prom.buildId)

65

7) A“SQLServerLogin”windowshouldpopuppromptingyouforthelogonusernameandpassword.Thetableshouldlooklikeourexampleshowninfigure7.14:

Figure7.14:Importedtable8) Itisbesttohavealltimerelatedcolumnsdisplayedasadate.Clickany“time”column

andthenright-clickandselect“FormatCells,”asseeninfigure7.15.Selecttheappropriatedate:

Figure7.15:FormatCells

66

9) Nextcomesthepivottablecreationprocess.Selectapartofthetable.Inthe“TableTools,Design”tab,click“SummarizewithPivotTable,”asseeninfigure7.16.MakesuretheentireTableisselected,thenclickOK:

Figure7.16:SummarizewithPivotTable

67

10) Youshouldnowbeonanewworksheet,withthefollowingPivotTableFieldListlocatedontherightsidebar,likefigure7.17shown:

Figure7.17:PivotTableFieldList

68

11) Drag-and-drop the fields into the areas as shown below (draggingpromotionBuildResulttoValuesshouldautomaticallysetitas“Count”:

Figure7.18:Customizerowlabels

12) Right-clickanyofthedateswithinthepivottableandclick“Group”,asseenintheleftoffigure7.19.Hereyoucanselectthestartdateandenddate,asseenintherightoffigure7.19,aswellasifyouwishtoviewitbyyears,quarters,months,days,etc.Forthistutorial,wewillusemonths:

Figure7.19:Groupbutton(left)andgroupingwindow(right)

69

13) Clickanywhereinthepivottable.Fromthetabsattheverytop,click“Options”under

“PivotTableTools.”Thenclick“PivotChart”,asseenintheleftoffigure7.20.Youcanselectanyappropriatechart,asseenintherightoffigure7.20,butonesuggestionisa“stackedcolumn”chart.Nowyouhaveapivottableandassociatedpivotchart!Youcanmakeanyadjustmentsthankstotheversatilityofpivottables.

Figure7.20:PivotChartbutton(left)andChartstyleselection(right)14) Figure7.21isthepivotchartforourexampledataset:

Figure7.21:SamplepivotchartIf you would like to have only environments on the X-axis and the column bars beingprojectnames,thenfollowthesesteps:

70

15) Makesurethedrag-and-dropfieldsaresetupasshowninfigure7.22:

Figure7.22:Customizecolumnsandrows

16) Createapivotchartfromthenewpivottable.Onceagainwerecommend“stackedcolumns.”Nowyouhaveapivotchartshowingallenvironmentsforeachmonththathascolumnsfilledwithtotalpromotionsperproject.Youcanadjustthedesignasyouseefit.Ourexamplechartisshowninfigure7.23:

Figure7.23:Samplechart

71

Appendix F: Excel Reports using CSVdata

1) Asseeninfigure7.24,fromthefirstcell,intheabovecaseit’sG1,highlightalldatauntilthelastcell(youcanaccomplishthisquicklybydoingctrl+shift+rightarrowkey,andthenctrl+shift+downarrowkey).

Figure7.24:raw.CSVdatasetinExcel2) Intheleftoffigure7.25,intheInserttab,clickPivotTable.Usethedefaultselected

tableorrange.Intherightoffigure7.25,youmaychoosewhereyouwouldlikethePivotTablereporttobeplaced;itwillbeplacedinanewworksheetbydefault.

Figure7.25:InsertPivotTablebutton(left)andcreatePivotTablewindow(right)

3) ThefollowingstepsaresimilartotheinstructionofreportingusingSQLdatabase.

72

Appendix G: Features of our pivottableconfigurationGroupreportfilter

By default, all applications and their respective information from the original data aredisplayed. Using group report filtering, you can specify either one or more applications to beviewable.Clickingthedropdownarrownextto(All)showstheotherapplications.Seefigure7.26:

Figure7.26:Groupreportfilter.

Promotionresultcolumnlabel

Eachvaluecolumnrepresentsapromotionresult:SUCCESS,FAILURE,andUNSTABLE.Youcanchangetheorderingviasorting,orfilterouttoseeonlydesiredpromotionresults.ClickingthedropdownarrownexttoColumnLabelsshowsallpromotionresults.Seefigure7.27:

Figure7.27:Promotionresultcolumnlabel.

73

Promotiontimerowlabel

Ourpivot table configurationhas it setuponamonthlybasis. It’s possible to change thepromotiontimegroupingtodaily,quarterly,annually,ora“Xnumberofdays”variation.Youcanalsospecifytheendandstartdaterange.Rightclickamonth,select“Group”andchangethedatesettings.Seefigure7.28:

Figure7.28:Promotiontimerowlabel.

Promotionnamerowlabel

Bydefault,applicationsonPrimeFinancing’s JenkinshaveQA,UAT,andPRODpromotionenvironments. It’s possible to set up a custom promotion environment, such as DEV, in theonboardingprocess.Onecansetonlycertainpromotionenvironmentstobeviewableinthepivottable. Click a promotion name and then click the dropdown arrownext toRowLabels to see allpromotionenvironments.Seefigure7.29:

74

Figure7.29:Promotionnamerowlabel. Thefinalpivottable/pivotchartaswellastheSQLDatabaseconnectionconfigurationcanserveasa template fornewreports forPrimeFinancing tocreate in the future.Forexample, sayyou’reamanagerforagroupofapplicationteamsandyoucurrentlyhavepromotiondataforeverymonth exceptDecember this year. Towards the endofDecember, if youmanually tell Jenkins toextractthenewdataintotheongoingSQLdatabase(orsetJenkinsuptoextractonaregularbasis),thenwhenyourefreshthepivottableitwillhavethenewDecemberpromotioninformation.Onecanadjust therefreshsettings for thedatabase tableretrievalaswellaspivotchartrefresh.Thishelps in further removing manual steps for reporting. Additionally, when considering thepromotiontrendorevenaforecastofthepromotioncount,onecaneasilyuseatrendlineontopofthe pivot chart, as seen in figure 7.30. The figure has a linear trendline that also forecasts twoperiods ahead. For our pivot chart configuration, the trendline can only be viewed for a specificpromotionresult.

75

Figure7.30:Pivotchartwithtrendline.

76

AppendixH:DashboardViewPluginHereareinstructionsonhowtousetheDashboardViewPlugin(Hayes&Ambu,2015):

• OntheJenkinsmainpage,clickthe“+”tabtostartanewviewwizard.

• Giveitaname,andbesuretoselect“Dashboard”fromthelistofpossibleviewsasshownin

figure7.31

Figure7.31:AddanewviewwhenusingDashboardViewPlugin• For the new view configuration, besides selecting the appropriate jobs, themain area to

focuson is the“DashboardPortlets”area.Youcanaddadashboardportlet tothetop, left

column,rightcolumn,andbottomoftheview.Theviewlayoutisasfollows:

Topportlet1…………………………………………………………………….Leftportlet1 Rightportlet1Leftportlet2 Rightportlet2Bottomportlet1………………………………………………………………..

DefaultAvailablePortlets

There are several default available portlets within this plugin, and we introduce these

portletsinthissection.

StandardJenkinsjobslist

InFigure7.32,thisisthedefaultJenkinsjoblistasseeninanormalJenkinsview.

77

Figure7.32:Jenkinsjobslistunderdashboardview

Jobstatistics

Infigure7.33,thisportletshowsstatisticsbasedonjobs’health.

Figure7.33:Jobstatisticsunderdashboardview

78

Buildstatistics

Infigure7.34,thisshowsstatisticsbasedonbuildstatus.

Figure7.34:BuildstatisticsunderDashboardView

Jobsgrid

Infigure7.35,thisjobsgridportletdisplaysamulti-columntableajob’swithcurrentstatus,

weather,clickablelinktothejob,andbuildbutton.

Figure7.35:JobsgridunderDashboardView

Unstablejobs

Infigure7.36,thisshowsthestatus,health,andaclickablelinktoanunstablejob.

Figure7.36:UnstablejobsunderDashboardView

79

Teststatisticsgrid

Figure7.37showsdetailedtestdatafortheconfiguredjobs.

Figure7.37:TeststatisticsunderDashboardView

Teststatisticschart

Figure7.38showsapiechartoftheconfiguredjobs,includingpassing,failing,andskipped

jobs,withtotalnumberandpercentages.

Figure7.38:TeststatisticschartunderDashboardView

Testtrendchart

This is a chart, as seen in figure 7.39, which shows all tests over time in aggregate. For

everyday, since the first job was built in the Dashboard View, the trend chart shows the total

80

numberofpassing,skipped,andfailingtestsinaggregateacrossthebuild.Itassumesthatifabuild

didnotoccuronagivenday,thepreviousversionofbuildresultswillbeused.

Figure7.39:TesttrendchartunderDashboardView

ThepluginalsohasanIframePortlet,ImagePortlet,andSlavesStatisticsportletavailable.

Below,figure7.40,isanexampleofafullDashboardviewwithdefaultportlets:

Figure7.40:DashboardViewwithdefaultportlets

81

AdditionalExtendPortlets

BesidesthesedefaultavailableportletswithintheDashboardViewPlugin,newportletscan

becontributedviaotherpubliclyavailablepluginsoryourowncustomplugin.

ProjectStatisticsplugin

In figure 7.41, this portlet is called “Number of builds” that shows each job’s successful,

unstable,orfailedbuilds(Ambu,2015).

Figure7.41:Numberofbuildsplugin

ProjectBuildTimes

Theportlet,showninfigure7.42,iscalled“RMBuildTimesChart”whichshowscumulative

andseparatebuildtimes(Rhine,2014).

Figure7.42:RMBuildTimesChart

LatestBuildswithBadges

Asfigure7.43shown,itisanewsupportivepluginofDashboardViewcalledLatestBuilds

withBadges,whichwascreatedbyour team.Thispluginserves thesamepurposeas thedefault

LatestBuildsportlet,butwithanadditionalcolumnthatshowstheavailablebadgesassociatedwith

ajob’sbuild,suchaspromotions,configurationchangesandsoon.

82

Figure7.43:LatestBuildswithBadges

ExtensiblenatureofDashboardViewplugin

TheDashboardViewplugin’sarchitectureallowsforextensions/customportletsfromany

additionalplugin.Asretrievedfromtheplugin’swikipage,thesegeneralstepsshouldbefollowed:

• Extend the DashboardPortlet class and provide a descriptor that extends the

Descriptor<DashboardPortlet>.

• Createajellyviewcalledportlet.jelly.

Forportletsthatwishtousecustomparameterssuchasadefaultnumberofbuildstoshow,

onecandothatasfollows:

• Createajellyfilecalledconfig.jellytobeusedwhentheportletisconfigured.

• Modifyconstructor(with@DataBoundConstructor)toreceivethenewparameters.

Trialanderrorcanhelpwiththecreationprocess,aswellassimply lookingatthesource

code for Dashboard View or any extension plugins such as the previously mentioned ones.

Examplesofsamplecodeforanextensionpluginareshowninfigure7.44andfigure7.45.

83

Figure7.44:samplecodeofMyPortlet.java(Hayes&Ambu,2015)

Figure7.45:samplecodeofportlet.jelly(Hayes&Ambu,2015)

84

Appendix I: Extension of High SpeedEquitiesTradingResearchStudy1.Introduction

InJanuary2015,HansR.StollpublishedastudyintheAsia-PacificJournalofFinancial

StudiesexaminingHighSpeedEquitiesTrading,A.K.A.HighFrequencyTrading(HFT),anditseffect

onthegeneralmarketduringtheyears1993to2012(Stoll,2015).Thisappendixseekstoexamine

Stoll’sstudyandattempttoreplicateandextendanumberofhisfindingsusingpubliclyandfreely

availabledata.Inparticularweaimtoexpandhisobservationsthroughthemostrecentfewyears

todetermineiftheconclusionsthathehaddrawnremainrelevantandaccurateinthecontextof

thisnewinformation.WhileStolluseddatafromboththeNewYorkStockExchange(NYSE)and

Nasdaq/OMXtosupporthisclaimsoftheeffectthatHFThasonthemarket,ourteamfocusesonly

oncollectingandanalyzingdatafromtheNYSEwhichwouldbothlimitthescopeofourresearch

andallowmorefocusonextendingtheexistingconclusionstothecurrentday.Ourconclusionsare

drawnfromthisresearchandseektobolstertheconclusionsStollformedaswellasintroduceour

ownanalysisofthedatatodeterminetheimplicationsofHFTonthegreatermarket.

2.Background

Withtheriseofcomputingtechnology,andtheintroductionofthistechnologyintothe

financialsector,afamilyofalgorithmictradingstrategiesknownbroadlyasHigh-Frequency

Trading(HFT)emerged.Thissetofstrategiestakesadvantageofelectronictradingtoolsthathave

replacedhumanbrokers,aswellaslow-latencytechnologyandhigh-speedconnectionstoperform

tradespreciselyandefficiently.Thesetoolsallowtraderstoimplementnewtechniquessuchas

algorithmicallytradingshort-termpositionsquicklyandinhighvolumestomakeaprofitonthe

minutemarginsofchangeinequityprices.Theyalso,however,allowtechniquessuchas“quote

85

stuffing”whichattemptstodenyinformationtoothertradersbyoverwhelmingthetradequeue

withquotesthatarethenrescindedatthelastminute.Thesestrategies,andotherderived

strategies,however,havebeencriticizedascontributingfactorstoso-called“flashcrashes”that

havecostinvestorstrillionsofdollars.

OneofthemorewellknownflashcrasheswasthecrashofMay6,2010.Inlessthananhour,

securitiessuchastheDowJones,NasdaqComposite,andS&P500allcollapsedandreboundedvery

quickly;theDowJonesinparticulardroppedmorethan600pointsbeforerebounding.Nobody

expectedorcouldpredictthe2010flashcrash.Oneofthemostdiscussedpotentialreasonsforthe

flashcrashisHFT,sinceonecanexecuteatechniqueinHFTwherethecomputerprogramwould

setupalargetradetobeplaced,butthencancelitattheverylastsecond.Thistechniqueknownas

quotestuffing,aspreviouslydefined,allowstraderstoartificiallymanipulatethemarket.Itcanalso

disruptthemarketandcauseextremeanomaliesasseeninthe2010flashcrash.

3.Data

ToinvestigatetheeffectofHFTonthemarket,andtoreplicateaswellasextendStoll’s

researchpaper,ourteamanalyzeddatafromtheNewYorkStockExchange(NYSE),bothforthefull

exchangeandindividualequities.Duetotimeconstraintsaswellastheresourcesavailabletous,

wechosetoonlyextendthestudy’sanalysisfortheNYSEratherthanalsoincludinganalysisonthe

NasdaqasseeninStoll’spaper.Inordertoreproducetheresultsfromthestudy,theteamexamined

300totalNYSEstockswithdataavailablefrom1993untilpresent;wegrouped100stocksfrom

eachofthreemarketcapitalizationgroups:large,mid,andsmallcap.Marketcapitalizationis

calculatedbymultiplyingacompany’ssharesoutstandingbythecurrentmarketpriceofoneshare

(MarketCap,2016).ThisdatawaspulledandanalyzedfromYahooFinanceusingpythonscriptsto

accesstheirAPI.Wethenusedtheavailabledataforthesestockstocalculateseveralrelevant

measurestodeterminethepotentialimpactofHFTontheNYSEduringthistime.Theresultofthe

team’sinvestigationfollows.

86

4.Tradesizeandtradefrequency

Thefirstmeasurementtheteamattemptedtoreplicatefromthestudywasthechangein

averagetradesizeoftransactionsontheNYSEduringtheperiodinquestion.Sincetherequisite

datatocalculatethismeasureforindividualstockswasnotfreelyavailable,theteamcompromised

tocompareStoll’sstudy’sfindingstoourexaminationoftradesizechangesacrosstheentire

exchangeoftheNYSE.WeusedthedataavailableontheNYSE’swebsite(NYSE,2016)tofindthe

dailyvaluesfortheamountofavailablesharesintheexchangeandtheamountoftradesonthe

givenday.Wehadtoconcatenatethe1993-2000datawiththerecent2000-presentdatainorderto

considerallvalues.Theaveragetradesizeonagivendaywascalculatedbytakingofthetotal

amountofsharesavailableanddividingbytheamountoftradesdone.Theaveragetradesizein

sharesisplottedforcomparisonandanalysisinFigure1.

Figure1:AveragetradesizeacrosstheNYSEforyears1993-2015

87

It’sworthnotingthespikeanomalyinourgraphataroundthebeginningof2004.Thisis

duetoaninconsistencyinthedatathatisprovidedbytheNYSEthroughtheConsolidatedTape

Association(CTA)whichprovides“supplemental”tradedatafortradespriortoJanuary2004,but

notfortradedataafterthisdate.Thisinconsistencymanifestsitselfasasharpjumpinthe

calculatedAverageTradeSizeatthattimeduetoadecreaseinthereportednumberoftrades

performed.

WhenlookingattheoveralltrendoftheNYSE’saveragetradesizeacrosstime,Stoll’s

researchpaperfoundasteadydecreaseinrecentyearsinaveragetradesize.Thiscorrelateswith

anincreaseintheuseofalgorithmstoperformtradesquicklyaswellasadeclineintheauxiliary

costsinperformingtradesthatwouldallowfortraderstoperformahighernumberofsmaller

volumetrades.Fortunately,ignoringthespikeinthebeginningof2004,ourdatafitsthistrendas

well,demonstratingthattheeffectisconsistentevenacrosstheexchangeasawholeandinthe

recentyearsof2012-2015.

5.Effectofhighfrequencytradingonmarketquality

5.1Bid-Askspread

Oneofthesuggestedeffectsoftheintroductionofhighspeedtradingwasadecreaseinthe

bid-askspreadonaverage.TheStollpapersuggestsatrenddownwardswithasharpdeclinein

2001thattheyattributetoaswitchfromfractionaltodecimalstocks,andageneraldownward

trendthereafter.

Inordertocalculatebid-askspreadweusedanestimatorasdescribedinCorwinand

Shultz’sresearchpaperusingdailyhighandlowprices(Corwinetal.,2012).AsseeninFigure2,we

donotseeasimilardramaticdropin2001thatwasattributedtoachangefromfractionalto

decimalpricing,whichmaybearesultoftheestimatorassumingthatbid-askspreadiscontinuous

ratherthandiscrete.Wedohoweversee,from2003to2007bid-askspreadismostlyconstant,and

after2010itisaswell,whichsuggeststhatevenwiththeintroductionandadoptionofhighspeed

88

trading,wedonotseeasignificantdropintheaveragebid-askspreadinthemarket.Thismatches

Stoll’sobservationthat“thereisnoindicationthatspreadsincreasedafter2005asHFtrading

becamemorecommon,”(Stoll,2015,p.777).Wedoseeaslightdownwardtrendforbid-askspread

forthehigh-capgroupcontinuingafter2013,whichmaysuggestaneffect,howevernotethatthe

decreaseisseenafteraveragetradesizehasstabilized.

Figure2:PercentagespreadofCorwin-Schultzbid-askestimator.

5.2Volatility

InasimilarveintoStoll’spaper,wewantedtoseeiftheeffectofHFThasanyeffecton

volatilityinthemarket,especiallywhenlookingatrecentyears.Generallyspeaking,onewould

guessanincreaseinvolatilitybecauseofthelargeandrapidamountoftradesoccurringduringHFT.

OneofthemethodsofcalculatingvolatilitythatStollusedwastocomputetheintra-dayvarianceof

trade-to-tradepricesandthenaveragethemforthedesignatedmonth.Unfortunatelyourteamdid

nothaveaccesstointra-daydataforourchosenstocks.Wedecidedtousethesecondmethodfrom

Stoll’spaperwhichinvolvescalculatingvarianceofdailystockreturns.

89

Foreachofthemarketcapstockgroups,weretrievedthedailyclosepricesandcalculated

theaverageofallthestocksforadesignatedday.Wethencalculatedthe365-dayvarianceofour

averagedailyclosepricesandplottedourresultsacrosstimeasseeninFigure3.

Figure3:VarianceofdailyNYSEstockreturns.

WhencomparingourfiguretoStoll’sdailyvariancevolatilitygraph,ourfigurefollowsthe

samegeneraltrend.Fromjustlookingatthehighcapstocks,thedot-combubbleandthe2007-

2008financialcrisisareverypronounced.Thestockvolatilitygenerallyhoversaroundtheaverage

closepricevarianceof10,withtheexceptionofanomalies.OurvolatilityanalysissuggeststhatHFT

canbecorrelatedwiththegradualincreaseinvolatilityovertherecentyears.

6.Conclusionandcomments

Althoughtheintroductionofelectronictradingbeganaround1999afterelectronic

exchangeswereauthorizedbytheSecuritiesandExchangesCommissionin1998(SEC),itis

importanttorememberthattheintroductionandusageofHFTstartedbecomingmoreprevalent

around2005.Takingthisintoconsiderationandwhenlookingattheaveragetradesizetrend,the

averagetradesizehasdecreasedasexpected.Thisdownwardtrendcoincideswiththeintroduction

90

andcontinuedadoptionofHFT.Wealsocanobservethattheaveragetradesizereachesalevelof

saturationafterthe2007-2008financialcrisis.Onemightinferthatfortheaveragetradesizeto

reachthissteadylevel,theproportionoftraderswhouseHFThasplateaued,leadingtoastability

inthismeasure.

Whenlookingatourresultsforbid-askspread,althoughouranalysisusedabid-ask

estimationformula,itisclearthatbid-askspreadhasstayedsteadysincetheintroductionofHFT

around2005-ignoringthe2007-2008crisisanomaly.EvenintheyearsofincreasedHFTadoption

inthelateryears,bid-askspreaddoesnotseemtocorrespondwiththeadoptionofHFT.

InregardstoHFT’seffectonvolatilityontheNYSE,itcanbeinferredfromourvolatility

analysisthatHFTcouldbeafactorinthegradualincreaseinvolatilityontheoverallmarket.After

2005,evenwhenignoringthespikeofthefinancialcrisis,volatilityhassteadilyincreasedacross

therecentyears.Thiscanbecorrelatedwiththeincreaseintradeactivitythankstotherapid

natureofHFT.

WecanthereforeconcludethatthereisnoevidencethattheintroductionofHFThasa

meaningful,negativeimpactonthemarket,atleastinthecaseoftheNYSEandbasedonthe

measurementswehaveexamined.OurresultisconsistentwiththeresultofStoll’spaperwherehe

alsoconcludesthatHFTfailedtohaveanegativeeffectonthemarket.

91

References:Corwin,ShaneA.,andPaulSchultz,2012,“Asimplewaytoestimatebid-askspreadsfromdailyhighandlowprices”,JournalofFinance,67,pp.719–759.MarketCapitalizationDefinition.Website.Retrievedfrom:http://www.investopedia.com/terms/m/marketcapitalization.aspQuoteStuffingDefinition.Website.Retrievedfrom:http://www.investopedia.com/terms/q/quote-stuffing.aspNYSEDataLibrary.Website.Retrievedfrom:https://www.nyse.com/data/transactions-statistics-data-library(SEC)SECURITIESANDEXCHANGECOMMISSION,“RegulationofExchangesandAlternativeTradingSystems”,17CFRParts202,240,242and249,ReleaseNo.34-40760;FileNo.S7-12-98;RIN3235-AH41.Retrievedfrom:https://www.sec.gov/rules/final/34-40760.txtStoll,H.,2015,“HighSpeedEquitiesTrading:1993–2012”,Asia-PacificJournalofFinancialStudies,43,pp.767-797.