in this webinar - cloudbees | enterprise jenkins and devops · in this webinar today’s presenters...

1©2015SOASTAInc.-Allrightsreserved.

InThisWebinar

TODAY’SPRESENTERS

TomChavez:SOASTA,Sr.ProductMarkeEngManagerAndyPemberton:CloudBees,SeniorDirector,SoluEonsArchitecture&ConsulEngTWOCOMPANIES,ASINGLESOLUTION

SOASTA:TheleaderinCloudandMobileTesEng

CloudBees:TheEnterpriseJenkinsCompany

Together,weofferacompletecloud-basedservice:

ConEnuousLoadTesEng&ConEnuousDeliverywithJenkins

QuesEons?AskinQuesEonwindow

3©2015SOASTAInc.-Allrightsreserved.

PollQuesEon

WhatbestdescribesyourstatuswithConEnuousIntegraEon/ConEnuousDelivery(CI/CD)?(singleanswer) a)  DeterminingifCI/CDisagoodopEon

a)  ImplemenEngCI/CD

a)  PracEcingCI

b)  PracEcingCD

c)  WhatisCI/CD?

4©2015SOASTAInc.-Allrightsreserved.

Agenda•  LoadtesEng:why?•  Whydon’tweloadtestearlier?•  ConEnuousLoadandPerformanceTesEng•  CloudBeesandtheEnterpriseJenkinsPlaYorm•  ConnecEngloadteststoyourCDPipeline•  Q&AQuesEons:SubmitinthequesEonboxduringevent

5©2015SOASTAInc.-Allrightsreserved.

WhyShouldWeLoadTest?

o  Ensurethatsitewillperformwellatload•  TesthardwareconfiguraEons

•  TestapplicaEon,database,components

•  Testthirdpartypartners

o  Tobereadyforpeakseason•  BlackFriday,CyberMonday,Christmas

•  Taxfiling,Insurancefiling,SuperBowlcommercial

6©2015SOASTAInc.-Allrightsreserved.

WhyShouldWeLoadTest?

7©2015SOASTA,Inc.-Allrightsreserved.

WhyDon’tWeLoadTestEarlier?

Copyright © 2014 SOASTA, Inc. - All rights reserved.

8©2015SOASTAInc.-Allrightsreserved.

WhyDON’TWeLoadTestEarlier?

“Because it’s hard.” o  Tools require coding skills we don’t have (C, VB?) o  We don’t have a scalable test environment o  We don’t have hardware for load generation o  It’s a different tool o  There are too many other tests to do

9©2015SOASTAInc.-Allrightsreserved.

WhyDON’TWeLoadTestEarlier?

“Because it’s hard.” o  Tools require coding skills we don’t have (C, VB?) o  We don’t have a scalable test environment o  We don’t have hardware for load generation o  It’s a different tool o  There are too many other tests to do

“Because it takes too long.” o  Scripting takes forever o  Results are slow o  Fast setup of environments is impossible o  We’re dependent on IT or other teams o  We’re too agile

10©2015SOASTAInc.-Allrightsreserved.

WhyDON’TWeLoadTestEarlier?

“Because it’s hard.” o  Tools require coding skills we don’t have (C, VB?) o  We don’t have a scalable test environment o  We don’t have hardware for load generation o  It’s a different tool o  There are too many other tests to do

“Because it takes too long.” o  Scripting takes forever o  Results are slow o  Fast setup of environments is impossible o  We’re dependent on IT or other teams o  We’re too agile

“Because no one ever told us to.” o  There are no “performance coverage” requirements o  There isn’t a performance baseline for reference o  There isn’t a defined performance target o  No one is managing to a performance metric

11©2015SOASTAInc.-Allrightsreserved.

WhyDON’TWeLoadTestEarlier?

“Because it’s hard.” o  Tools require coding skills we don’t have (C, VB?) o  We don’t have a scalable test environment o  We don’t have hardware for load generation o  It’s a different tool o  There are too many other tests to do

“Because it takes too long.” o  Scripting takes forever o  Results are slow o  Fast setup of environments is impossible o  We’re dependent on IT or other teams o  We’re too agile

“Because no one ever told us to.” o  There are no “performance coverage” requirements o  There isn’t a performance baseline for reference o  There isn’t a defined performance target o  No one is managing to a performance metric

Until Now.

12©2015SOASTAInc.-Allrightsreserved.

PollQuesEon

WhatdoyouseeholdingbackamoreiteraEveloadtesEngapproach?(checkallthatapply)

a)  Skillsgaps

b)  Lackoftestenvironment(s)

c)  Slowtestdevelopment

d)  TesttoolslimitaEons

e)  3rdPartyDependencies

f)  UndefinedPerformanceRequirements

g)  LackofManagementPush

h)  Other

13©2015SOASTAInc.-Allrightsreserved.

LoadTesEngHasBeenanEliteEvent

STheIssue:

• HighexpectaEonsfromtestsrunwithdiminishingEmewindows

• Code-levelperformancebugsfoundlatesetenEreprojectback

• Separateteamandscheduleconflictwithdeliverygoals

• MajorissuesescapetoproducEon

o  pecialtyskillsandtasksarealegacybarrier

Iteration 1

Automated unit tests

Iteration 2

Automated unit & functional tests

Iteration 3

Regression test suite

Iteration n

Regression test suite

Load & Performance Testing

Deploy

Issues?

14©2015SOASTAInc.-Allrightsreserved.

ConEnuousLoadTesEngValidateperformancepiecebypiece

Iteration 1 Automated unit

tests

Deploy

Production perf tests

Iteration 2 Automated unit & functional tests

Iteration 3 Regression test

suite Incl:

Iteration n Regression test

suite Incl:

End to End Production Scale

Test Small perf test Small perf tests

Perf smoke tests Perf smoke tests

TheSoluEon:•  RunincrementalperformancetestsautomaEcallywitheachiteraEon

•  Establishatrendlineforperformanceearly•  Developersfindcodeandarchitectureissuesearly

•  Performanceteamsfocusonend-to-endinfrastructureanduserrelatedtesEng

•  MinimizeproducEonescapesandimplementproducEontesEng

Development/Test team Performance/Ops team DevOps

15©2015SOASTAInc.-Allrightsreserved.

ConEnuousPerformance(forWeb&Mobile)ThePathtoaPerformanceCulture I JUST MADE A

CODE CHANGE

CloudTest runs load tests via Jenkins Daily (or more frequent) updates keep

teams “performance focused”

Identify bottlenecks & inform teams,

just like unit and functional

tests in Jenkins

Jenkins launches suitable cloud-based test environment

Manage with a new metric!

16©2015SOASTAInc.-Allrightsreserved.

PollQuesEon

WhattypesofLoadTesEngareyoudoingtoday?(SingleAnswer)

a)  NotdoinganyloadtesEng

b)  Non-WebProtocols(notmobile)

c)  Primarilywebprotocols

d)  Primarilymobilewebtraffic

e)  Bothwebandmobile

f)  Allweb,mobile&non-webprotocols

17©2015SOASTA,Inc.Allrightsreserved.

EnablingConEnuousPerformanceTesEngwithJenkinsPipeline

Andy Pemberton Twitter: @apemberton

Email: apemberton@cloudbees.com

18©2015SOASTAInc.-Allrightsreserved.

HaveyoumetJenkins?

#1 CI/CD Automation Engine

•  Open Source

•  Highly Extensible

About 8 years old

Easy to install/use

1000+ plugins

Widely adopted

•  100K+ installations

19©2015SOASTAInc.-Allrightsreserved.

Jenkins is the #1 Automation Platform Used for CI

Continuous Integration

Developer

Source Code

Control System

Compile/Build Code

Scan Tests Code Commit

Results Report

Build Commit Test

20©2015SOASTAInc.-Allrightsreserved.

Jenkins is Also Your Key to Continuous Delivery

Developer

Source Code

Control System

Compile/Build Code

Scan Tests Code Commit

Results Report

Build Commit Test Stage Deploy

Deploy Prod

Deploy Stage

Complex Delivery Pipelines

Delivery of App and Config

Continuous Delivery

Prod Dev

21©2015SOASTAInc.-Allrightsreserved.

Over 1000 Jenkins Plugins for Tool Integration

Integrated Development Environments (IDEs), editors/compilers/debuggers, code review and collaboration tools.

Source code control systems and related asset management tools.

Automated API-based testing frameworks.

Release management and release automation software

Automated API-based testing frameworks, static code analysis tools.

Load Testing Tools and Security Scanning technology

Automated and manual UAT tools

Release management, release automation, and change control software.

Environment and Configuration Management Software Develop

Commit

Build & Integrate

Unit Tests

Release (to Test)

Automated Regression Tests

Release (to Prod)

User Acceptance Tests

Load & Security Tests

22©2015SOASTAInc.-Allrightsreserved.

Jenkins is the Hub of the CD Ecosystem

On-premise or in the cloud

Plug-ins for all your tools

Run Deploy Stage Test Build Commit

23©2015SOASTAInc.-Allrightsreserved.

AreYouUsingToday?

23

Chances are someone in your

organization is.

24©2015SOASTAInc.-Allrightsreserved.24

25©2015SOASTAInc.-Allrightsreserved.25

How Do You Manage Jenkins and Testing at Enterprise Scale?

26©2015SOASTAInc.-Allrightsreserved.

About CloudBees

Jenkins founder, Kohsuke Kawaguchi, CloudBees CTO We optimize Jenkins for the enterprise

•  Jenkins Operations Center

•  Jenkins OSS Support

• Enterprise-class plugins

• Commit code and support the Jenkins Community

•  Jenkins training and consulting

•  Jenkins User Conferences (JUC) and newsletters

…we are the Jenkins Enterprise experts!

27©2015SOASTAInc.-Allrightsreserved.

CloudBees Jenkins Enterprise

27

Professional support

High availability

Large installations

Security

Optimized utilization

Continuous Delivery

28©2015SOASTAInc.-Allrightsreserved.

Additional Jenkins Services from CloudBees

28

Jenkins Certified Training •  Delivered via CloudBees partners

Jenkins Consulting •  Delivered via CloudBees Professional Services Team

Jenkins Newsletter •  Community news and expert tips

Jenkins User Conferences •  Learn from other community members

29©2015SOASTAInc.-Allrightsreserved.

HowCanCloudBeesJenkinsEnterpriseHelpwithYourCDStrategy?

29

PipelineStage View Provides deep insight into the execution of your workflow stages so you can detect problems and performance issues Manage complex “level-of-detail” testing pipelines

Jenkins Pipeline Created by CloudBees to provide Jenkins with a way to define flexible delivery pipelines that allow for a rich set of new delivery scenarios Manage and share “testing-as-code”

Dynamic Environment Creation Spin up new test/deployment environments automatically as part of your delivery pipeline

Deployment Tracking and Verfiication Track the history of your deployments – including who kicked off a deployment and what code changes were included in it Create pipelines that leverage verification tests to determine the quality of a new deployment

Jenkins Analytics Monitor and analyze performance across all your Jenkins instances Monitor, measure and manage adoption of testing, CI and CD best practices Restart builds, tests, and deployments where they

break Leverage pipeline checkpoints to restart delivery pipelines without a full (expensive) rebuild

...central source of orchestration to bridge dev, test, and ops alike

And more…

30©2015SOASTAInc.-Allrightsreserved.

#Jenkins2GreatlyEnhancedUI/UXforeasierstartupandbeOersecurity

31©2015SOASTAInc.-Allrightsreserved.

#Jenkins2JenkinsPipeline-as-Codefrontandcenter

32©2015SOASTAInc.-Allrightsreserved.

Jenkins Pipeline and Pipeline Stage View Enable Full Continuous Delivery Pipelines

Continuous Delivery

Prod Dev

Developer

Source Code

Control System

Code Commit

33©2015SOASTAInc.-Allrightsreserved.33

CloudBees Jenkins Pipeline Stage View

Restartable Checkpoints

Artifact Downloads

34©2015SOASTAInc.-Allrightsreserved.

Monitoring, Visualization and Analytics

Jenkins Pipeline Stage View Visibility into Pipeline status

Jenkins Analytics Cross-Team Build Analytics

… among many other features, tailored for enterprise organizations using Jenkins @ scale

35©2015SOASTAInc.-Allrightsreserved.

36©2015SOASTAInc.-Allrightsreserved.

ConEnuousLoadTesEng

o  SOASTACloudTestComponents/Architecture

o  Demo:

•  CloudTestUIOverview

•  JenkinsSetup•  DownloadingandinstallingCloudTestplug-in

•  CreaEngaPerformanceJobinJenkins

•  CreaEngaPipelineinJenkins

•  ReviewingResults

Whatarewegoingtoshow

37©2015SOASTA,Inc.Allrightsreserved.

Architecture:

Jenkins Jill

Sri Raj

DevOps Team 1

CloudTest Manager

Always up at: Cloudtestmanager.soasta.com

CloudTest server

any IP Address/ Domainname Up ‘as needed’ at

“US Grid1”

2 Azure California

2 AWS Virginia

2x Google Iowa

“WW Grid1”

2 Azure California

2 AWS Virginia

2x Google Iowa

CloudTest “Main” server

any IP Address/ Domainname Up ‘as needed’ at

dev.acme.com

CPU,DISK, Memory,I/O Etc.

38©2015SOASTA,Inc.Allrightsreserved.

Jenkins Enterprise on CloudBees

Jill

Sri Raj

DevOps Team 1

CloudTest Manager

Always up at: Cloudtestmanager.soasta.com

dev.acme.com

“US Grid1”

2 Azure California

2 AWS Virginia

2x Google Iowa

CloudTest “Main” server

any IP Address/ Domainname Up ‘as needed’ at

DemoPreview:

39©2015SOASTA,Inc.Allrightsreserved.

Demo

QuesEons?AskinQuesEonwindow

41©2015SOASTAInc.-Allrightsreserved.

Resources

ContactSOASTA:www.soasta.com/cloudtest/info@soasta.com866.344.8766Followus:

twiOer.com/SoastaIncfacebook.com/cloudtest

KnowledgeCenter•  WhitePapers•  WebinarRecordings•  CaseStudiesCloudLinkCommunity•  Support•  Tutorials•  Video

ContactCloudBeeswww.cloudbees.cominfo@cloudbees.com1.781.404.5100Followus:

twiOer.com/cloudbeesfacebook.com/CloudBees

www.SOASTA.com

www.CloudBees.comTryit@www.cloudbees.com/get-started

Documenta]on•  CloudBees Jenkins Platform •  Workflow in Jenkins WhitePapers:•  Making the Leap from Continuous Integration to

Continuous Delivery •  The Business Value of Continuous Delivery

HowtoIntroduceCloudBasedLoadTesEngtoyourJenkinsConEnuousDeliveryPipelineApril20,2016TomChavez,SOASTA@TomChavezAndyPemberton,CloudBees@apemberton