Scott Glasersalesforce, Sr. Manager Software

Engineering Vamshi Gandham

salesforce, Lead Software Engineer

DevOps in App Cloud

Salesforce IT

Forward-Looking StatementsStatement under the Private Securities Litigation Reform Act of 1995:

This presentation may contain forward-looking statements that involve risks, uncertainties, and assumptions. If any such uncertainties materialize or if any of the assumptions proves incorrect, the results of salesforce.com, inc. could differ materially from the results expressed or implied by the forward-looking statements we make. All statements other than statements of historical fact could be deemed forward-looking, including any projections of product or service availability, subscriber growth, earnings, revenues, or other financial items and any statements regarding strategies or plans of management for future operations, statements of belief, any statements concerning new, planned, or upgraded services or technology developments and customer contracts or use of our services.

The risks and uncertainties referred to above include – but are not limited to – risks associated with developing and delivering new functionality for our service, new products and services, our new business model, our past operating losses, possible fluctuations in our operating results and rate of growth, interruptions or delays in our Web hosting, breach of our security measures, the outcome of any litigation, risks associated with completed and any possible mergers and acquisitions, the immature market in which we operate, our relatively limited operating history, our ability to expand, retain, and motivate our employees and manage our growth, new releases of our service and successful customer deployment, our limited history reselling non-salesforce.com products, and utilization and selling to larger enterprise customers. Further information on potential factors that could affect the financial results of salesforce.com, inc. is included in our annual report on Form 10-K for the most recent fiscal year and in our quarterly report on Form 10-Q for the most recent fiscal quarter. These documents and others containing important disclosures are available on the SEC Filings section of the Investor Information section of our Web site.

Any unreleased services or features referenced in this or other presentations, press releases or public statements are not currently available and may not be delivered on time or at all. Customers who purchase our services should make the purchase decisions based upon features that are currently available. Salesforce.com, inc. assumes no obligation and does not intend to update these forward-looking statements.

DevOps Vision

Continuallyimprovingtheapplicationlifecycle,makingteamsmoreproductive,improvingcollaboration,anddeliveringhighqualityproductsfaster

• SalesforceOhana• Families are bound together and members cooperate and remember one another• SocialIntelligence isacombinationofmental,emotional,physicalandcommunity

intelligence

Collaborative Culture

Continuous Delivery Pipelines

Plan DeployBuildCode Test Release Monitor

Onepipelinepertechnologystack(apex,java,ruby,mobile,etc)

Continuous Delivery Transformation

Organizeteam

Environments

VersionControlSystem

ContinuousIntegration

TestAutomation

Metrics&Dashboard

ContinuousImprovement

Trail Starts Here

Team Organization

• Aprojectconsistsofoneormorescumteamso Averageteamsizeis7,sprintevery2weekso 250Engineers,50Teamso Useaprojectplaybook

• WeuseAgileAccelerator(GUS)

• Westoredocumentsingoogledocs

• Runlargeprojectswithsmartsheet

• Weusesalesforcechatter

Sandbox EnvironmentsDEV TEST UAT

Dev

UAT

Dev

Team

A

Dev

QA

Team

B

Dev

Team

C

Dev

Team

DTe

am E

Prod

PRDD

Config

Sand-box

fullSandbox

Production

PerProjectSandboxes

INT

Perf

PerOrgSandboxesRestrictedAccess

E2E(opt)

VCS Branching Strategy

Feature/HotFix

localDev1 Dev2

git.soma.salesforce.com

UAT

Project

CICI CICI CICI

Dev3 Dev4 Dev1 Dev2

MultiFeatureorFeatureDependency IndependentFeature

QA

INT QA

R 1.0 tag

Prod

Master

Dev

CI CI

Dev

CI

Continuous Delivery Flow

Artifacts

Commit

Test UAT ProdDev

Deploy

Build

LogsTest

ApexUnitCheckmar

xKumoniu

miPromoteGrinder

Test

Status

Tools

Metrics Dashboard

Apex CD Pipeline

ApexUnitTests(opensourced)

KumoniumUITests

CheckmarxScan(async)

CodeCoverage

Results

Results

Results

Results

BurpScan(async)

Results

GrinderPerfTests

QA&Perf

Results

Build&Deploy

Check-inVCS

JenkinsiPromote

GUS

Hadoop

UAT

CD Metrics

LeadTimeandCycleTime

ChangeSuccessRate

DeploymentSuccessRate

MeanTimeToRecover

CDMaturityIndex

Storiesandsprintacceptancemet,definitionofdone

Requirementstraceability,noopensprintstories

Allprojectcodeonmasterbranch

Completerunlistcreatedandtested

Designatedtestsexecutedandlogged

AllP1andP2bugsareclosed,openvsclosed

Deciding When To Release

Continuous Improvement - CD Maturity

Howdoyouknowyourcontinuously improving? Howdoyoumeasureteameffectiveness? Howdoyouprovideconsistencyacrossteams?

• Frequent DailyCheck-instoVCS

• Integration Builds on Dedicated Environment

• Force.comSecurityEssentials(SECDEV2)

• Use a CD Pipeline Status Dashboard

• Trend Bug Rates• Trend Manual vs

Auto Test Cases• % of SECDEV2

Completed

• 100% Auto Unit Test• Post Deploy Smoke

Tests• 50%Automated

Regression Testing• Passive Integration

with Static Analysis

• Automated Deployments

• Triggered Upon Successful Builds

• PostDeploySmokeTests

• VCSTriggeredBuildandTesting

• BuildsStoredInArtifactRepository

• Designated Security Owner Established

• Security Code Reviews

• Story Level Sec Bugs

• GenerateBuildIdtoTrackinCDPipeline

• DependencyManagement

• FeatureFlags• AutomaticCode

Promotion&BuildFromDynamicVMs

Building TestingDeployment Reporting

Leve

l2Le

vel5

Leve

l4Le

vel3

• AutoDeploysFromArtifactRepository

• Auto Deployment to Any Environments

• Auto Env Data Generation

• DeployToAnyEnv• PartialAutomated

RunlistDeploys• Auto Provisioning

of Any Environment• Auto Rollbacks

Upon Unsuccessful Deployments

• Blue-Green/Canary Deployments

• Dark Launches• FullyAutomated

RunlistDeploys

• Automatic Rerun Test Failures

• PreCheck-inVerification

• Manual Penetration Testing

• 100%AutomatedAcceptanceTesting

• Automated PerfTesting

• Integration with Penetration Testing Suites (Burp, Zap)

• 100%AutomatedRegressionTesting

• Use Test Suites to Categorize Tests

• Mocked Testing• Active Integration

with Static Code Analysis

• ContinuousAPM• Penetration Testing

Bug Trends• Req, Code, Test

Traceability

• Auto Create Bugs into Bug Tracking System upon Failure

• Monitor & Report Production App KPIs

• Automated Penetration Testing

• Trend Testing Coverage Percentage

• Track Code Changes to Pipeline Failures

• Security Bug Trending Per Sprint

Adv

ance

dIn

t

• Use a Branching Strategy and CheckAll Code into VCS

• Code Reviewed Before Every Check-in

• SecureDevelopment(SECDEV1)

• Auto Notification of CD Pipeline Stages

• Track Story Completion and Sprint Velocity

• Various Logging Levels For Each Pipeline

• % of SECDEV1

• All Test Cases Documented and Execution Tracked

• 75% Code Coverage• All Bugs Tracked In A

Bug Tracking System

• SeparateDev,Test,UATEnvironments

• Streamlined Deployment RunlistLe

vel1

Beg

inne

rB

ase

Exp

ert

CD MaturityMatrix

Followascrumpractice

Usesourcecontrol&branchingstrategy

Continuouslyintegrate(CI)

Automatetesting&deployment

Ifanythingfailsstoptheline

Immediatevisibility&feedback

Trackandmeasure

Useamaturitymodel

ContinuousDeliveryGuidingPrinciples

Open Source Projects

ApexUnit:

https://github.com/forcedotcom/ApexUnit

Kumonium:

Planned Release March 2017

Thank Y u