IBM Cloud Forum

20 novembre 2019New Cap Event Center, Paris

Cloud Forum / @ 2019 IBM Corporation

Philippe Mulet

Distinguished Engineer

IBM Cloud Dev Tools

Hybrid Cloud DevOps :

Dev & Test in public

Run anywhere

• IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice and at IBM’s sole discretion.

• Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision.

• The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract.

• The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.

• Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.

Please note

No clearly defined best practices

Best practices are often defined by the tools being used, not by the needs of the organization

Poorly defined tool integrations

Integrations between different proprietary and open source tools are often difficult to create and maintain

Limited ability to deploy across platforms

Solutions are often platform specific, and are hard to scale, secure and deploy across multiple clouds in hybrid environments

Challenges in implementing new technologies

Organizations are challenged to implement new technologies and best practices like Helm, Kubernetes, Istio, etc.

3IBM Cloud / © 2019 IBM Corporation

DevOps Transformation is Cloud-native, platform agnostic and hybrid focused

In the landscape of fragmented tools and processes, enterprises often find it difficult to integrate tools and practices from multiple vendors and open source projects

This results in an inability to enhance speed and quality, reduce maintenance costs and unify teams to engineer towards DevOps transformation success.

Develop in a public Cloud

Don’t assume development can’t use public cloud tools

just because production is on private

Data considerations

• Avoid sensitive data in issue tracking, else integrate on prem tool

• Remove sensitive data from source code, else integrate on prem tool

• Dev and Test environment with anonymized/public data

• Auth tokens/secrets stored on-premise vault, bound during deployment to actual environment (env secrets directly read from the apps, not flowing through CI/CD)

Leverage public cloud for Dev & Test regardless of where prod is deployed• Elasticity, TCO, Availability

• Security, Isolation (private network, per tenant encryption)

5

Hybrid Cloud DevOps : Dev & Test in public

Run anywhere

IBM Cloud / © 2019 IBM Corporation

Develop at the Speed of Cloud – with Control

Need both Speed and Control:

• Deploy >20 times per day with quality

• Automate toolchain with quality gates

Agile transformation requires Cloud to maximize benefits

DevOps is the way to develop efficiently for Cloud

IBM Cloud Continuous Delivery

Git Repos and

Issue Tracking

• Git repositories

• Modern Git workflow

• Merge requests

• Issue boards

• HA setup (multiple

availability zones)

Based on GitLab CE

Delivery

Pipeline

• CI/CD

• Easy setup

• Deploy to any cloud,

including hybrid

• Build pull requests• Our container

image or your own

New: Tekton

Eclipse Orion

Web IDE

• IDE in a browser

• Code completion

• Refactoring

• Git client

Based on Eclipse Orion

DevOps

Insights

• Collect quality data

• Establish policies

• Implement gates

• Analyze trends

Open

Toolchain

• Setup new projects quickly

• Integrate IBM and third-

party tools

• Reproduce best practices

with templates

• Access tools in one place

https://www.ibm.com/cloud/continuous-delivery

IBM Cloud DevOpsIntegrated Cloud Experience

8

Available Worldwide

• US South (Dallas), US East (Washington, DC), Frankfurt,

London, Tokyo

• Integrated with Identity Access Management

Security

• Regional data isolation

• Security - auditing, data encryption in motion and at rest,

continuous vulnerability scanning including QRadar for

application logs

• Compliance: ISO27K, GDPR, EU-managed, SOC2 (soon)

• Backup in geo - encrypted and GDPR compliant

Reliability

• Rearchitected on Kubernetes in 2018 for Increased

Reliability - exploits 3 availability zones per region for HA

60+IBM Cloud data centers

across 18 countries & 5

continents

ONECloudArchitecturerunning Watson, Data,

IBM Z, Blockchain

170+services with public,

private & local models

1,900Cloud –technology

patents granted in 2017

to IBM

1st

cloud provider to deliver hyper-data protection & commit

to GDPR compliance

9Page

IBM Cloud DevOps: Quick Overview

Speed with control

Create an integrated DevOps

toolchain

Deliver continuously & manage

composite pipelines

Edit your code from anywhere

Git repos & issue tracking

To learn more visit:

https://www.ibm.com/cloud/continuous-delivery

Continuous Delivery

Improve quality through insights

IBM Cloud DevOps / June 2018 / © 2018 IBM Corporation

10Page

Open Toolchain

A sample open toolchain for building, and deploying and managing three microservices

• Toolchains provide an integrated set of tools that

support the best practices to build, deploy and

manage your apps.

• You can create toolchains that include IBM Cloud

services, open source tools, and third-party tools

that make development and operations

repeatable and easier to manage.

• Rapidly instantiate new toolchains from templates

to on-board new teams quickly.

To get started visit:

https://cloud.ibm.com/devops

Create and manage toolchains of best of breed industry tools

IBM Cloud DevOps / June 2018 / © 2018 IBM Corporation

Get started quickly with toolchain templates1-click setup of sample code and fully configured toolchain

Get your teams up and running quickly

Customizable templates, so you can define your own

IBM Cloud DevOps / June 2018 / © 2018 IBM Corporation

Page 12

Add more tools or mix and match… it’s open!

Create new toolchain templates

Choose from a growing list of open source, third party, and other IBM tools

IBM Cloud DevOps / June 2018 / © 2018 IBM Corporation

• Cloud hosted social coding

• Commits, pull requests, issue tracking, graphs, …

• High resilience, multi zone architecture with regular off-site backups

• Based on GitLab™ Community Edition

Git Repos & Issue Tracking

Page 14

Delivery Pipeline

Easy Setup• Deploy an application from a Git repo in a few clicks.

Continuous Integration• Automate builds, tests and deploys for many types of code, running

builds automatically when code changes.• Follow GitOps best practices by building pull requests.

Deliver to Multiple Cloud Platforms• Deploy applications to any environment with network path, in

particular IBM Cloud Kubernetes, Cloud Foundry, Virtual Machines or other cloud providers.

Custom jobs with your own docker images• Use curated version of build and dev tools in pipelines; or bring your

own images with your tools for building, testing and deploying.

Bring your own pipeline worker• Register your own pipeline worker to poll and execute workloads on-

prem, on iOS machines.

Pipeline as code• Define your pipeline as code with Tekton open standard, and

manage it as code in your repo.

IBM Cloud DevOps / June 2018 / © 2018 IBM Corporation

Eclipse Orion Web IDE15

• Edit your code from anywhere

• Persisted user workspace

• Cloud-hosted, Web-based IDE

• Built-in Git client

• Create, edit, run, debug

• Powerful syntax highlighting, code assist, and refactoring

• ”Live Edit” – push hot changes directly to the Cloud

• For Cloud Foundry apps

• Node.js, HTML, CSS

DevOps Insights

16IBM Cloud DevOps / June 2018 / © 2018 IBM Corporation

Speed: Aggregate data from a variety of testing and code scanning tools to provide a comprehensive dashboard view of the risk profile of projects.

Quality: Ensure quality through automated enforcement of policies and gates based on test metrics.

Control: Track deployment risk and measure results over time as teams react to the trends in their DevOps practices

Deployment Risk Analysis17

Think 2019 / 4593A / Feb 14, 2019 / © 2019 IBM Corporation

Understand build status, security scan results, code coverage, and test coverage to evaluate whether to promote your app to the next environment.

Ensure quality through automated enforcement of policies and gates based on quality metrics

Identify your project’s development risk.

18Page

Develop a Kubernetes app with Helm toolchain

– Create a DevOps toolchain to code, build, and deliver a node.js app

– Build app in a Docker image

– Validate image using Vulnerability Advisor

– Deploy app to a Kubernetes Cluster using Helm charts

– Locally develop and test with Minikube

– Add a tool integration for notifications (Slack)

– Modify app and redeploy

– Deploy app to a staging serverhttps://www.ibm.com/cloud/garage/tutorials/use-

develop-kubernetes-app-with-helm-toolchain

IBM Cloud DevOps / June 2018 / © 2018 IBM Corporation

Develop and test microservices in IBM Cloudusing Cloud DevOps, Kubernetes and Helm

• Teams building an online store application, formed with 3 microservices kept in separate Git repos

• Using Kubernetes with Continuous Delivery to build, test and deploy the application

• Using Helm release management within Delivery Pipeline, separation of duties between squad developing and SRE deploying

• Speed with control, using Insights quality gates and traceability across toolchains

• Selenium testing with SauceLabs, notifications through Slack and alerting through Pagerdutyhttps://www.ibm.com/cloud/garage/tutorials/use-

develop-test-microservices-with-kubernetes-and-helm-toolchain

https://cloudcontent.mybluemix.net/cloud/garage/tutorials/use-canary-testing-in-kubernetes-using-istio-toolchain

Canary test in IBM Cloudusing Cloud DevOps, Kubernetes and Istio

– Create a DevOps toolchain to code, build, and deliver a node.js app

– Build app in a Docker image

– Validate container using Vulnerability Advisor

– Deploy app to a Kubernetes Cluster

– Develop in a canary branch, with CI/CD pipeline; automatic inference of canary deployment manifests

– Canary test using Istio service mesh for controlled roll out

– Merge back to master branch, finalize canary

Pipeline CANARY- “canary” branch- unit test- build Docker image- vulnerability advisor- deploy dark- progressive rollout

Pipeline STABLE- “master” branch- unit test- build Docker image- vulnerability advisor- functional test- Kube deployment- route 100% traffic

Image Registry(private)

Microservice repoincl. Docker file, deploy manifest

project/app:1.1

project/app:1.0

project/app:canary-0.9

Eclipse WebIDE- develop in the cloud- live edit, refactor- debug

Develop locally

Prod Cluster

Pod-3Pod-2

Pod-1

Pod-2-canary

master branch

canary branch

https://github.com/open-toolchain/kube-razee-toolchain

Deploy at scale in IBM Cloudusing Cloud DevOps, Kubernetes and Razee

– After installing Razee in target cluster

– Create a DevOps toolchain to code, build, and deliver a node.js

– Build app in a Docker image

– Validate container using Vulnerability Advisor

– Deploy to first cluster using Razee pull-modelagent

– Add Razee in more clusters, and see next change deployed in multiple clusters at once.

– Rollback deployments by reverting Razee deployment information once

Pipeline STABLE- “master” branch- unit test- build Docker image- vulnerability advisor- update Razee

deploy info

Image Registry(private)

Microservice repoincl. Docker file, deploy manifest

project/app:1.1

project/app:1.0

Eclipse WebIDE- develop in the cloud- live edit, refactor- debug

Develop locally

Cluster #2

Pod-3Pod-2

Pod-1

master branch

Deployment repoincl. Razee

deployment manifest

Cluster #3

Pod-3Pod-2

Pod-1

Cluster #1

Pod-3Pod-2

Pod-1

Razee

pulling

Razee

observing

Case Study – European Transportation System

IBM Cloud / © 2019 IBM Corporation

• Project to build customer-facing station information displays.

• Developed in conjunction with BP

• Developed on IBM Cloud Public (Frankfurt, EU-managed)

• Using a coordinated microservices CI/CD process, following template: https://www.ibm.com/cloud/garage/tutorials/use-develop-test-microservices-with-kubernetes-and-helm-toolchain

• Utilizing IBM Cloud Continuous Delivery: Delivery Pipeline for deployment, and Gitlab Repos for repository (migration from GitHub Enterprise on Cloud Dedicated)

• Test deployments began in 2018, with full station rollouts beginning in Q1 2019

23

Hybrid Cloud DevOps : Dev & Test in public

Run anywhere

IBM Cloud / © 2019 IBM Corporation

Develop on Cloud Public, Run anywherewith IBM Cloud Continuous Delivery

• Dev & test environments benefit from public cloud elasticity regardless of where production is deployed

• Don’t assume development can’t use public cloud tools just because production is on private

• IBM Cloud Continuous Delivery toolchains are able to reach any compute targets in IBM Cloud or outside

• Deploy across regions

• Target any public target directly or use private pipeline workers to reach private or local targets

Public

Private

OpenShift,Other cloud

Virtual servers

IBM Cloud Foundry

IBM Cloud Foundry Enterprise Environment

Private pipeline workers

• Enables pipeline work to run on• Particular platforms, e.g. MacOS (for iOS builds), Windows, ICP

• Particular networks, i.e. behind a firewall

• Bigger workers, for faster execution

• More workers, for reduced queuing

• Without execution time limits (jobs on shared public workers limited to 60 minutes)

• Private worker polling on CD public control plane (no inbound traffic into worker)

• Built on Tekton open-standard

Hybrid Continuous Delivery with Tekton Pipeline Workers

Managed Pipeline Worker

Continuous DeliveryPipeline

Managed Pipeline Worker

Private Pipeline Worker

Pipeline executions,

logs

CD-managed pipeline workersshared multitenant

Private/localpipeline worker

Kubernetes cluster

Cloud Foundry

Virtual servers

Cloud Functions

Multi Cloud Manager

Target environments

Source code with pipeline

yaml

Continuous DeliveryGit Repos and Issue Tracking

Other Cloud provider

FIREWALL

Polling for work

Private Git Repository

Other private Tools

Source code with pipeline

yaml

Private pipeline worker isolated in public Cloud

Other Git provider

Polling for work

Kubernetes cluster

Virtual servers

…Private Pipeline

Worker

Target environments

IBM Cloud

Other public

Private

• “Open-standard” backed Pipeline-as-code

• Multi-vendor collaboration with foundation backed governance

• Uses Kubernetes CRDs and leverages existing tooling (helm, kustomize, ksonnet)

• Future of CI/CD Runtimes

• Evolving CI/CD pattern support in lock-step with latest Kubernetes patterns

• Already adopted for products by IBM DevOps, Kabanero, OpenShift, Jenkins X, Puppet Nebula, with internal POCs at Google, eBay, Alibaba, …

• Tekton Community!

• Catalog of best practices for authoring pipelines and tasks

• Sub-projects that extend Tekton for important use-cases and support experimentation(Dashboard UI, CLI, Triggers, Webhooks Extension, Operator)

• Committers and contributors willing to lend a hand.(Plumbing, Knative Build, Kubeflow)

https://tekton.dev/

Continuous Delivery Foundation

• CDF believes in the power of Continuous Delivery to empower developers and teams and to produce high quality software more rapidly

• CDF believes in the open-source solutions collectively addressing the whole SDLC

• CDF fosters and sustains the ecosystem of open-source, vendor neutral projects through collaborations and interoperability

https://cd.foundation/

Integrating on-prem and cloud-native Insights

IBM Cloud / © 2019 IBM Corporation

1. grunt-idra3: Downloadable npm

package:

• Use this utility to integrate

Insights with:

• Travis CI

• Concourse

• IBM Urban Code Deploy

• IBM Continuous Delivery

Pipelines

• …other CI/CD tools.

• Avail also with ibmcloud CLI

ibmcloud doi(ibmcloud plugin install doi)

2. If using Jenkins, download IBM

Cloud DevOps plugin from

Jenkins plugin site

Continuous Delivery

https://www.npmjs.com/package/grunt-idra3

https://plugins.jenkins.io/ibm-cloud-devops

IBM Cloud / © 2019 IBM Corporation

BNP Paribas […] will now integrate the IBM

Cloud hosted in data centers dedicated to

the bank. BNP Paribas will also strengthen

its Hybrid Cloud “As a Service” capabilities

using IBM solutions offered via its public

Cloud to support the development of new

services, including test and applications

environments.

In line with its Cloud strategy and in order to

ensure the security of its customers' data,

BNP Paribas will not use Public Cloud for

either customer data or production

environments with sensitive information.https://group.bnpparibas/en/press-release/bnp-paribas-signs-

agreement-ibm-services-deploy-cloud-strategy

Case Study – BNP Paribas

• Data protection guiding principles, e.g. unless sensitive code,

use IBM Cloud git hosting, else use on-prem git

• Utilizing IBM Cloud Continuous Delivery, DevOps Insights for

dev & test in public, orchestrate deployments either in public or

private cloud

• Leveraging Private Pipeline Workers to reach more sensitive

targets or tools

31

Case Study – IBM Watson HealthHealthcare @ Cloud Compliance• HIPAA / HITECH (Security / Privacy / Breach Reporting)

• ISO 9001:2015 (Quality Management System)

• ISO 27K (Information Security Controls)

• SOC2 (Data Security, Availability, Processing Integrity,

Confidentiality, and Privacy)

• Watson Foundation for Health Alpha went live on July 1st 2019

• Implementing Healthcare development compliance (HIPAA) in IBM

Cloud Public using Cloud DevOps with a coordinated microservices

• Migrated away from managing own tools (e.g. Jenkins)

” The Continuous Delivery and DevOps Insights

services have proven to be the right investment for us

as we pivot into more work (scale) and Hybrid Cloud ! ”

32

Case Study – DevSecOps• Security vulnerabilities in software are “eating

business results” in the form of damaged brand

reputation, client trust, and financial fines.

• Ongoing requirements to increase compliance

certification: SOC2 type2, FedRamp, …

• What if clients could fix these known vulnerabilities

before they make it into production applications ?

• Note: Automated prod scans still needed, as existing apps may

rot, and ultimately needed for audit materials.

• Numerous issues could be detected earlier in dev

cycle by augmenting CI/CD pipelines and allowing

to shift compliance to the left of the dev cycle;

avoiding promoting bad code and even better

avoiding accepting bad pull requests into team

source control.

Challenge: Keep the “assembly line” running always,

as an emergency fix might always be needed to

remediate a critical situation; hence need for pull

request validation.

33

Developer & DevOps Tools

ModernizationToolkit

Frameworks and Runtimes

• Move and modernize apps - using insights about your current infrastructure to appropriately refactor, optimize resources and costs, and reduce complexity

• Develop cloud native apps with containers, starting with open source, common services, developer tools of choice, and integrated DevOps

• Optimized set of frameworks and runtimes for cloud native and traditional

• Accelerate development with governance, supported by IBM enterprise expertise

• Investment protection as you modernize at your pace

Use cases Competitive differentiationKey capabilities Client examples

IBM Cloud / June 2019 / © 2019 IBM Corporation

Build, test, move, and deploy applications in a modern, microservice-based framework

IBM Cloud Pak for Applications

Pre-integratedSystems

Notices and disclaimerscontinued

34

IBM Cloud / © 2019 IBM Corporation

•Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products about this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products.Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the ability of any such third-party products to interoperate with IBM’s products. IBM expressly disclaims all warranties, expressed or implied, including but not limited to, the implied warranties of merchantability and fitness for a purpose.

•The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual property right.

•IBM, the IBM logo, ibm.com and [names of other referenced IBM products and services used in the presentation] are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at: www.ibm.com/legal/copytrade.shtml.