gcp - continuous integration and delivery into kubernetes with github, travis and artifactory

Continuous Integration and Delivery into Kubernetes with GitHub, TravisCI, and Artifactory.Google Cloud Platform

Continuous Integration and Delivery into Kubernetes with GitHub, TravisCI, and ArtifactoryGoogle Cloud Platform

Oleg ShalyginSADA Systems

Continuous Integration (CI) is a development practice that requires developers to integrate code into a shared repository several times a day. Each check-in is then

verified by an automated build, allowing teams to detect problems early.

By adopting both Continuous Integration and Continuous Deployment, you not only reduce risks and catch bugs quickly, but also move rapidly to working software.

https://www.thoughtworks.com/continuous-integration

GCP Free Tierhttps://cloud.google.com/free/

○ 12 Months : $300○ GAE○ DataStore○ Compute Engine○ Container Engine○ Cloud Storage○ Pub/Sub○ Cloud Functions(beta!)○ Stackdriver○ BigQuery○ Vision/Speech/Natural Lang

API

https://cloud.google.com/sdk/

Registering with GCP

https://console.cloud.google.com

○ Login with your Google account

○ Agree to the Terms

○ Add a Credit Card

○ Start the Trial!

Google Cloud SDK

● Download the Google Cloud SDKhttps://cloud.google.com/sdk/

○ Python 2.7 required○ Download the tarball ○ Extract the package○ $ ./google-cloud-sdk/install.sh

○ $ gcloud init

https://cloud.google.com/sdk/

Proprietary + Confidential

GCP Essentials if you when not using gcloud init

$ gcloud auth login

# List what the current settings are$ gcloud projects list

$ gcloud config list

# Set the project and compute zone$ gcloud config set project <YOUR_PROJECTNAME>

$ gcloud config set compute/zone <ZONE>

Cloud LoadBalancing

WebClients

Android

iOS

1Relational DatabaseCloud SQL

Async MessagingCloud Pub/Sub

AnalyticsBigQuery

Image BucketCloud Storage

Source ControlGitHub

Application Infrastructure

MonitoringStackdriver

Docker image storageContainer Registry

Container Engine50GB PD

1 10.61

NoSQL DatabaseDataStore

Real Time MessagingFirebase

Google Container Engine

Node● Worker Machine that runs the

containers● Managed by master

components● Included services:

○ Container runtime○ kubelet○ kube-proxy


● API centric system.○ No private APIs○ No system only calls○ EVERYTHING

communicates throughthe API.

● REST API in terms ofresources/verbs.

API Server


● Fundamental to how Kubernetes works● Piece of code that watches the

Kubernetes API and reacts accordingly. ○ DNS, Kubelet, Services, ReplicaSet

Controllers


● Small group of containers and volumes

● Tight coupling - live and die together.

● Shared networking and lifecycle

○ Containers within a pod share an IP address, they find each other via localhost

Pods


● Declarative updates for pods.● Specify a desired state =>

Deployment controller handles the rest.

● Self healing

Deployments


● Abstraction that defines a logical set of Pods and the policy by which they are accessed.

● Recall that every pod has an IP.

● Services provide a stable VIP● The pods behind the service

can scale up/down, be removed, etc, their IP is constantly changing but the service VIP does not change.

Services


● Holds sensitive information such as passwords, OAuth tokens, encrypted files, etc.

● Numerous automatic secrets are created by Kubernetes to store credentials to access the API.

● Secrets are base64 encoded(not encrypted)● ConfigMaps are used to store non-sensitive

information.● imagePullSecrets are used to set container

registry credentials.

Secrets / ConfigMaps


● CLI that allows you to run commands against the cluster.○ List pods○ List nodes○ Apply changes○ “matrix” into a container on a running

pod:$ kubectl exec -it my-pod -- bash

● Enabled via:$ gcloud components install kubectl

kubectl

Google Cloud Platform

● Simplify and automate cloud deployment via CLI.○ Repeatable○ Codified○ Declarative

● Templates can be pulled once created manually for resource re-creation

GCP Deployment Manager

Google Cloud Platform

● Create a service account that is used to manage the cluster and orchestrate all of the commands that we need.

● Used in the orchestration process.

● Located under IAM(Identity Account Management)

● Encrypt and store as part of the CI process

GCP Service Accounts

Travis CI

● Download and install the latest GCloud SDK○ https://cloud.google.com/

sdk/downloads● Set the Zone, Cluster, and Project● Activate the Service Account● Install the kubectl CLI

Scripting out GCP Authentication

Build Orchestration● Application code is decoupled from the

infrastructure code.

● Orchestrating deployments on various environments or specific containers is completely disassociated from the application code.

● Process Flow:

○ Application code is built, compiled or packaged.

○ Code is pushed to the binary repository which houses binaries of all sorts, such as: npm modules, Java Archive (.jar), or Docker images.

○ Upon deployment, the binary is pulled from Artifactory and consumed as part of another build orchestration step or deployed to the appropriate resource.

● Modularity and flexibility!

Source Code built (.jar, .js, .dll,

images)

Infrastructure Changes

Orchestrated

Deployed

Artifactory on GCP

● During the build, create a docker image.● The image is pushed to Artifactory / GCR

○ There is native support within Kubernetes on GKE with GCR.

○ In the case of a private repository such as Artifactory, we need to create a imagePullSecrets link via a Secrets config type: kubernetes.io/dockerconfigjson

● Artifactory provides a free plan for open source projects which allows users to ramp up and experience the benefits of a binary repository:https://www.jfrog.com/artifactory/free-trial/

Pushing Images from the build

https://www.jfrog.com/artifactory/free-trial/

Updating the Deployment

● If the build is successful, update the dockerimage tag.

● Recall that the deployment specifies thedesired state of the pod.

Update container image through pipeline

# ...spec: replicas: 2 strategy: rollingUpdate: maxUnavailable: 0 type: RollingUpdate template: metadata: labels: app: md-web spec: containers: - name: md-web image: ‘us-gcr.io/project-name/web-applciation:1.9.1’ env: - name: MONGO_IP valueFrom: secretKeyRef: name: env-variables-secret key: mongo_ip - name: MONGO_PORT valueFrom: secretKeyRef: name: env-variables-secret key: mongo_port imagePullPolicy: Always ports:# ...

Jack Weaver

I'm wondering how this will show up on a projector.Should probably be OK.The alternative is black text, white background

Apply Changes

● Depending on how many deployments describe the new state of the application, the following command will apply the changes to the cluster:

$ kubectl apply -f ./

Applying changes to the cluster

Artifactory - Universal Artifact Repository● Software package binary repository● Supports software packages created in any

language or package.● CI/CD integration with nearly every CI

provider● Hook Properties for simple querying● Atom Uses:

○ Docker○ NPM○ Maven○ Gradle○ Debian○ Generic Repository

SaaS basedAsset and ResourceManagement Application

Cloud LoadBalancing

WebClients

Android

iOS

Container Engine- GB PD

1

Redis

Compute Engine- GB PD

1


1 1

MongoDB


2 1


1 1

Relational DB #3Cloud SQL



Async MessagingCloud Pub/Sub

User Account ManagementStormpath

Client AD

Reporting & AuditCloud Bigtable

AnalyticsBigQuery

Image BucketCloud Storage

Audit BucketCloud Storage

Binary RepositoryArtifactory

Source ControlGitHub

Atom Infrastructure

MonitoringStackdriver

Application Source Code (FrontEnd) - master branchGitHub

Web Pod - DEV branchGitHub

1. Build checks out the DEV branch in the associated pod repository and updates a deployment container image.2. The changes to deployment.yaml are committed to the pod repository in the DEV branch.

Successful build generates a new image and pushes it to Artifactory.

1. Pull request created.2. Pull request is approved.3. Code is merged into master with a new application version.


Web Pod Build - DEV branchTravis CI

1. GCloud SDK, GCloud Authentication and build scripts are pulled from Artifactory.2. Successful build deploys the resource configurations to the Kubernetes cluster.

The new commit triggers a build on the master branch in Travis CI

Web Project Build - DEV branchTravis CI

The new commit triggers a build on the DEV branch in Travis CI.

1. The images specified in the deployment are pulled from Artifactory.2. A rolling update updates the underlying resources.

Continuous Integration - DEV Pipeline

Backend Pod - DEV branchGitHub

Backend Pod Build - QA branchTravis CI

1. A manual pull request is opened against the QA branch.2. The pull request updates a single/multiple image tags in the deployment.yaml to the versions in DEV.3. The pull request is reviewed and approved.4. Code is merged into the QA branch.


1. GCloud SDK, GCloud Authentication and build scripts are pulled from Artifactory2. Successful build deploys the resource configurations to the Kubernetes cluster.

Backend Pod - QA branchGitHub

The new commit triggers a build on the QA branch in Travis CI

1. The images specified in the deployment are pulled from Artifactory2. A rolling update updates the underlying resources

Continuous Integration - Single/Multiple Container Promotion

0.0.170

DEV

fe-web: 1.3.3

Environment to Environment Releases

be-users: 2.8.1

be-trx: 1.6.0

be-alerts: 1.2.9

0.0.45

UAT

fe-web: 1.2.2

be-users: 1.0.0

be-trx: 1.2.0

be-alerts: 1.2.9

0.0.3

PRD

fe-web: 1.1.0

be-users: 1.0.0

be-trx: 1.1.0

be-alerts: 1.0.0

This repository has a JSON schema representation of every pod and all of the

underlying containers per pod. The values to each pod represent the current version in that

environment.

Environments Repository - UAT branchGitHub

Continuous Integration - Environment to Environment Promotion (QA to UAT)

Web Pod - UAT branchGitHub

1. Build individually checks out the UAT branch in every pod defined in the environments.json configuration.2. Every image in the pod repository UAT branch is updated to reflect the version in environments.json of the environments repository for that pod.3. The changes to each pods deployment.yaml are committed in the UAT branch of the pod repository.

1. Pull request created in GitHub from QA to UAT. 2. Pull request is approved.3. Code is merged into UAT with the versions from QA.


The new commit triggers a build on the UAT branch in Travis CI

Environments Repository Build - UAT branchTravis CI

1. The images specified in the deployment are pulled from Artifactory.2. A rolling update updates the underlying resources.

Backend Pod (Inventory Service) - UAT branchGitHub

Backend Pod (User Service) - UAT branchGitHub

The new commits trigger builds on every pod UAT branch in Travis CI.

Backend Pod (Inventory Service) Build - UAT branchGitHub

Backend Pod (Inventory Service) Build - UAT branchGitHub

1. GCloud SDK, GCloud Authentication and build scripts are pulled from Artifactory.2. Successful build deploys the resource configurations to the Kubernetes cluster.

Web Pod Build - UAT branchTravis CI

Future Development

● Meta-tagged software releases

● Helm

● Reduction of build steps

● Cached directories in Travis

● Optimized Docker image sizes

● Templated Kubernetes deployments

Questions?

gcp - continuous integration and delivery into kubernetes with github, travis and artifactory

Technology