Templatized Application Configuration

Ben Parees, Principal Engineer, OpenShift

Agenda● OpenShift Template Specification● Template Usage● Kubernetes Templates● Other Relevant Technologies

OpenShift Template Goals● Encapsulating artifact for application components

○ Services, PodTemplates, BuildConfigs, DeploymentConfigs, etc○ Common label for components from the same template

● Parameterizable○ Customize at time of use○ Share common values between components

● Fit with the Kubernetes API model○ Json syntax, api object specification, CRUD storage

What does OpenShift do with this?● Define Replication Controller + Service together● Parameterize replica counts, resource limits

○ memory and cpu limits with recommended defaults

● Share environment variables○ passwords, hostnames, configuration values

● Point to default source repository for a build○ allow user to override with their own

● Parameterize service names for repeated deployment○ referenced by multiple components

Specification - Metadata{

"kind": "Template",

"apiVersion": "v1",

"metadata": {

"name": "ruby-helloworld-sample",

"annotations": {

"description": "Creates a simple ruby app",

"iconClass": "icon-ruby",

"tags": "quickstart,ruby,mysql"

}

},

"labels": { "template": "ruby-sample-template", “appname”: “my_ruby_app” }

Specification - Body"objects": [

{

"kind": "Service",

"apiVersion": "v1",

"metadata": {

"name": "frontend",

},

"spec": {

…………………..

},

},

{ "kind": "DeploymentConfig", "apiVersion": "v1", "metadata": { "name": "frontend", "creationTimestamp": null }, "spec": { ……………… }}]

Specification - Parameters"parameters": [

{

"name": "MYSQL_USER",

"description": "administrator username",

"generate": "expression",

"from": "admin[A-Z0-9]{3}"

},

{

"name": "MYSQL_DATABASE",

"value": "app_db",

"required": true

}]

<podtemplate definition>"env": [ { "name": "MYSQL_USER", "value": "${MYSQL_USER}" }, { "name": "MYSQL_PASSWORD", "value": "${MYSQL_PASSWORD}" }, { "name": "MYSQL_DATABASE", "value": "${MYSQL_DATABASE}" }]</podtemplate definition>

Usage● API endpoints for storage

■ oc create -f mytemplate.json■ oc get templates■ oc delete template mytemplate

● And instantiation○ oc process mytemplate | oc create -f -○ oc new-app mytemplate○ Can override parameter values

● Instantiation returns substituted template○ Client iterates []Objects and creates each one

Kubernetes Templates● aka “Parameterized List”● Similar to OpenShift● Less focus on application topologies● More focus on internal usage

○ ReplicationController equivalent for component groups○ eg PetSets, DaemonSets

● No parameter generation○ Use secret generation (coming soon) instead

● More parameter type support

K8S Specification - Parameters"parameters": [{

"name": "MONGODB_USER",

"description": "Username for MongoDB

"value": "password"

"required": true

}, {

"name": "REPLICA_COUNT",

"description": "Number of replicas",

"value": "1",

"required": true,

“type”: “integer”

} ]

<ReplicationController definition>"spec": { "replicas": "$((REPLICA_COUNT))", "selector": { "name": "$(DATABASE_SERVICE_NAME)" } ………. <PodTemplate definition> ……... "env": [ { "name": "MONGODB_USER", "value": "$(MONGODB_USER)" }] </PodTemplateDefinition></ReplicationController definition>

Parameter Substitution Examplessomefield: "$(FOO)" -> somefield: "BAR"

somefield: "$((FOO))" -> somefield: BAR

somefield: "prefix_$(FOO)_suffix" -> somefield: "prefix_BAR_suffix"

somefield: "prefix_$((FOO))_suffix" -> somefield: prefix_BAR_suffix

somefield: "prefix_$((FOO))_$(FOO)_suffix" -> somefield: "prefix_BAR_BAR_suffix"

Templating Alternatives● Many external projects for defining deployments● Pros (depending which one you pick)

○ More sophisticated templating○ Pick your favorite templating language○ Support for includes/dependencies/references

● Cons○ More sophisticated templating○ Separate set of tooling required○ Less tightly coupled into the cluster experience

Helm by Deis● Templates are “Charts”● Provides a (git) repository of Charts● Chart definition consists of

○ chart metadata yaml○ 1..N k8s object definition yaml files

● Charts can depend on other charts● Does not offer parameterization

Helm Example.

├── Chart.yaml

├── manifests

│ ├── jenkins-rc.yaml

│ └── jenkins-svc.yaml

└── README.md

<Chart.yaml>name: jenkinshome: https://jenkins-ci.org/version: 0.2.0description: The leading open-source continuous integration server.maintainers: - Matt Fisher <mfisher@deis.com>details: Jenkins is the leading open-source continuous integration server.</Chart.yaml>

jenkins-svc.yamlapiVersion: v1

kind: ReplicationController

metadata:

name: jenkins

labels:

app: jenkins

heritage: helm

spec:

replicas: 1

template:

metadata:

labels:

app: jenkins

spec:

containers:

- name: jenkins

image: jenkins:1.642.1

ports:

- containerPort: 8080

Usage

Usage

Deployment Manager● Jinja or Python syntax● Very hierarchical

○ Define components in terms of other components

● Very powerful parameterization● Extremely flexible template syntax● Aspires to reconciliation/redeployment

Deployment Manager Exampleresources:

- name: frontend

type: github.com/kubernetes/application-dm-templates/common/replicatedservice:v1

properties:

service_port: 80

container_port: 80

external_service: true

replicas: 3

image: gcr.io/google_containers/example-guestbook-php-redis:v3

- name: redis

type: github.com/kubernetes/application-dm-templates/storage/redis:v1

properties: null

Redis Cluster Schemainfo: title: Redis cluster description: Defines a redis cluster, using a single replica replicatedservice for master and replicatedservice for workers.

properties: workers: type: int default: 2 description: Number of worker replicas.

Redis Cluster Template{% set REDIS_PORT = 6379 %}{% set WORKERS = properties['workers'] if properties and properties['workers'] else 2 %}

resources:………………….

- name: redis-slave type: github.com/kubernetes/application-dm-templates/common/replicatedservice:v1 properties: # This has to be overwritten since service names are hard coded in the code service_name: redis-slave service_port: {{ REDIS_PORT }} container_port: {{ REDIS_PORT }} replicas: {{ WORKERS }} container_name: worker image: kubernetes/redis-slave:v2………………….

Replicated Service Schemainfo:

title: Replicated Service

description: Defines a ReplicatedService type by creating both a Service and an RC.

required:

- image

properties:

container_name:

type: string

description: Name to use for container. If omitted, name is used.

service_name:

type: string

description: Name to use for service. If omitted, name-service is used.

………………..

Replicated Service Templateconfig = {'resources': []}

container_name = context.properties.get('container_name', name)

namespace = context.properties.get('namespace', 'default')

service = {

'name': service_name,

'type': service_type,

'properties': {

'apiVersion': 'v1',

'kind': 'Service',

'metadata': {

'name': service_name,

'namespace': namespace,

'labels': GenerateLabels(context, service_name),

}

Conclusion● Don’t define raw resources, templatize them● The great thing about standards…..

○ More coming (Noel, RedSpread, KDeploy, ….. )● Join the Kubernetes SIG-Config to keep on top of them

References● OpenShift Templates

○ https://docs.openshift.org/latest/dev_guide/templates.html● Kubernetes Templates (Proposed)

○ https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/templates.md

● Kubernetes Secret Generation○ https://github.com/kubernetes/kubernetes/issues/12732

● Helm○ https://helm.sh/

● Deployment Manager○ https://github.com/kubernetes/deployment-manager

Thank You