Infrastructure Design for Kubernetes
Fabio ChiodiniAdvisory Platform Architect - EMEA
@FabioChiodini
Credits to @cdavisafc and @pczarkowski for a lot of EPIC contents ;)
© Copyright 2019 Dell Inc.5
Kubernetes.. What?
© Copyright 2019 Dell Inc.6
Kubernetes 101 - Containers, Runtime and Orchestrator
Infrastructure
C1
C2
C3
Operating System
Container Runtime
Ku
be
let
Ku
be
-pro
xy
Physical Network
API serverSchedulerController manager
etcd
Node 1Master
Overlay Network (Flannel/Open vSwitch/Weave)
OS
Infra Infrastructure
C1
C2
C3
Operating System
Container Runtime
Ku
be
let
Ku
be
-pro
xy
Node 2
Infrastructure
C1
C2
C3
Operating System
Container Runtime
Ku
be
let
Ku
be
-pro
xy
Node 3
© Copyright 2019 Dell Inc.7
Kubernetes is a Container Orchestrator
> kubectl
Storage NetworkingCompute
Kubernetes Dashboard
Dev / Apps IT / Ops
App User
© Copyright 2019 Dell Inc.8
But we need a little bit more to RUN IT in production
Storage NetworkingCompute
Dev / Apps
App User
IT / Ops
> kubectl
Kubernetes Dashboard
Load Balancing / Routing
Container Image
Registry
App Monitoring
App Logging
OS Updates
OS Images
K8S Updates
K8S Images
Log & Monitor
Recover & Restart
Backup & Restore
ExternalData Services
Cluster Provisioning
Provision & Scale
Command Line / API
Management GUI
Monitoring GUI
© Copyright 2019 Dell Inc.9
What will WE cover today?
Storage NetworkingCompute
Dev / Apps
App User
IT / Ops
> kubectl
Kubernetes Dashboard
Load Balancing / Routing
Container ImageRegistry
App Monitoring
App Logging
OS Updates
OS Images
K8S Updates
K8S Images
Log & Monitor
Recover & Restart
Backup & Restore
ExternalData Services
Cluster Provisioning
Provision & Scale
Command Line / API
Management GUI
Monitoring GUI
StorageHost Networking
Install, Maintain and Upgrade
Cluster
Provisioning
Routing/Load Balancing/More
Cluster Scaling
© Copyright 2019 Dell Inc.10
#SpoilerAlert
© Copyright 2019 Dell Inc.11
With Pivotal Container Service we got You covered!!
Storage NetworkingCompute
Dev / Apps
App User
IT / Ops
> kubectl
Kubernetes Dashboard
Load Balancing / Routing
Container ImageRegistry
OS Updates
OS Images
K8S Updates
K8S Images
Log & Monitor
Recover & Restart
Backup & Restore
ExternalData Services
Cluster Provisioning
Provision & Scale
vRealize LogInsight
Service Broker
PKS Control Plane
> pks
Operations Manager
vRealize Operations*
*integration
© Copyright 2019 Dell Inc.12
Upgrades
© Copyright 2019 Dell Inc.13
Here’s (in 5 sec) why You need automatic upgrades
Kubernetes 1.9 Kubernetes 1.10 Kubernetes 1.11 Kubernetes 1.12
December 2017 March 2018 June 2018 September 2018
Kubernetes 1.13
December 2018
Kubernetes 1.14
March 2019
CVE-2018-1002105
Kubernetes API
CVE-2019-6486
Go
December 2018 February 2019
CVE-2019-5736
runC
February 2019
CVE-2019-1002100
API Server DOS
March 2019
Kubernetes
Releases
Kubernetes
CVEs
20 Linux CVE in 2019
(454 in 2017)
Operating System
© Copyright 2019 Dell Inc.14
Let’s do some math…. In a Year:
4 major K8s updates x (30 clusters x 7 VMs) = 840 upgrades
30 clusters x 7 VMs = 210 OS upgrades
how many linux release this year?
Operating System
And what about CVEs?
© Copyright 2019 Dell Inc.15
DEMO:Upgrading K8s (and OS) in a click
© Copyright 2019 Dell Inc.16
© Copyright 2019 Dell Inc.17
Host types
© Copyright 2019 Dell Inc.18
Yes you need VMs (most of the times)• Have you automated your
baremetal deployments today?
• How much time does it take to provision a cluster?
• How do you patch (OS/K8s Firmware etc)?
• How do you remediate Failures?
• Can you repair/reinstall a node in a few minutes?
• [How many people do you need to manage it?]
• Scale unit: scale a VM vs scaling a server
• Time required to scale up a cluster
© Copyright 2019 Dell Inc.19
DEMO:Self Healing Kubernetes
© Copyright 2019 Dell Inc.20
DEMO:Scaling Kubernetes
© Copyright 2019 Dell Inc.21
I’ll give you some homeworks for these demos ;)
https://www.youtube.com/playlist?list=PLAdzTan_eSPTJ8yIslkADNxtbxzwF1PAu
© Copyright 2019 Dell Inc.22
Storage
© Copyright 2019 Dell Inc.23
Storage for Kubernetes.. How?
Pod
• Is [effectively] a Directory, possibly with data in it,
available to all containers in a Pod
• Usually Shares lifecycle of a Pod (Created when Pod
is created, destroyed when Pod is destroyed)
• Persistent Volumes outlive Pods
• Can be mounted from local disk, or from a network
storage device such as a EBS volume, iscsi, NFS, etc
Infrastructure
C1
C2
C3
Operating System
Container Runtime
© Copyright 2019 Dell Inc.24
https://github.com/container-storage-interface
Container Storage Interface (CSI)
© Copyright 2019 Dell Inc.25
DEMO:Adding Storage to Kubernetes
© Copyright 2019 Dell Inc.26
© Copyright 2019 Dell Inc.27
Wrapping Up
© Copyright 2019 Dell Inc.28
What You should plan to add to Kubernetes
Advanced
Networking Flexible Host types
Self Healing Hosts Automated
Upgrades
AutomationStorage Integration
Automated
Cluster
Provisioning
PKS Enterprise value-added features
Built into Kubernetes
Multi-container pods
Stateful Sets of pods
Persistent disks
Single tenant ingress
Pod scaling and high availability
Rolling upgrades to pods
Cluster provisioning and scaling
Embedded, hardened Operating System
Monitoring and recovery of cluster VMs and processes
Rolling upgrades to cluster infrastructure
Secure multi-tenant ingress
Secure container registry
© Copyright 2019 Dell Inc.29