load-balancing and scaling in kubernetes

Load-balancing and Scaling in Kubernetes

Tyler Rowland

Computer Science Senior SeminarDivision of Science and Mathematics

University of Minnesota, MorrisMorris, Minnesota, USA

28 October 2021

Rowland (U of Minn, Morris) Load-balancing and Scaling in Kubernetes 28 October 2021 1 / 27

Introduction

Introduction

What is Kubernetes?Automation of deployment andmanagementCreated by Google in 2015Software companies use own versionnowScaling and load balancing increaseperformance of application


Introduction Outline

Outline

Background ConceptsKubernetes HierarchyLoad-balance vs ScalingCentral Processing Unit (CPU)

MethodsLoad-balancingScaling

Conclusion


Background Concepts The Kubernetes Hierarchy

Client Server Architecture

UserInternetNodes



The Kubernetes Hierarchy: Containers

Containers:Run faster and moreconsistent betweenenvironments

Application runs inside ofthe containers

API server

Kubelet

Master Node

Worker node Worker node

Cluster

Container

Pods

Nodes

APP AAPP A APP B APP B

Kube-ProxyKube-Proxy Kubelet



The Kubernetes Hierarchy: Pods

Containers run inside ofpods

1 or more container per pod

Pods: smallest unit ofexecution

API server

Kubelet

Master Node


Cluster

Container

Pods

Nodes





The Kubernetes Hierarchy: Nodes

Two types of nodes inKubernetes:

Worker NodeMaster node

API server

Kubelet

Master Node


Cluster

Container

Pods

Nodes





The Kubernetes Hierarchy: Cluster

A cluster is made up of:containers, pods, andnodesUnique cluster IP

API server

Kubelet

Master Node


Cluster

Container

Pods

Nodes




Background Concepts API server

API server

Application ProgrammingInterface (API)

Communicates with Kubeletand Kube-Proxy

Changing the number ofnodes

API server

Kubelet

Master Node


Cluster

Container

Pods

Nodes




Background Concepts Scaling

Scaling Background

Kubernetes providesHorizontal scaling

Changing number ofpods or nodesDistribute workload

Ex. CPU usage of a podhits 80 percent of capacity


Background Concepts Load-Balancing

Load-Balancing Background

Distributes traffic toreplicated applications

Bottleneck?

Traffic

Server

Master Node

Pod Pod Pod


Background Concepts CPU

Central Processing Unit (CPU)

CPUs are measured in cores

Measures the compute processing in a machine

Most cases CPU usage in Kubernetes is measured in millicores

Millicore is one thousandth of a core

Ex. 2000 millicores (or 2000m) is equal to 2 cores

This is standard for every unit in Kubernetes


Methods Scaling

Server Scalability Using Kubernetes

L. P. Dewi, A. Noertjahyana, H. N. Palit and K. Yedutun, "ServerScalability Using Kubernetes,"

Study uses 3 servicesGet- get dataPost- send dataDelete- delete data

Evaluates CPU usage single vs. multiple server (Using scalability)

3 Kubernetes nodes with multiple servers1 master node2 worker nodes


Methods Scaling

Kubernetes Horizontal Pod Autoscaling

What is KHPA?

Increase or decrease based on the number of concurrent users.

KHPA takes the target input as a percentage of CPU usage

The output is the pods target number


Methods Scaling

CPU Usage

Simulated behavior using acombination of services

1,650 requests in each task


Methods Scaling

CPU Usage

Users send 8000 requests of each service

Evaluates CPU usage (millicores) of single vs. multiple servers

Averages of previous scenario:


Methods Scaling

Scaling Results

Single ServerIn both scenarios the CPU usage was much higherWhen the number of requests went up the CPU usage increased

Multiple Servers (Scaling)Servers distribute workloadCPU usage is smaller when there are more requests


Methods Load-balancing

Raft Algorithm

Leader selection algorithm forKubernetes

Used to select pod leaders in anode

Majority of traffic goes toleadersCandidates try to become theleaderFollowers are replicaapplications that don’t get themajority of traffic



Toward Highly Scalable Load-Balancing in KubernetesCluster

N. Nguyen and T. Kim, "Toward Highly Scalable Load Balancing inKubernetes Clusters," in IEEE Communications Magazine, vol. 58, no.7, pp. 78-83, July 2020, doi: 10.1109/MCOM.001.1900660.

Concentrated vs. Balanced Leaders5 leaders distributed to 3 nodesBalanced leaders uses the leader electionMajority of traffic is sent to the leaders



Leaders

Concentrated vs. BalancedLeaders

Concentrated: 5leaders in node 1Balanced: 5 leadersdistributed throughout 3nodes

Node 1 Node 2 Node 3

Balanced Leaders Cluster

Node 1 Node 2 Node 3

Concentrated Leaders Cluster



CPU Utilization

4 clients send requests for 150s

The requests are handled by theleader pods



Throughput

Clients increasing from 1 to32

Clients send 2000 requestseach

Throughput- Number ofrequest per second thecluster can recieve



Load-Balancing Results

Concentrated leadersCPU utilization 60 percent higher in node 1 versus the other twonodesOnce 4 clients send requests throughput is constant

Balanced leadersMore balanced CPU utilizationMore requests per second


Conclusion

Conclusion

Scaling and load-balancing increase performanceScaling focused on distributing to different nodesLoad-balancing focused on distributing network traffic


Acknowledgements

Acknowledgements

I would like to thank Nic McPhee and Elena Machkasova for greatfeedback and guidance throughout this process. Also thanks to theCSCI department for great planning of this event.


Questions

Questions

Questions?


Resources

L. P. Dewi, A. Noertjahyana, H. N. Palit and K. Yedutun, "ServerScalability Using Kubernetes," 2019 4th Technology InnovationManagement and Engineering Science International Conference(TIMES-iCON), 2019, pp. 1-4, doi:10.1109/TIMES-iCON47539.2019.9024501.

N. Nguyen and T. Kim, "Toward Highly Scalable Load Balancing inKubernetes Clusters," in IEEE Communications Magazine, vol. 58, no.7, pp. 78-83, July 2020, doi: 10.1109/MCOM.001.1900660.


load-balancing and scaling in kubernetes

Documents