!"

#

$

%

&'

$./run

{

Created by Fabrício Epaminondas ©2015-2020

Designing SCALABLE APPLICATIONS

Short Talk

WHAT IS SCALABILITY?

The concept

“Scalability is the capability of a system, network, or process to handle a growing amount of work, or its potential to be enlarged in order to accommodate that growth.”

Bondi, André B. (2000). Characteristics of scalability and their impact on performance.

3

Let's go deeper…

4

5Scalability for sandwiches

How hungry are you?

A bigger sandwich may fit your need (vertical scaling)

6

7

How many people are hungry?

Think about it…

8

Fast-food restaurant CHAINS

You need to scale the entire solution (horizontal scaling)

Scaling Computing Resources 9

• Processing (CPU)• Memory• Storage• Network

Scale-up vs Scale-out

Vertical Scaling (Scale-up) 10

Generally refers to adding more processors and RAM, buying a more expensive and robust server. • Pros

•Generally less challenging to implement •Less power consumption than running multiple servers •Less licensing costs • Less network hardware than scaling horizontally.

• Cons• Hardware costs •Greater risk of hardware failure causing bigger outages

•Generally severe vendor lock-in and limited upgradeability in the future

Horizontal Scaling (Scale-out): 11

• Generally refers to adding more servers with less processors and RAM. This is usually cheaper overall and can possibly scale infinitely.

• Pros•Much cheaper than scaling vertically •Easier to run fault-tolerance •Easy to upgrade

• Cons•More licensing fees •Bigger footprint in the Data Center •Higher utility cost (Electricity and cooling) •Possible need for more networking equipment

(switches/routers)

Related concepts 12

• Performance•Average Response Time (ART) •Latency (usually network and disk I/O) •Resource consumption

• High Availability•Single Point of Failure (SPOF)•Fault tolerance

USE CASE Weather Forecast

Designing the software

14

• Mobile App to show world-wide weather forecasts

• You planned to reach 1k users at first week.

• Important notes:•How many users will access

the system at same time? •How to model workload? •How to predict the future?

Weather Forecast App

15

• Use performance tests based on scenarios.•Test types:

• Performance•Validates the ART on a scenario (e.g.: systems

response takes 3secs with 100k users)• Load

• Aims to understand required resources for running the system (planning infrastructure).

• Stress•Look for system failures under stress conditions (DoS,

outages, etc.)

Notes

16Solution #1: Two Tier Architecture

(O )Client Server

Internet

4Gb Mem 1 vCPU*

* This virtual CPU unit represents the processing power of a CPU with 2 cores

17Solution #1: Two Tier Architecture

• Pros• Easy setup

• Cons•Single point of failure (1 SPOF) • Vertical scaling costs (buy better hardware, CPU, etc.)

18

• Your app is growing fast and now it reached 10k users after one month.

Weather Forecast App

19Solution #2: Three Tier Architecture

O )Client

Web Server

Data Server

(

4Gb Mem 1 vCPU

4Gb Mem 1 vCPU

Network Latency

Internet

Local/Private Network

20

• Pros• Doubled the computing power • Dedicated and optimized infrastructure (disk, memory, etc.)

• IT operations specialists (database, server management).

• Cons• More points of failure on server side (2 SPOFs) •Vertical scaling costs • More network latency

Solution #2: Three Tier Architecture

21

• Forecast app is being used in most important country cities and it has at about 500k users after two months.

•Some users are noticing failures and outages while trying to get the latest weather information.

Weather Forecast App

22Solution #3: Clustering

O )Client

Web Cluster

Data Server

(

4Gb Mem 1 vCPU

4Gb Mem 1 vCPU

Internet

Local/Private Network

(Load

Balancer

+

23

• Pros•High availability and fault tolerance •Grid computing •Out-of-box setup scaling for stateless applications

• Cons•IT management more complex, e.g.: Deployment and

rollback procedures. •Need to use a data sharing service for stateful

applications or use sticky sessions which may not scale properly

•If Data Server is the bottleneck, you may also need a cluster for it.

•More network latency

Solution #3: Clustering

24Solution #3: Stateful vs Stateless

O)

User2Web

Cluster

Data Server

( Internet

Local/Private Network

(Load

Balancer

A

B

OUser1

Session 1

Session 2

Web session stickiness example

+

25

• After a global market campaign, some users started to complain about a slow UI loading in mobile application, when opening the latest weather information from the internet.

Weather Forecast App

26

• Example of browser caching

Solution #4: Caching

[1] Reference: https://developers.google.com/web/fundamentals/performance/optimizing-content-efficiency/http-caching#cache-control

27Solution #4: Caching

O)

User2Web

Cluster

Data Server

(

Internet +

(Load

Balancer

OUser1

(Cache Server

In Memory Caching

Local Caching

Local Caching

Client Caching

Client Caching

Proxy Caching

In Memory Caching

Example: CDN for Static Web Content: html, css, images, etc.

28Solution #4: Caching

Content Delivery Network(CDN)

29

• Pros•Reduce wait time caused by latency problems •Avoid waste of computing resources by caching long-

running operations like remote calls, disk I/O, etc. •Cons

• Need to define metadata for control the cache storage. •May require to minimal hardware for getting better

results (SSD, RAM, etc.), some devices such as mobile have critical limitations.

• You may see out-of-date results while the old cache was not invalidated yet.

Solution #4: Caching

30

• Your app is growing too fast, it is known throughout the world with more than 1 million users in two month only.

•It is time to have your product running in a cloud solution.

Weather Forecast App

31Solution #5: Full Distributed Cloud Service

Comments 32

,

!"

#

$

%

&'

$./run

{

Created by Fabrício Epaminondas ©2015-2020

ContactFABRICIO EPAMINONDAS - linkedin.com/in/fabricioepa

. fabricioepa.wordpress.com

/ fabricioepa

0 @fabricioepa