Serverless Computing: Micromanagement on the Clouds

1

Microlab’s Fridays Tech Talks, #3

November 6, 2020

Achilleas Tzenetopoulos, Ph.D. Student

Overview1. Introduction - Going through the Post-PC era

2. Defining Serverless

3. Limitations & Challenges

4. Resource Management related opportunities

5. Other opportunities for computer architects

2

Introduction - Going through the “Post-PC” era

3

Physical Servers / Bare-metal

Introduction - Going through the “Post-PC” era

4

Physical Servers / Bare-metal

Introduction - Going through the “Post-PC” era

5

Physical Servers / Bare-metal

Infrastructure as a Service

Introduction - Going through the “Post-PC” era

6

Physical Servers / Bare-metal

Infrastructure as a Service

Container as a Service

Introduction - Going through the “Post-PC” eraPhysical Servers / Bare-metal

Infrastructure as a Service

Container as a Service

Function as a Service

7

Defining Serverless Computing (1/2) - Benefits

8

Jonas, Eric, et al. "Cloud programming simplified: A berkeley view on serverless computing." arXiv preprint arXiv:1902.03383 (2019).

Serverless computing is a platform that hides server

usage from developers and runs code on-demand

automatically scaled and billed only for the time

the code is running.

1

Defining Serverless Computing (2/2) Serverless enables the deployment of microservices in a more scalable and cost

efficient way.

9

Yu Gan, et. al 2019. An Open-Source Benchmark Suite for Microservices and Their Hardware-Software Implications for Cloud & Edge Systems. In Proceedings of 2019 Architectural Support for Programming Languages and Operating Systems (ASPLOS’19). ACM, New York, NY, USA, 16 pages. https: //doi.org/10.1145/3297858.3304013

Defining Serverless Computing (2/2) Serverless enables the deployment of microservices in a more scalable and cost

efficient way.

10

Yu Gan, et. al 2019. An Open-Source Benchmark Suite for Microservices and Their Hardware-Software Implications for Cloud & Edge Systems. In Proceedings of 2019 Architectural Support for Programming Languages and Operating Systems (ASPLOS’19). ACM, New York, NY, USA, 16 pages. https: //doi.org/10.1145/3297858.3304013

Serviceful

Limitations & Challenges (1/3) - Excessive Data Movement

Rapid scaling

11

(Num)PyWren GEMM read and write amplification

A

B

C

D

E F

G

Fan-out

Fan-in

Benjamin Carver et.al. 2020. Wukong: a scalable and locality-enhanced framework for serverless parallel computing. In Proceedings of the 11th ACM Symposium on Cloud Computing (SoCC '20). Association for Computing Machinery, New York, NY, USA, 1–15. DOI:https://doi.org/10.1145/3419111.3421286

Limitations & Challenges (1/3) - Excessive Data Movement

Rapid scaling

12

(Num)PyWren GEMM read and write amplification

A

B

C

D

E F

G

Fan-out

Fan-in

Benjamin Carver et.al. 2020. Wukong: a scalable and locality-enhanced framework for serverless parallel computing. In Proceedings of the 11th ACM Symposium on Cloud Computing (SoCC '20). Association for Computing Machinery, New York, NY, USA, 1–15. DOI:https://doi.org/10.1145/3419111.3421286

Rapid scaling

13

(Num)PyWren GEMM read and write amplification

A

B

C

D

E F

G

Fan-out

Fan-in

Limitations & Challenges (1/3) - Excessive Data Movement

Benjamin Carver et.al. 2020. Wukong: a scalable and locality-enhanced framework for serverless parallel computing. In Proceedings of the 11th ACM Symposium on Cloud Computing (SoCC '20). Association for Computing Machinery, New York, NY, USA, 1–15. DOI:https://doi.org/10.1145/3419111.3421286

14

(Num)PyWren GEMM read and write amplification

A

B

C

D

E F

G

Fan-out

Fan-in

Limitations & Challenges (1/3) - Excessive Data Movement

Benjamin Carver et.al. 2020. Wukong: a scalable and locality-enhanced framework for serverless parallel computing. In Proceedings of the 11th ACM Symposium on Cloud Computing (SoCC '20). Association for Computing Machinery, New York, NY, USA, 1–15. DOI:https://doi.org/10.1145/3419111.3421286

AWS and Azure fail to provide proper

performance isolation between coresident

instances, and so contention can cause

considerable performance degradation.

1

15

1.Liang Wang, Mengyuan Li, Yinqian Zhang, Thomas Ristenpart, and Michael Swift. 2018. Peeking behind the curtains of serverless platforms. In Proceedings of the 2018 USENIX Conference on Usenix Annual Technical Conference (USENIX ATC '18). USENIX Association, USA, 133–145.2.Tianyi Yu, Qingyuan Liu, Dong Du, Yubin Xia, Binyu Zang, Ziqian Lu, Pingchao Yang, Chenggang Qin, and Haibo Chen. 2020. Characterizing Serverless Platforms with ServerlessBench. In ACM Symposium on Cloud Computing (SoCC ’20), October 19–21, 2020, Virtual Event, USA. ACM, New York, NY, USA, 15 pages. https: //doi.org/10.1145/3419111.3421280

Limitations & Challenges (2/3) - Performance isolation

AWS and Azure fail to provide proper

performance isolation between coresident

instances, and so contention can cause

considerable performance degradation.

1

16

1. Liang Wang, Mengyuan Li, Yinqian Zhang, Thomas Ristenpart, and Michael Swift. 2018. Peeking behind the curtains of serverless platforms. In Proceedings of the 2018 USENIX Conference on Usenix Annual Technical Conference (USENIX ATC '18). USENIX Association, USA, 133–145.2.Tianyi Yu, Qingyuan Liu, Dong Du, Yubin Xia, Binyu Zang, Ziqian Lu, Pingchao Yang, Chenggang Qin, and Haibo Chen. 2020. Characterizing Serverless Platforms with ServerlessBench. In ACM Symposium on Cloud Computing (SoCC ’20), October 19–21, 2020, Virtual Event, USA. ACM, New York, NY, USA, 15 pages. https: //doi.org/10.1145/3419111.3421280

Limitations & Challenges (2/3) - Performance isolation

A serverless function will contend

with others for memory bandwidth,

which implies the platform should

provide an essential isolation

mechanism to guarantee sufficient

bandwidth budgets.

2

17

Limitations & Challenges (2/3) - Performance isolationCloud providers charge customers MB(Ram)/s User is able to define only the memory usage of

her function

AWS tries to allocate a fixed amount of CPU

cycles to an instance based only on function

memory.

AWS employs bin packing techniques in order

to maximize Memory usage per VM instance

However, while this technique is optimal for revenue maximization, it results in resource

contention and prolonged execution time.

1. Liang Wang, Mengyuan Li, Yinqian Zhang, Thomas Ristenpart, and Michael Swift. 2018. Peeking behind the curtains of serverless platforms. In Proceedings of the 2018 USENIX Conference on Usenix Annual Technical Conference (USENIX ATC '18). USENIX Association, USA, 133–145.2.Tianyi Yu, Qingyuan Liu, Dong Du, Yubin Xia, Binyu Zang, Ziqian Lu, Pingchao Yang, Chenggang Qin, and Haibo Chen. 2020. Characterizing Serverless Platforms with ServerlessBench. In ACM Symposium on Cloud Computing (SoCC ’20), October 19–21, 2020, Virtual Event, USA. ACM, New York, NY, USA, 15 pages. https: //doi.org/10.1145/3419111.3421280

18

Liang Wang, Mengyuan Li, Yinqian Zhang, Thomas Ristenpart, and Michael Swift. 2018. Peeking behind the curtains of serverless platforms. In Proceedings of the 2018 USENIX Conference on Usenix Annual Technical Conference (USENIX ATC '18). USENIX Association, USA, 133–145.Tianyi Yu, Qingyuan Liu, Dong Du, Yubin Xia, Binyu Zang, Ziqian Lu, Pingchao Yang, Chenggang Qin, and Haibo Chen. 2020. Characterizing Serverless Platforms with ServerlessBench. In ACM Symposium on Cloud Computing (SoCC ’20), October 19–21, 2020, Virtual Event, USA. ACM, New York, NY, USA, 15 pages. https: //doi.org/10.1145/3419111.3421280

Limitations & Challenges (3/3) - Startup Latency● Οn-demand request execution.

● Startup overhead (including

sandbox preparation, function

loading and initialization)

● Overhead is considerable with

respect to the small and short-lived

execution unit

Resource Management opportunities (1/5)

19

Function compositions

A

B

C

D

E F

G

Fan-out

Fan-in

Data-locality benefits

Parallel execution

benefits

Resource Management opportunities (2/5)

20

Function compositions

Coldstart Latency B E F

A F

Resource Management opportunities (2/5)

21

Function compositions

Leverage hints of the expected arrival time of the next request

Coldstart Latency B E F

A F

Resource Management opportunities (3/5)Function compositions

Coldstart Latency

Interference-aware placement

L3 cacheL2 L1

C

VM

VM

RAM

RAM

VM

RAM

RAM

Cache Misses

Memory Bandwidth

Network Bandwidth

Resource Management opportunities (4/5)Function compositions

Coldstart Latency

Interference-aware placement

Resource fine-tuning on

runtime

A

B

C

D

E F

G

Resource Management opportunities (4/5)Function compositions

Coldstart Latency

Interference-aware placement

A

B

C

D

E F

G

Horizontal & Vertical scaling and runtime

resource management

Satisfy QoS guarantees by increasing

resource allocation of future tasks

Resource fine-tuning on

runtime

Resource Management opportunities (5/5)

System

monitoring

DAG generator

A

B

C

D

E F

GWorkload

partitioning

BEF

AQoS Controller

Application

QoS monitoring

25

Function Scheduling

Runtime resource

tuning

Cost minimization

High utilization

Optimization

Goals s.t QoS

Low power

consumption

26

Other opportunities for computer architects

● Static code analysis

● Leverage domain specific architectures (FPGAs, GPUs, TPUs, etc)

● Language-specific custom processors through hardware and software

co-design

Thank you!

27

Bonus: Serverless on Edge

28

Edge servers

Sensors

Computing power

Data movement cost

Function’s fine granularity enables an even more distributed placement

Efficient, interference-aware co-location on edge-devices with respect to QoS requirements can dodge unnecessary data movement