utilitarian performance isolation in shared ssds...flash memory summit 2017, background design...

Memory & Storage Architecture Lab.@ Seoul National University

Utilitarian Performance Isolationin Shared SSDs

Bryan S. Kim

(Seoul National University, Korea)


Flash storage landscape

ConclusionDesignBackground Evaluation

Flash device

Flash

chip

Flash

chip

Flash

chip

Flash

chip

Host system

NVMe

• MQ blk IO (SYSTOR13)

• NVMeDirect (HotStorage16)

• SPDK (CloudCom17)

… and more

• Ozone (TC11)

• SSDsim (ICS11)

• Sprinkler (HPCA14)

… and more

Increase

parallelism!

(2013)Expose

parallelism!

Exploit

parallelism!


Noisy neighbors

C. Petersen and A. Huffman, “Solving Latency Challenges with NVM Express SSDs at Scale”,

Flash memory summit 2017,



Unified sharing of resources (free-for-all)

Blue tenant : large capacity; infrequent accessRed tenant : Write-intensiveGreen tenant : QoS-sensitive


flash chip 0 flash chip 1 flash chip 2 flash chip 3



Partitioning of resources (egalitarian)

Under-utilized parallelism High GC overhead

Lack of read parallelism






Slashing parallelism for isolation


0

1

2

DAS-AS DAS-PS DTRS LM-TBE MSN-CFS RAD-AS RAD-BE

Norm

. avg. R

T4x4 2x4 1x4 1x2

3.42

Performance suffers from reduced parallelism!


Dynamic allocation of resources (utilitarian)

Reduced GC overhead

Altruistic sharing of parallelism High-degree of read parallelism






Utilitarian performance isolation

Lessons from storage arrays

Monitor each tenant’s fair share of I/O

Determine optimal data placement

• To balance the load across multiple storage devices…

• … while considering data relocation overheads

Key insight

Flash memory’s challenges Flash memory’s opportunities

• Need to maintain mapping Easy to balance load

• Need to garbage collect Easy to relocate data

The utilitarian approach

Compute tenant’s utility (measure of received service)

Determine the allocation set (a set of chips for writing data) for each tenant

• Allocation sets are mutually exclusive and collectively exhaustive

Allow data relocation among sets if needed



Utility of tenants




Blue tenant’s utility : 0.9Red tenant’s utility : 0.7Green tenant’s utility : 0.8


Load balancing




Red tenant’s writes are striped across a larger set of flash memory chips.Blue’s performance loss is minor.


Data relocation




Garbage collection in chip 1 isolates some of Blue tenant’s data by relocating them to its set.Red tenant’s GC efficiency improves.


Utility function

Utility

Utility of a tenant is high when its reads experience less traffic

Traffic

Traffic of a chip indicates the overall busyness of the chip



Set allocation & data relocation

Set allocation

Objective

• Find allocation set that minimizes max-min ratio of utility across all tenants

Approximation

• Transfer one chip from max utility tenant to min utility tenant

• Avoid thrashing by transferring only if it balances the utility

• Select a chip that experienced least number of reads

Data relocation

Considering the number of reads of a “foreign” block during garbage collection

• “Foreign” blocks that high number of reads are incentivized to relocate to its own set

• Infrequently accessed cold data may remain in another set



Evaluation environment & methodology

Storage system configuration

150GB storage with 28% over-provisioning

• 3 channels x 4 chips/chan

Garbage collection: reclaims space for writes + considers “foreign” block reads

Workload configuration

3 real-world I/O traces collected from MS production servers

• DAS-AS: lowest throughput, highest read-to-write ratio

• DTRS: relatively random workload with bursts of writes

• LM-TBE: large sequential reads and writes



Average performance

Partitioned : dedicates channel to each tenantUnified : shares all resources among tenantsUPI : dynamically allocates based on utility


0

0.2

0.4

0.6

0.8

1

1.2

DAS-AS DTRS LM-TBE

Norm

. av.g

rea

d R

T

Partitioned Unified UPI

0

0.2

0.4

0.6

0.8

1

1.2

DAS-AS DTRS LM-TBE

No

rm.

99

% r

ead

RT

Partitioned Unified UPI

1.34

38.5% reduction

for latency-critical

application

16.1% reduction

for throughput-

oriented application


QoS performance

Partitioned : dedicates channel to each tenantUnified : shares all resources among tenantsUPI : dynamically allocates based on utility


0.95

0.96

0.97

0.98

0.99

1

0 0.5 1 1.5 2Cu

mu

lati

ve

pro

ba

bil

ity

Read response time (ms)

UnifiedPartitionedUPI

0.95

0.96

0.97

0.98

0.99

1

0 0.5 1 1.5 2Cu

mu

lati

ve

pro

ba

bil

ity

Read response time (ms)

UnifiedPartitionedUPI

DAS-AS LM-TBE


Microscopic view


0

10

20

63200 63300 63400 63500 63600 63700

Wr.

BW

(M

B/s

)

Time (seconds)

DAS-AS DTRS LM-TBE

0

100

200

300

400

63200 63300 63400 63500 63600 63700

Av

g.

rea

d R

T (μ

s)

Time (seconds)

DAS-AS DTRS LM-TBE

0

100

200

300

400

63200 63300 63400 63500 63600 63700

Av

g.

rea

d R

T (μ

s)

Time (seconds)

DAS-AS DTRS LM-TBE

Partitioned UPI


Conclusion

Dynamic allocation of resources based on utility

Decouple parallelism, isolation, and capacity

Balancing the load by distributing write traffic

Relocate data through existing SSD management mechanisms


Utilitarian Performance Isolation

reduces average response time

by 16.1% for high-throughput workload

reduces 99% QoS

by 38.5% for latency-critical workload

utilitarian performance isolation in shared ssds...flash memory summit 2017, background design...

Documents