Skyway: Connecting Managed Heapsin Distributed Big Data Systems

Khanh Nguyen1, Lu Fang1, Christian Navasca1, Guoqing Xu1, Brian Demsky1, and Shan Lu2

1University of California, Irvine2University of Chicago

Skyway: Connecting Managed Heapsin Distributed Big Data Systems

Khanh Nguyen1, Lu Fang1, Christian Navasca1, Guoqing Xu1, Brian Demsky1, and Shan Lu2

1University of California, Irvine2University of Chicago

Motivation

• Inter-node data transfer is a CPU-intensive bottleneckin distributed big data systems

Tim

e (s

ec)

Serializers

Read I/OWrite I/ODeserializationSerializationComputation

0

350

700

1050

1400

1750

Serializer Ser. Deser. S+D

Java 16.3% 17.8% 34.1%Kryo 18.2% 14.1% 32.3%

Performance breakdown of Spark’s TriangleCounting over the LiveJournal graph

•Existing approaches– Kryo is faster, but developer must create S/D functions– User-defined functions are invoked for every field of every object– Correct and efficient S/D functions are labor-intensive to write

We want a systematic solution to reduce S/D costs

Skyway Approach

Main Idea:transfer entire object

•Does not invokeserialization/deserializationfunctions

•Leads to savings in:

1. CPU time spent on runtimereflection, and

2. manual developer involvement

*Type Reference

Hashcode

Other(GC flag, lock flag, etc.)

Field 1

*Field 2(reference)

Field 3

Type Reference

Hashcode

Other(GC flag, lock flag, etc.)

Field 1

Field 2(reference)

Field 3

Type String

Field 1

Inlined referenced

object

Field 3

SkywayAn object

Java Serializer

Object layout in JVM and how an object is handled bythe Java Serializer and Skyway

•Skyway vs. Conventional Data Transfer

Machine dependent meta data, reference, etc. Machine independent meta data, reference, etc.

Machine independent object dataObject reference

Object domainSkyway

Binary domain

Sender

Serialization

Deserialization

Receiver

Network

Design and Implementation

Challenges Solutions– Reference fields −−−→ X Relative addresses with linear time adjustment– Type representation −−−→ X Automated global type numbering system

• Implemented in Oracle’s production JVM OpenJDK 8 build 25.71

– Across-the-stack modifications: the object/heap layout, the classloader subsystem, theproduction Parallel Scavenge garbage colletor

– Skyway library to interact with runtime systems

Evaluations

Java Serializer Benchmark Set

– An extensive, comprehensive comparison with the existing 90 serializers

Time(ms)

0

5000

10000

###

skyw

ay #

##

colfe

r

pro

tost

uff

pro

tost

uff

-ma

nu

al

pro

tob

uf/

pro

tost

uff

da

take

rne

l

pro

tost

uff

-gra

ph

pro

tost

uff

-ru

ntim

e

pro

tob

uf/

pro

tost

uff

-ru

ntim

e

pro

tost

uff

-gra

ph

-ru

ntim

e

kryo

-ma

nu

al

smile

/jack

son

/ma

nua

l

kryo

-op

t

kryo

-fla

t-p

re

avr

o-g

en

eri

c

cbo

r/ja

ckso

n/m

an

ua

l

avr

o-s

pe

cific

wo

bly

kryo

-fla

t

wo

bly

-co

mp

act

cbo

r/j.s

on

+a

ft.b

urn

er/

da

tab

ind

cap

np

roto

cbo

r-co

l/ja

ckso

n/d

ata

bin

d

smile

/j.so

n+

aft

.bu

rne

r/d

ata

bin

d

smile

-co

l/ja

ckso

n/d

ata

bin

d

thri

ft-c

om

pa

ct

fst-

flat-

pre

thri

ft

Oth

er

63

S/D

lib

rari

es

Serialization Deserialization Network

Skyway outperforms all existing S/D libraries

Apache Spark 2.1.0 (released December 2016)

– An 11-node cluster: each has 16 cores, 32GB memory, 1 SSD, connected via InfiniBand– Four applications on four real-world graphs: LiveJournal, Orkut, UK-2005, and Twitter

Serializers

Tim

e (s

ec)

Serializers

Tim

e (s

ec)

Serializers

Tim

e (s

ec)

Serializers

Tim

e (s

ec)

UK-TCUK-PRUK-CCUK-WC

LJ-TCLJ-PRLJ-CCLJ-WC OR-WC OR-CC OR-PR OR-TC

TW-WC TW-PR TW-TCTW-CC

0

50

100

150

200

0

300

600

900

0

2000

4000

6000

0

1000

2000

3000

0

50

100

150

200

250

0

500

1000

1500

0

1000

2000

3000

0

4000

8000

12000

16000

0

300

600

900

1200

0

2000

4000

6000

0

5000

10000

15000

0

15000

30000

45000

0

1500

3000

4500

0

5000

10000

15000

0

5000

10000

15000

20000

25000

0

15000

30000

45000

60000

Makes Spark run 36% and 16% faster thanwith Java and Kryo S/D

Read I/OWrite I/ODeserializationSerializationComputation

Apache Flink 1.3.2 (released August 2017)

– An 11-node cluster: each has 16 cores, 32GB memory, 1 SSD, connected via InfiniBand

– 5 TPC-H SQL Queries using 100GB of input

Tim

e (s

ec)

Serializers

QA QB QC QD QE

0

15000

30000

45000

60000

0

2500

5000

7500

10000

0

1500

3000

4500

6000

7500

0

2500

5000

7500

10000

0

10000

20000

30000 Read I/OWrite I/ODeserializationSerializationComputation

Makes Flink run 19% faster than with built-in S/D

Conclusions

•Skyway is novel : the first to provide S/D-free data transfer

•Skyway is efficient :– Outperforms all existing S/D libraries by 2.2× — 67.3×– Improves Apache Spark by up to 73% (Java), and up to 54% (Kryo)– Improves Apache Flink by 12% — 29%

•Skyway is practical : a JVM-based solution, applicable to allJVM-based languages while requiring zero user effort