build-up ruby interpreter what is easy and what is...
TRANSCRIPT
Incremental GCfor Ruby interpreter
Koichi [email protected]
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 1
2014Very important year for me
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 2
10th
Anniversary
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 3
10th
AnniversaryYARV development (2004/01-)
First presentation at RubyConf 2004K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 4
Garbage CollectionImprovements
Good throughput and short pause time
Ruby 2.2 will be released soon.K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 5
RGenGC: Micro-benchmark
1699.805974
87.230735
704.843669
867.740319
0
500
1000
1500
2000
2500
3000
no RGenGC
Tim
e (m
s)
total mark total sweep
6K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
x2.5 faster
RGenGC: Pause time
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 7
Today’s topic
•Use incremental GC algorithm for major GC to reduce long pause time
•Ruby 2.2 will have it!!
BeforeRuby 2.1
Ruby 2.1 RGenGC
Incremental GC
Ruby 2.2 Gen+IncGC
Throughput Low High Low High
Pause time Long Long Short Short
Goal
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 8
Achievements: RGenGC+RincGC
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 9
Who am I?Koichi Sasada as a Programmer
•CRuby committer since 2007/01
•Original YARV developer since 2004/01
• From Japan
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 10
Who am I?Koichi Sasada as a Employee
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 11
Who am I?Koichi Sasada as a Employee
•A member of Matz team• Full-time CRuby developer•Working in Japan•Mission of our team is to improve “QUALITY” of
CRuby interpreter
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 12
Upcoming Ruby 2.2What’s next?
http://www.flickr.com/photos/adafruit/8483990604
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 13
Ruby 2.2Syntax
•No notable changes (maybe)
• Symbol key of Hash literal can be quoted• {“foo-bar”: baz}#=> {:”foo-bar” => baz}
#=> not {“foo-bar” => baz} like JSON
TRAP: easy to misunderstand
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 14
Ruby 2.2Classes and Methods
• Some methods are introduces•Kernel#itself• String#unicode_normalize• Etc.nprocessors•…
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 15
Ruby 2.2Internal changes
• Remove obsolete C-APIs
• Hide internal definitions of data type
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 16
Ruby 2.2Improvements
• Improve GC• Symbol GC• 4 ages generational GC• Incremental GC (today’s topic)
• Improve the performance of keyword parameters
• Use frozen string literals if possible
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 17
Ruby 2.2Symbol GC
before = Symbol.all_symbols.size
1_000_000.times{|i| i.to_s.to_sym} # Make 1M symbols
after = Symbol.all_symbols.size; p [before, after]
# Ruby 2.1
#=> [2_378, 1_002_378] # not GCed# Ruby 2.2 (dev)
#=> [2_456, 2_456] # GCed!
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 18
Ruby 2.2Fast keyword parameters
# time of this program
def foo(k1: nil, k2: nil, k3: nil, k4: nil, k5: nil, k6: nil)
10_000_000.times{foo(k1: 1, k2: 2, k3: 3, k4: 4, k5: 5, k6: 6)}
17.170841
1.155340
5
10
15
20
Execution time (sec)
Before opt. After opt.
x15 faster!
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 19
Break
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
http://www.flickr.com/photos/donkeyhotey/8422065722
20
http://www.flickr.com/photos/circasassy/6817999189/
Garbage collectionThe automatic memory management
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 21
History of CRuby’s GC
•1993/12 Ruby 0.9: Conservative mark and sweep GC•Simple algorithm•Easy to implement C extensions
•2011/10 Ruby 1.9.3: Lazy sweep•To reduce pause time on sweep
•2013/02 Ruby 2.0: Bitmap marking•To make CoW friendly
•2013/12 Ruby 2.1: RGenGC•To improve throughput
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 22
marked
marked marked
markedmarked
Since birth of RubySimple Mark & Sweep
1. Mark reachable objects from root objects
2. Sweep unmarked objects (collection and de-allocation)
Root objects
free
traverse
traverse traverse
traverse traverse
free
free
Collect unreachable objects
23K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
Since Ruby 2.1RGenGC
•Weak generational hypothesis:
“Most objects die young”
→ Concentrate reclamation effort
only on the young objects
http://www.flickr.com/photos/ell-r-brown/5026593710
24K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
Since Ruby 2.1RGenGC
• Separate young generations and old generations• Create objects as youngest generation• Promote to old generations after surviving GCs
• Many minor GC and rare major GC• Usually, GC on only young space (minor GC)• GC on both spaces if no memory (major/full GC)
→ Improve total throughput
25K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
Root objects
new
new new
new/free
newnew
traverse
traverse traverse
traverse traverse
new/free
old/free
Don’t collect old objecteven if it is unreachable.
collect
1st MinorGC
old
old old
oldold
26
Since Ruby 2.1RGenGC [Minor M&S GC]
• Mark reachable objects from root objects.• Mark and promote to old
generation• Stop traversing after old objects
→ Reduce mark overhead
• Sweep not (marked or old) objects
• Can’t collect Some unreachable objects
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
Since Ruby 2.1RGenGC [Minor M&S GC]
• Mark reachable objects from root objects.
• Mark and promote to old generation
• Assumption: “Old objects only refer old objects”
• Stop traversing after old objects
→ Reduce mark overhead
• Sweep not (marked or old) objects
• Can’t collect Some unreachable objects
Root objects
old
old old
new/free
oldold
traverse
ignore ignore
ignore ignore
new/free
old/free
Don’t collect old objecteven if it is unreachable.
collect
2nd MinorGC
27K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
Since Ruby 2.1RGenGC [Remember set]
• Assumption: “Old objects only refer old objects”
• However old objects can refer young objects by adding reference from old to new objects
→ Ignore traversal of old object
→ Minor GC causes marking leak!!
• Because minor GC ignores referenced objects by old objects
Root objects
new
old
new
oldold
traverse
traverse
ignore ignore
oldCan’t mark new object!
→ Sweeping living object! (Critical BUG)
ignore
28K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
1. Detect creation of an [old->new] type reference
2. Add an [old object] into Remember set (RSet) if an old object refer new objects
Root objects
new
old
new
oldold
traverse
traverse
Rememberignore ignore
old
Rememberset (RSet)
29K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
Since Ruby 2.1RGenGC [Remember set]
1. Mark reachable objects from root objects• Remembered objects are
also root objects
2. Sweep not (marked or old) objects
Root objects
new
old
new
oldold
traverse
traverse
traverseignore ignore
old
Rememberset (RSet)
collect
new/free
new/free
traverse
30
Since Ruby 2.1RGenGC [Remember set]
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
Since Ruby 2.1RGenGC [Write barrier]
• To detect [old→new] type references, we need to insert “Write-barrier” into interpreter for all “Write” operation
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 31
newold
“Write barrier”[Old->New] type reference detected!
Since Ruby 2.1RGenGC: Challenge
• Trade-off of Speed and Compatibility• Introducing “Write barriers” completely is very hard• Can we achieve both speed-up w/ GenGC and keeping
compatibility?
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 32
Since Ruby 2.1RGenGC: Key idea
IntroduceWB unprotected objects
33K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
Since Ruby 2.1RGenGC: Key idea
• Separate objects into two types• WB protected objects
• WB unprotected objects
• Decide this type at creation time• A class care about WB → WB protected object
• A class don’t care about WB → WB unprotected object
34K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
Since Ruby 2.1RGenGC: Key idea• Normal objects can be changed
to WB unprotected objects• “WB unprotect operation”• C-exts which don’t care about
WB, objects will be WB unprotected objects
• Example• ptr = RARRAY_PTR(ary)• In this case, we can’t insert WB
for ptr operation, so VM shade “ary”
WB p.obj
WB unp.obj
VM
Unprotect
Create
Now, WB unprotected objectcan’t change into WB p. object
35K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
Since Ruby 2.1RGenGC: Rules
• Treat “WB unprotected objects” correctly• At Marking
1. Don’t promote WB unprotected objects to old objects
2. Remember WB unprotected objects pointed from old objects
• At WB unprotect operation for old WB protected objects
1. Demote objects
2. Remember this unprotected objects
36K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
Since Ruby 2.1RGenGC: [Minor M&S GC w/WB unp. objects]
• Mark reachable objects from root objects
• Mark WB unprotected objects, and *don’t promote* them to old gen objects
• If WB unprotected objects pointed from old objects, then remember this WB unprotected objects by RSet.
→ Mark WB unprotected objects every minor GC!!
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 37
Root objects
new
new
WBunp.new
traverse
traverse
traverse traverse
old
Rememberset (RSet)
collect
new/free
new/free
traverse
1st MinorGC
mark and remember
rememberold
old
old
newold
38
Root objects
old
old
old
WB unp.old
traverse
ignore
ignoreignore
old
Rememberset (RSet)
collect
new/free
new/free
traverse
traverse
2nd MinorGC
new
traverse
Since Ruby 2.1RGenGC: [Minor M&S GC w/WB unp. objects]
• Mark reachable objects from root objects
• Mark WB unprotected objects, and *don’t promote* them to old gen objects
• If WB unprotected objects pointed from old objects, then remember this WB unprotected objects by RSet.
→ Mark WB unprotected objects every minor GC!!
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
old
Since Ruby 2.1RGenGC: [Unprotect operation]
• Anytime Object can give up to keep write barriers
→ [Unprotect operation]
• Change old WB protected objects to WB unprotected objects• Example: RARRAY_PTR(ary)
(1) Demote object (old → new)(2) Register it to Remember Set
old
WB unp.
old
new
Rememberset (RSet)
39K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
RGenGC: Micro-benchmark
0
5
10
15
20
251 6
11
16
21
26
31
36
41
46
51
56
61
66
71
76
81
86
91
96
10
1
10
6
11
1
11
6
12
1
12
6
13
1
13
6
Tim
e (m
s)
n-th GC
mark time/no sweep time/no mark time/RGenGC sweep time/RGenGC
40K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
RGenGC: Micro-benchmark
1699.805974
87.230735
704.843669
867.740319
0
500
1000
1500
2000
2500
3000
no RGenGC
Tim
e (m
s)
total mark total sweep
41K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
RGenGC: Rdoc application
0
20
40
60
80
100
120
140
160
180
200
1 5 9
13
17
21
25
29
33
37
41
45
49
53
57
61
65
69
73
77
81
85
89
93
97
10
1
10
5
10
9
11
3
11
7
12
1
12
5
12
9
13
3
13
7
14
1
14
5
Tim
e (m
s)
nth-GC
mark time/no sweep time/no mark time/RGenGC sweep time/RGenGC
42K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
RGenGC: Rdoc application
total mark total sweep
no 8210.724461 3524.425873
RGenGC 1049.27184 3579.584833
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
Tim
e (m
s)
no RGenGC
43K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
RGenGC: Rdoc application
no RGenGC
total sweep 3524.425873 3579.584833
total mark 8210.724461 1049.27184
others 100907.4225 100640.8472
0
20000
40000
60000
80000
100000
120000
Tim
e (m
s)
others total mark total sweep
44K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
RGenGC: Rdoc application
0
500,000
1,000,000
1,500,000
2,000,000
2,500,000
3,000,0001 5 9
13
17
21
25
29
33
37
41
45
49
53
57
61
65
69
73
77
81
85
89
93
97
10
1
10
5
10
9
11
3
11
7
12
1
12
5
12
9
13
3
13
7
14
1
14
5
available slots live slots old objects unprotected objects remembered unprotected objects
45K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
RGenGC: Rdoc application
0
10,000
20,000
30,000
40,000
50,000
60,000
70,000
1 5 9
13
17
21
25
29
33
37
41
45
49
53
57
61
65
69
73
77
81
85
89
93
97
10
1
10
5
10
9
11
3
11
7
12
1
12
5
12
9
13
3
13
7
14
1
14
5
unprotected objects remembered unprotected objects
46K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
Since Ruby 2.1RGenGC timing chart
Ruby Mark Sweep
Stop the (Ruby) World
Sweep Sweep Sweep Sweep
2.0.0 GC (M&S w/lazy sweep)
w/RGenGC (Minor GC)
RubyMark
Sweep
Stop the(Ruby)World
Sweep Sweep Sweep Sweep
47
Ruby Mark Sweep
Stop the (Ruby) World
Sweep Sweep Sweep Sweep
w/RGenGC (Major GC)
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
Issue of RGenGC: Long pause time
RGenGC achieves high throughput
Minor GC stops only short pause time
Major GC still stops long pause time
→ Introducing Incremental GC for major GC
Generational GC Incremental GC Gen+Inc GC
Throughput High Low High
Pause time Long Short Short
48K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
RincGC: Restricted Incremental GC algorithms
RincGC algorithm is implemented for Ruby 2.2.
49K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
Incremental GCWell-known GC algorithm to reduce pause time
• Do GC steps incrementally• Interleaving with Ruby’s execution (mutator) and GC process.
• Lazy sweep is part of an incremental GC
Ruby Mark Sweep
Stop the (Ruby) World
Sweep Sweep Sweep Sweep
STW GC
RubyMark
Sweep Sweep Sweep
Inc GCMark Mark Mark Mark
Short pause time No total time changeK.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 50
Terminology: Tri-color GC
•Define three colors for objects• White objects is not traversed objects• Grey objects are marking objects• Black objects are marked objects
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 51
Incremental GC 1. Color all objects “white”
2. Grey root objects
3. Choose a grey object and grey all reachable white objects, and black the chosen object (incremental marking)
4. Finish marking when no grey objects
5. Sweep white objects as unmarked objects
Root objects
traverse
traverse traverse
traverse traverse
Collect white objects
52K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
Incremental GC requires WBs
Root objects
traverse
traverse
Mutator can add reference from a Black object to a white object!!→ Mark miss!
No more traversal from black objects
53K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
Incremental GC requires WBs
Root objects
traverse
traverseUse write barrier to detect an addition of references from black objects to white objects, and grey black objects
54K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
RincGC: Restricted Incremental GC using WB-unprotected objects
•Use WB unprotected objects like RGenGC
• Introducing a new rule: “Scan all black WB unprotected objects at the end of incremental GC at once”• WB unprotected objects can point white objects
• Scan from all (“Black” and “WB unprotected objects”) at once (stop the world marking)
55K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
Root
Root objects
WB unp.
1. Color all objects “white”2. Grey root objects3. Choose a grey object and grey
reachable white objects, and black the chosen object (incremental marking)
4. Finish marking when no grey objects
5. Scan all black WB unprotected objects at once
6. Sweep white objects as unmarked objects
56
WB unp.
RincGCRestricted Incremental GCusing WB-unprotected objects
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
RincGC: Discussion
•Long pause time than usual incremental GC step• This technique can introduce long pause time, relative to
the number of WB unprotected objects at last. This is why this algorithm is named “Restricted”
• Similar/shorter pause time than “Minor GC” of RGenGC.
57K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
Implementation
58K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
Ruby’s implementationState chart
garbage_collect()
marks_start()
marks_step()
marks_finish()
sweep_start()
sweep_step()
sweep_finish()
marks_continue()
sweep_continue()
State: marking State: sweeping
State: none
newobj()
if (incremental_marking)
if (sweep_pages)
if (no pages)
Direct transition
To mutator
59
Ruby program (mutators)
Ruby’s implementationWB protected/unprotected
• Make popular class instances WB protected• String, Array, Hash, and so on
• Implement “unprotect operation” for Array and so on
• Remain WB unprotected objects• Difficult to insert WBs: a part of Module, Proc (local
variables) and so on.• Minor features
60K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
Ruby’s implementationData structure
• Introduce 2 bits age information to represent young and old objects
• Age 0, 1, 2 is young object
• Age 3 is old object
• Surviving 1 GC increase one age
• Add 3 bits information for each objects (we already have mark bit)
• WB unprotected bit
• Long lived bit (old objects or remembered WB unprotected objects)
• Remembered old object bit / Marking (Grey) bit• They can share 1 bit field because the former is used only at minor GC and the
latter is used only at major GC (incremental GC)
61K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
Ruby’s implementationBitmap technique
• Each bits are managed by bitmap technique• Easy to mange remember set• Fast traversing• Easy to get a set
• Remember set: (Remembered old object bitmap) | (Long lived bitmap & WB unp. Bitmap)
• Living unprotected objects: (mark bitmap & WB unprotected bitmap)
62K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014
RincGC: Evaluation
• Measure pause times for <https://github.com/tenderlove/ko1-test-app> by Aaron Patterson
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 63
Evaluation
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 64
Evaluation
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 65
0
0.002
0.004
0.006
0.008
0.01
0.012
0.014
0.016
0.018
16
91
37
20
52
73
34
14
09
47
75
45
61
36
81
74
98
17
88
59
53
10
21
10
89
11
57
12
25
12
93
13
61
14
29
14
97
15
65
16
33
17
01
17
69
18
37
19
05
19
73
20
41
21
09
21
77
22
45
23
13
23
81
24
49
GC
pau
se t
ime
(sec
)
2,500 GC's pause time since one major GC
pause time (rgengc) pause time (rincgc)
Long major marking time of RGenGC
Incremental Major GC of RincGC
Lazy sweep pauses only short time
Same pause time for Minor marking time
Evaluation
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 66
rgengc rincgc
max 0.015923438 0.004662491
0
0.005
0.01
0.015
0.02
maximum pause time
NOTE: Incremental GC is not silver bullet
• Incremental GC does not guarantee improving your application’s response time• Incremental GC does not reduce total GC time, so
that a big task includes several major GC doesn’t improve its response time.•Check GC counts with GC.stat(:major_gc_count)
and GC.stat(:minor_gc_count) for each request.
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 67
Summary
• Introducing incremental GC algorithm into major GC to reduce long pause time
•Ruby 2.2 will have it!!
BeforeRuby 2.1
Ruby 2.1 RGenGC
Incremental GC
Ruby 2.2 Gen+IncGC
Throughput Low High Low High
Pause time Long Long Short Small
Goal
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 68
Thank you for your attention
Koichi Sasada<[email protected]>
K.Sasada: Incremental GC for Ruby interpreter, RubyConf2014 69