garbage collection without paging

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

Garbage CollectionWithout Paging

Matthew Hertz, Yi Feng,Emery Berger

University of Massachusetts Amherst

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

Garbage Collection Performance

Garbage collection now performs reasonably well

High throughputLow pause timesGiven large heap and sufficient memory

But: what happens when there’s not enough RAM?

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

GC Performance While Paging

RAM

Hard Disk

Heap: most pages in RAM, one on diskGC begins

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

GC Performance While Paging

Collector touches an evicted page...

RAM

Hard Disk

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

GC Performance While Paging

... bringing the page into memory ...

RAM

Hard Disk

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

GC Performance While Paging

RAM

Hard Disk

... but triggers another page eviction.

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

GC Performance While Paging

RAM

Hard Disk

... but triggers another page eviction.

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

GC Performance While Paging

Collector touches newly-evicted page...

RAM

Hard Disk

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

GC Performance While Paging

RAM

Hard Disk

Collector touches newly-evicted page...

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

GC Performance While Paging

... leading to the eviction of yet another page…

RAM

Hard Disk

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

GC Performance While Paging

… which eventually triggers more paging.

RAM

Hard Disk

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

Program Throughput

Paging causes a 40 to 62-fold increase in runtime

Execution Time of pseudoJBB w/ 77MB Heap

0

100

200

300

400

500

600

212 163 153 143 133 123 113 103 93Available Memory

Exec

uti

on T

ime

(in s

)

GenCopy

GenMS

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

Outline

MotivationGC without paging

Cooperative garbage collectionBookmarking

Results

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

The problem

Garbage collector: VM-obliviousExamines all reachable objectsLacks knowledge of page residency

Treats evicted and resident pages identically

Virtual memory: GC-obliviousGC & application have different access patterns

Likely to evict pages needed by GC

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

Cooperative Garbage CollectionBookmarking

collectorExtended virtual memory manager

Select victim(s)Update

residency

page eviction notification

victim page(s)Page

replacement

In response to notifications, BC:Adjusts heap size to fit in main memoryPrevents eviction of important pagesAvoids touching non-resident pages

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

Avoiding Page Evictions

When notified, avoid a pending eviction…

0 000

RAM

Hard Disk

0

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

Avoiding Page Evictions

…find a page BC knows to be empty…

0 000

RAM

Hard Disk

0

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

Avoiding Page Evictions

… and discard it…

0 000

RAM

Hard Disk

0

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

Avoiding Page Evictions

… eliminating the need to evict a page.

0 000

RAM

Hard Disk

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

Limit of Heap Sizing

Could collect, compact, compress, etc.Eventually:

Will run out of pages to discardGoing to have to evict non-empty pagesResult: Paging

Can we avoid this?

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

Bookmarks

We introduce bookmarks:Summaries of connectivity info on evicted pages

References from objects on the page

These summaries enable GC w/o paging

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

0 000

Bookmarking

RAM

Hard Disk

Process page before eviction…

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

0 000

Bookmarking

B

B

... by following pointers & bookmark-ing targets...

RAM

Hard Disk

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

0 000

Bookmarking

B

B

... increment the referring page count...

RAM

Hard Disk

1

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

1 000

Bookmarking

B B

B B

B

B

B

... conservatively bookmark objects on the page...

RAM

Hard Disk

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

0 00

Bookmarking

... then tell extended VM to evict the page.

RAM

Hard Disk

0B B

B B

B

B

B

1

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

Bookmarking Details

Cheap summary of connectivityOne bit per object: freeOne word per page: referring page countBookmarks cleared when count = zero

Use bookmarks as secondary rootsduring garbage collection

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

0B B

B B

B

1 00

RAM

Hard Disk

Collection with Bookmarks

Process objects as usual, but...

B

B

roots

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

0B B

B B

B

1 00

RAM

Hard Disk

Collection with Bookmarks

... ignore any references to evicted pages.

B

B

roots

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

0B B

B B

B

1 00

RAM

Hard Disk

Collection with Bookmarks

Use bookmarks to recreate evicted references...

B

B

roots

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

0B B

B B

B

1 00

RAM

Hard Disk

Collection with Bookmarks

... and continue collection.

roots

B

B

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

0B B

B B

B

1 00

RAM

Hard Disk

Collection with Bookmarks

B

B

roots

Result: Garbage collection without paging!

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

0B B

B B

B

1 00

RAM

Hard Disk

Collection with Bookmarks

B

B

roots

Note: can waste space on evicted pages.

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

Bookmarking Incompleteness

Space waste not just on evicted pagesCollection with bookmarks is necessarily incomplete

Not guaranteed to reclaim all memory

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

0B B

B B

B

1 00

RAM

Hard Disk

Bookmarking Incompleteness

B

B

When a reference to an evicted object changes…

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

0B B

B B

B

1 00

RAM

Hard Disk

Bookmarking Incompleteness

B

B

When a reference to an evicted object changes…

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

1 0

0

0

RAM

Hard Disk

Bookmarking Incompleteness

B B

B B

B

B

B

…it can make evicted objects unreachable.

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

0B B

B B

B

1 00

RAM

Hard Disk

Bookmarking Incompleteness

B

B

But bookmarks cannot be removed…

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

0B B

B B

B

1 00

RAM

Hard Disk

Bookmarking Incompleteness

B

B

…retaining unreachable heap objects.

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

Bookmarking Completeness

Worst-case: completeness requires duplicating evicted pages

See paper for more info

How can we preserve completeness?

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

0B B

B B

B

1 00

RAM

Hard Disk

Bookmarking Completeness

B

B

If the heap becomes full…

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

0

0 00

RAM

Hard Disk

Bookmarking Completeness

…BC removes all bookmarks…

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

0

0 00

???

???

Bookmarking Completeness

…and performs a VM-oblivious collection...

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

0

0 00

???

???

Bookmarking Completeness

…reclaiming all unreachable objects.

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

0 0

0

0

???

???

Bookmarking Completeness

…reclaiming all unreachable objects.

BC’s worst case is other collectors’ common case

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

BC Performance Optimizations

Uses generational design similar to GenMS

Yields good performance when not paging

Compacts heap to reduce pressurePrevents single object from tying down a page

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

Outline

MotivationGC without paging

Cooperative garbage collectionBookmarking

Results

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

Experimental Methodology

Extended Linux kernel 2.4.20Eviction notificationvm_relinquish()Added only 600 LOC

Jikes RVM 2.3.2 & MMTkCompare BC to MarkSweep, SemiSpace, CopyMS, GenCopy, GenMS

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

Throughput w/o Memory Pressure

BC runtime comparable to GenMS

Geo. Mean for Execution Time Relative to BC

0.95

1.00

1.05

1.10

1.15

1.00 1.25 1.50 1.75 2.00 2.25 2.50 2.75 3.00 3.25 3.50 3.75 4.00Relative Heap Size

Rel

ativ

e Ex

ecu

tio

n T

ime

BCGenCopyGenMS

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

Throughput while Paging

BC throughput closely follows ideal curve

Execution Time for pseudoJBB w/ 77MB Heap

0

100

200

300

400

500

600

212 163 153 143 133 123 113 103 93

Available Memory

Exec

utio

n Ti

me

(in s

)

IdealBCGenCopyGenMS

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

BMU Curve w/ Memory Pressure

Bookmarking crucial for good performance

0.0%1566 ms

71.9%12231 ms

0.0%81591 ms

39.1%467944 ms

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

Summary of Results

When not paging:BC as fast as GenMS

When paging:vs. GenMS (fastest when not paging)

41x faster, avg pause up to 218x smaller

vs. CopyMS (next fastest when paging)5x faster, avg pause up to 45x smaller

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

Conclusion

Bookmarking collector available at: http://www.cs.umass.edu/~hertz

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

Thank you

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

Other Pointer Summarizations

Pointer CompressionSave space by compressing pointersTracing pointers becomes expensive

Remembered SetsAdd evicted pointers to buffersRequires space upon pages eviction

Per Object Referring Page CountsIncreases BC overheads

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

Related Work

VM-Sensitive Garbage CollectionLinearizing LISP lists [Bobrow/Murphy]

Does not know or use which heap pages are memory resident

Independent heap regions [Bishop]Cannot avoid page faults when region contains evicted pages

Ephemeral garbage collection [Moon]Must access evicted pages when they contain pointers into generations being collected

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

Related Work

VM-Cooperative Garbage CollectionDiscarding empty pages [Cooper, et al.]

No mechanism for evicting non-empty pages

Shrinking heap size [Alonso/Appel]Responds to memory pressure changes only after a collection

Automatic heap sizing [Yang, et al.]Orthogonal approach that determines proper heap size

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

Throughput w/ Memory Pressure

BC outperforms fixed nursery collectors

212 163 153 143 133 123 113 103 930

50000

100000

150000

200000

250000

300000

350000

400000

450000

500000Execution Time for pseudoJBB w/ 77MB Heap

BCBC w/ Fixed NurseryBC w/ Fixed Nursery and Resizing OnlyGenCopy w/ Fixed NurseryGenMS w/ Fixed Nursery

Available Memory

Exe

cutio

n Ti

me

(ms)

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

Multiprogramming Performance

491 292 282 272 262 252 242 232 222 212 202 192 182 172 1620

100000

200000

300000

400000

500000

Elapsed Time for 2 runs of pseudoJBB, 77MB HeapBCGenCopyGenMSCopyMSSemiSpace

Available Memory (MB)

Tota

l Wal

l Clo

ck T

ime

(ms)

BC performs well in many environments

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science

Experimental Methodology

Benchmarks:SPECjvm98, ipsixql, jython, and pseudoJBB

“Opt-and-reset” methodologyFirst iteration optimizes all codeRecord results from second run