Heap Off Memory

WTF??

Reducing Heap memory stress

Abstract

* Java memory fundamental

* heap off memory principles

* heap-off cache with Apache DirectMemory

/me

Olivier Lamy

* Open Source Architect @Talend Apache Team* Apache Member: Maven, Archiva, Tomcat, DirectMemory, Build Infra, etc..* Jenkins* Add bugs in various OpenSource projects* twitter.com/olamy olamy.blogspot.com

Java Memory Fundamental

Java Memory Fundamental

The cool part

* Automatic memory allocation (do you remember when you used malloc)

* Garbage collector (GC) (no more free)

Java Memory Fundamental

The bad part

* stop-the-world mode

* proportionnal to memory's size

* unpredictable unresponsive application: complicated with tight SLA

Heap Off Memory

Not in the Heap process

Limited GC stop

Unlimited storage

Heap Off MemoryHow to use that?

Native libraries with JNAhttps://github.com/twall/jna

ByteBuffer @since 1.4 with nioByteBuffer.allocate( size ) (not off heap)ByteBuffer.allocateDirect( size )

You can use sun.misc.Unsafe but can cause issue and not portable

Heap Off MemoryHow to use that?

You can manipulate only Buffer (ByteBuffer byte[] etc..)!

So you must serialize/deserialize your datas

Maximum size with -XX:MaxDirectMemorySize=

Our use case: Cache

RAM:10-60 nsNetwork: 10000-30000 nsSSD Disk: 70000-120000 nsDisk: 3000000-10000000 ns

Cache must use memory!

On-Heap Cache

Objects stored by reference (no de/serialisation)

GC storm effect when refreshing/removing objects!

Off-Heap Cache

de/serialisation overhead (hopefully some very performant libraries exist for that)

Cache objects payload no more affecting GC

Solutions:

* Terracotta BigMemory (off-heap storage on top of ehcache)

* Infinispan (by Jboss)

* Huge collections

* Apache DirectMemory

Apache Direct Memory

Goals: Apache Direct Memory is a multi layered cache implementation featuring off-heap memory storage to enable caching of java objects without degrading jvm performance.

Apache Direct Memory

Joined Apache Incubator end 2011

12 developpers ATM

Under development:Memory allocation service just rewrite

APIs are subject to be changed and bugs to be found!

Design & principles

ByteBuffer.allocateDirect is the foundation of the cache

ByteBuffers are allocated in big chunk and splitted for internal use

Design & principles

Build on layersCachingService: serialize object (pluggable)

MemoryManagerService: Compute ByteBuffer access

ByteBufferAllocatorService: Eventually deals with ByteBuffer

ByteBuffers Allocation

Merging StrategyNo memory wasted

Suffers from fragmentation

Need synchronization at de/allocation

Fixed Size buffers allocationMemory wasted if size not correctly configured

No fragmentation

Use case (1)

Multi layers cacheMost used objects are cached on heap, the rest off-heap (maybe overflow to disk)

Sounds like ehcache with BigMemory

Hard coded class to use: net.sf.ehcache.store.offheap.OffHeapStore

So same package class name in Apache DirectMemory (https://jira.terracotta.org/jira/browse/EHC-940)

Demo

Use case (2)

Cache Server la memcache tru http(s)

But with a REST Api and written in Java

Client API available

Demo

> PUT /dm/cache/bordeaux HTTP/1.1> Content-Type:text/plain{"millesime":"2003","description":"so good so good"}< HTTP/1.1 200 OK< X-DirectMemory-SerializeSize: 58< Content-Length: 0

> GET /dm/cache/bordeaux HTTP/1.1> Accept:text/plain< HTTP/1.1 200 OK< Content-Type: text/plain< Content-Length: 51{"millesime":"2003","description":"so good so good"}> DELETE /dm/cache/foo HTTP/1.1

Next Steps

JSR 107?

Benchmarks

Components Integration (Cassandra, Tomcat etc..)

Dynamic cache size modification

Management/Monitoring

Links

http://incubator.apache.org/directmemory/

https://issues.apache.org/jira/browse/DIRECTMEMORY

Demo project: https://bitbucket.org/olamy/heap-off-memory-wtf

Slides: http://www.slideshare.net/olamy

Thanks&&Questions