![Page 1: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/1.jpg)
Microservices: Terabytes in Microseconds
Per Minborg CTO Speedment, Inc.
Malin WeissCMO Speedment, Inc.
![Page 2: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/2.jpg)
About Us
![Page 3: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/3.jpg)
Microservices
• Microservices bring together best practices from a variety of areas
• Most likely you are already using some of these best practices
![Page 4: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/4.jpg)
Microservices
• It sounds like marketing hype• It all sounds pretty familiar• It just a rebranding of stuff we already do• There is room for improvement in what we do• There are some tools, and ideas we could apply to our
systems without changing too much
![Page 5: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/5.jpg)
• Simple component based design• Asynchronous messaging• Automated, dynamic deployment of services• Service private data sets• Transparent messaging• Teams can develop independently based on well defined
contracts
Microservices
![Page 6: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/6.jpg)
Terabytes in Microseconds Solution
Latency
Data volumeGB TB EB
μs
ms
s
min
hour
days
![Page 7: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/7.jpg)
Speedment Chronicle
Terabytes in Microseconds Solution
Maps the data in a key-value store
Synchronizes SQL Data into an in-JVM-memory data store
Maps the data in Column key stores
Provides application APIs
Terabytes
1. Synchronization
2. Key Value Store
3. Column Key Stores
4. API
Microservice 3Microservice 2
Microservice
![Page 8: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/8.jpg)
1. Synchronization
• Synchronizes SQL Data with the in-JVM-memory solution:• Initially• Over time
• Periodically• Reactively
![Page 9: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/9.jpg)
1. Synchronization
Periodically•Dumps are reloaded periodically •All data elements are reloaded•Data remains unchanged between reloads•System restart is just a reload•MVCC so that several dumps can be active at the same time
Reactively•Changed data is captured in the Database•Changed data events are pushed into memory•Events are grouped in transactions•Cache updates are persisted•Data changes all the time•System restart, replay the missed events
or
![Page 10: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/10.jpg)
1. Synchronization
*
Periodically Reactively Poll
Max Data Age Dump period Replication Latency- ms Eviction time
Lookup Performance Consistently Instant Consistently Instant 20% very slow
Consistency Eventually Consistent Eventually Consistent Inconsistent – stale data
Database Cache Update Load
~ Total Size / Dump Period ~ Rate of Change ~ Cache size * Eviction time
Restart Complete Reload Down time*update rate ->10% of down time
Heat-up time
![Page 11: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/11.jpg)
1. Synchronization Properties
• Detect changes in a database • Buffer the changes• Can replay the changes later on• Preserve order• Preserve transactions• See data as it was persisted• Detect changes from any source
JVM
![Page 12: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/12.jpg)
1. Synchronization
How is data synchronized?•Periodic MVCC reload•Transaction log harvesting•Event Sourcing and CQRS
![Page 13: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/13.jpg)
2. Key Value Store
Makes it possible to• Having Maps that can exceed the server’s RAM by up to 40x• Opening up the path to mammoth JVMs with tens of terabytes• Data persistence inside JVM provides fast restart• When you have multiple instances of a service they can share the data
with a single copy in memory
![Page 14: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/14.jpg)
How Much Data Can We Have?
• JVM (32), 32-bit (2-3 GB)• Java 7 (64), –XX:+UseCompressedOops, 35 bit (32 GB)• Java 8 (64), –XX:ObjectAlignmentInBytes=16, 36 bit (64
GB)• JVM (64), 64 bit references, 100 GB+ (G1 collector, GC concerns) • NUMA Regions (40 bits or 46 bits), GC over NUMA borders is a
problem• Virtual Address Space (48-bit)• Memory Mapped Files (48+ bits)• Peta Byte JVMs (50+ bits) with library support
![Page 15: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/15.jpg)
Virtual Address Space (48-bit)
![Page 16: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/16.jpg)
Peta Byte JVMs (50+ bits)
![Page 17: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/17.jpg)
3. Column Key Store
• Column Key Stores are used much like database indexes• The Column Key Stores can produce data lazily• Can be mapped onto memory mapped files
![Page 18: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/18.jpg)
3. Column Key Store
• Features over ConcurrentNavigableMap• Handles null values• Can handle several values for one key• Can serialize data off-heap• Can map onto files• Can compress its keys• Contains a bi-directional skip dictionary• Remains performant for billions and trillions of entries• Supports transactions
![Page 19: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/19.jpg)
3. Column Key Store
FilesystemPage cache
User Space Kernel Space
Physical memory
Disk blocks
SSD
Page mapping
Microservice JVM
mappedmapped bufferbuffer filesystem pagesfilesystem pages
memory pagesmemory pages
![Page 20: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/20.jpg)
3. Column Key Store
FilesystemPage cache
User Space Kernel Space
Physical memory
Disk blocks
SSD
Page mappingfilesystem pagesfilesystem pages
memory pagesmemory pages
Microservice JVM 2
mappedmapped bufferbuffer
Microservice JVM 1
mappedmapped bufferbuffer
![Page 21: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/21.jpg)
3. Column Key Store
• O(1) operations:• equals, notEquals• isNull, isNotNull• in, notIn• sort
• O(log N) operations• <, <=, >, >=• between• startsWith(String)
• O(N)• matches(regexp)
![Page 22: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/22.jpg)
3. Column Key Store
![Page 23: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/23.jpg)
3. Column Key Store
Key-ValueStore
IDColumn Key
Store
User.AGE.greaterThan(42)
AGEColumn Key
Store
NAMEColumn Key
Store
greaterThan(42) Keys Values
User
![Page 24: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/24.jpg)
3. Column Key Store - The Stream Pipeline
Key-ValueStore
AGEColumn
Key Store
greaterThan(42) Keys Values
![Page 25: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/25.jpg)
3. Column Key Store - Parallelism
Key-ValueStore
AGEColumn Key
Store
greaterThan(42)
ParallelKeys
ParallelValues
OptionalParallel Strategy
![Page 26: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/26.jpg)
4. API
• Provides application APIs• Java 8 Stream API• REST API
![Page 27: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/27.jpg)
4. API• Generates Java code
automatically from SQL metadata• Free from hand coding errors • Reduces development time• Improves the application quality• Reduces maintenance for
applications• Uses standard streams for
querying• Custom code possibilities
![Page 28: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/28.jpg)
4. API
Java 8 Stream API
REST API
users.stream() .filter(AGE.greaterThen(42)) .forEach(someOperation());
https://mysite/user/?filter=[{"property":”age",”operator":”gt","value":42}]
![Page 29: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/29.jpg)
Querying In-Memory Data Using Streams
Map<Long, User> keyValueStore.values()
AGE.greaterThan(42)
forEach()
Source
Filter
Term.
Pipeline
users.stream() .filter(AGE.greaterThen(42)) .forEach(someOperation());
![Page 30: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/30.jpg)
Querying In-Memory Data Using Streams
AGE ColumnStore<Integer, Long>.greaterThan(42)
AGE.greaterThan(42)
forEach()
Source
Filter
Term.
Pipeline
keyValueStore::getMap
![Page 31: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/31.jpg)
Querying In-Memory Data Using Streams
AGE ColumnStore<Long, User>.greaterThan(42)
forEach()
Source
Term.
Pipeline
keyValueStore::getMap
![Page 32: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/32.jpg)
4. APIJava 8 Stream API
REST API
users.stream() .filter(AGE.greaterThen(42)) .skip(10) .limit(100) .sorted(NAME.comparator()) .forEach(someOperation());
https://mysite/user/?filter=[{"property":”age", ”operator":”gt","value":42}]&skip=10&limit=100&sort=[{"property":”name", ”direction":”ASC”}]
![Page 33: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/33.jpg)
4. API - Parallelism
• Java 8 Stream API
users.stream() .parallel() .filter(AGE.greaterThen(42)) .flatMap(users::findLogins) .forEach(expensiveOperation());
![Page 34: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/34.jpg)
Demo 1: Equality
• 300,000 Users • Off Heap Data Store and Key-Value map• Test
• Find all users with id = 42• Count them• Add the count to a sum• Iterate 1,000,000 times• JVM Warmup and then Benchmark• Sum will be 1,000,000
• A standard MySQL database will process 2,000 TPS on a 4 CPU laptop like mine
![Page 35: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/35.jpg)
Demo 2: Mixed Predicates
• 1,000,000 Users • Tests
• ID.equal(42)• ID.greaterOrEqual(42)• ID.in(42, 43, 44)• ID.between(42, 45)
![Page 36: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/36.jpg)
Demo 3: Parallelism
• 300,000 Users • 4 threads • Tests
• ID.equal(42)• ID.greaterOrEqual(42)• ID.in(42, 43, 44)• ID.between(42, 45)
![Page 37: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/37.jpg)
Compare Latencies Using the Speed of Light
During the time a database makes a 1 s query, how far will the light move?
Database CPU L1 cache
Conclusion : Do not place your data on the moon, keep it close by using in-JVM-memory technology!
![Page 38: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/38.jpg)
Epic Threshold for Mankind Passed in 2016
Conclusion: Quit drinking coffeeand buy more RAM for your application server instead!
![Page 39: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/39.jpg)
Microservices Solution
MS1
Speedment Chronicle
MS1 MS1 MS2
Speedment Chronicle
MS2 MS2 MS3
Speedment Chronicle
MS3 MS3
![Page 40: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/40.jpg)
Microservices JVM
• The solution maps Java NavigableMaps++ to persistent stores (files)
• The views of the maps are available for any JVM that has access to these files
• Microservice JVMs can be started or restarted very rapidly • Access to these maps will be gained in millisecond time• These maps can be shared instantly between several
microservice JVMs• New microservice instances are added, removed, or restarted
very quickly
![Page 41: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/41.jpg)
If You Only Remember One Thing….
Latency
Data volumeGB TB EB
μs
ms
s
min
hour
days
![Page 42: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/42.jpg)
Thank you!
Per Minborg
Malin Weiss
www.speedment.comwww.speedment.org
![Page 43: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/43.jpg)
Demo 1, Tool Work Flow
• Connect to an existing database• Extract the domain model• Generate code
![Page 44: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/44.jpg)
32 bit Operating System (31-bit heap)
![Page 45: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/45.jpg)
Compress Oops in Java 7 (35-bit)
• Using the default of
–XX:+UseCompressedOops
• In a 64-bit JVM, it can use “compressed” memory references
• This allows the heap to be up to 32 GB without the overhead of 64-bit object references. The Oracle/OpenJDK JVM still uses 64-bit class references by default
• As all object must be 8-byte aligned, the lower 3 bits of the address are always 000 and don’t need to be stored. This allows the heap to reference 4 billion * 8-bytes or 32 GB
• Uses 32-bit references
![Page 46: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/46.jpg)
Compressed Oops with 8 Byte Alignment
![Page 47: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/47.jpg)
Compress Oops in Java 8 (36 bits)
• Using the default of –XX:+UseCompressedOops –XX:ObjectAlignmentInBytes=16
• In a 64-bit JVM, it can use “compressed” memory references
• This allows the heap to be up to 64 GB without the overhead of 64-bit object references. The Oracle/OpenJDK JVM still uses 64-bit class references by default
• As all object must be 8 or 16-byte aligned, the lower 3 or 4 bits of the address are always zeros and don’t need to be stored. This allows the heap to reference 4 billion * 16-bytes or 64 GB
• Uses 32-bit references
![Page 48: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/48.jpg)
64-bit References in Java (100+ GB)
• A small but significant overhead on main memory use
• Reduces the efficiency of CPU caches as less objects can fit in
• Can address up to the limit of the free memory. Limited to main memory
• GC pauses become a real concern and can take tens of second or many minutes
• For larger heap sizes, using the G1 collector may be a good choice.
![Page 49: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/49.jpg)
NUMA Regions (~40 bits)
• Large machine are limited in how large a single bank of memory can be. This varies based on the architecture
• Ivy and Sandy bridge Xeon processors are limited to addressing 40 bits of real memory
• In Haswell and Boradwell this has been lifted to 46-bits
• Each Socket has “local” access to a bank of memory, however to access other bank it may need to use a bus. This is much slower
• The GC of a JVM can perform very poorly if it doesn’t sit within one NUMA region. Ideally you want a JVM to use just one NUMA region
![Page 50: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/50.jpg)
NUMA Regions (~40 bits)
![Page 51: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/51.jpg)
Virtual Address Space (48-bit)
![Page 52: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/52.jpg)
Memory Mapped Files (48+ bits)
• Memory mappings are not limited to main memory size
• 64-bit OS support 128 TiB to 256 TiB virtual memory at once
• For larger data sizes, memory mapping need to be managed and cached manually
• Can be shared between processes
• A library can hide the 48-bit limitation by caching memory mapping
![Page 53: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/53.jpg)
Peta Byte JVMs (50+ bits)
• If you are receiving 1 GB/s down a 10 Gig-E line in two weeks you will have received over 1 PB
• Managing this much data in large servers is more complex than your standard JVM
• Replication is critical. Large complex systems, are more likely to fail and take longer to recover
• You can have systems which cannot be recovered in the normal way. i.e. Unless you recover faster than new data is added, you will never catch up
![Page 54: NYJavaSIG - Big Data Microservices w/ Speedment](https://reader033.vdocuments.mx/reader033/viewer/2022051523/5882eb1e1a28ab3f1e8b469d/html5/thumbnails/54.jpg)
Peta Byte JVMs (50+ bits)