cbor - the better json
TRANSCRIPT
www.hazelcast.com@noctarius2k
Who’s that dude?• Chris Engelbert• Manager of Developer Relations @Hazelcast
• Java-Passionate (10+ years)
• Performance• Garbage Collection• JVM / Benchmark Fairytales
www.hazelcast.com@noctarius2k
Possible with custom tooling
• unique to this specific use case• involves model classes to be known• utilizes workarounds• generally feels weird :-)
Often:
www.hazelcast.com@noctarius2k
Human Readability, Expressiveness
<fun> <with> <xml dont:you="think" /> </with> </fun>
www.hazelcast.com@noctarius2k
Human Readability, Expressiveness
This is not a flame graph!
<plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-failsafe-plugin</artifactId> <version>${maven.failsafe.plugin.version}</version> <configuration combine.self="override"> <redirectTestOutputToFile>true</redirectTestOutputToFile> <argLine> ${jacoco.agent.argLine} -Xms129m -Xmx1G -XX:MaxPermSize=129M -Dxyz.feature1.enabled=false -Dxyz.feature2.enabled=false -Dxyz.feature3.type=none -Dxyz.feature4=true </argLine> <includes> <include>**/TestCast*.java</include> <include>**/IntegrationTestCase*.java</include> </includes> <groups> com.xyz.test.annotation.QuickTest, com.xyz.test.annotation.SlowTest, com.xyz.test.annotation.NightlyTest </groups> </configuration> <executions> <execution> <goals> <goal>integration-test</goal> </goals> </execution> </executions> </plugin>
www.hazelcast.com@noctarius2k
Human Readability, Concise
Does not play “well”with binary data :-(
base64 encoded image
www.hazelcast.com@noctarius2k
Features• Stores JSON-alike content• Extremely concise• Binary• Streamable• Type safe• Schemaless• Highly Extensible• Well defined specification• Standard (by the IETF, RFC7049)
www.hazelcast.com@noctarius2k
Real World (Size) Example
www.pathway-game.com
• Stores Geometry Information• Mainly Float Data• Often floatToIntBits for Speed• Lots of Sequences• Only a few Dictionaries
Really simple data structure
www.hazelcast.com@noctarius2k
Real World (Size) Example
2250000
4500000
6750000
9000000
XML JSON BSON CBOR MSGPACK
MSGPACK CBORBSONJSONXML
1.408.737 bytes1.415.215 bytes5.575.783 bytes5.755.188 bytes8.592.541 bytes
1.0x1.0x3.9x4.0x6.0x
www.hazelcast.com@noctarius2k
Debugging capability built-inDictionary [ ByteString{ ANI0 }=Sequence [ Sequence [ UInt{ 7264 }, UInt{ 4 }, UInt{ 2724 }, UInt{ 37 }, Sequence [ NInt{ -1053818880 }, UInt{ 1077214225 }, UInt{ 0 }, UInt{ 0 }, UInt{ 1068825617 }, ] ] ] ]
Since items carry their type information,writing a quick debugger or using theprovided to-string capabilities of libs
www.hazelcast.com@noctarius2k
Implementations
• JavaScript• browser• node.js
• Julia• C#• Java• Lua• PHP• Python• Go
• C• C++• Rust• Perl• Ruby• D• Swift• Erlang• Elixir
www.hazelcast.com@noctarius2k
Implementations
• JavaScript• browser• node.js
• Julia• C#• Java• Lua• PHP• Python• Go
• C• C++• Rust• Perl• Ruby• D• Swift• Erlang• Elixir
CBOR-java:https://github.com/peteroupc/CBOR-Java
Jackson Dataformat:https://github.com/FasterXML/jackson-dataformat-cbor
cbor-java:https://github.com/c-rack/cbor-java
JACOB:https://github.com/jawi/jacob
borabora:https://github.com/noctarius/borabora
www.hazelcast.com@noctarius2k
Implementations
• JavaScript• browser• node.js
• Julia• C#• Java• Lua• PHP• Python• Go
• C• C++• Rust• Perl• Ruby• D• Swift• Erlang• Elixir
CBOR-java:https://github.com/peteroupc/CBOR-Java
Jackson Dataformat:https://github.com/FasterXML/jackson-dataformat-cbor
cbor-java:https://github.com/c-rack/cbor-java
JACOB:https://github.com/jawi/jacob
borabora:https://github.com/noctarius/borabora
www.hazelcast.com@noctarius2k
borabora
•Skip-scan parser•Simple, fluent API•Supports object queries•Reading + writing•Supports mutating (hacky prototype)•Extended datatype support
www.hazelcast.com@noctarius2k
borabora
•Skip-scan parser•Simple, fluent API•Supports object queries•Reading + writing•Supports mutating (hacky prototype)•Extended datatype support
Not just a blur of color
www.hazelcast.com@noctarius2k
borabora
•Skip-scan parser•Simple, fluent API•Supports object queries•Reading + writing•Supports mutating (hacky prototype)•Extended datatype support
but the island ;-)
www.hazelcast.com@noctarius2k
Comparisons
“Alternative Facts”
in the comparisons, are all fake news!
www.hazelcast.com@noctarius2k
MessagePack• Extremely concise• Available in almost every language• UTF-8 and binary strings• No map keys datatype restriction• Schemaless• Binary
• 32bit size limitation for certain data types• Specification is concise too (~450 lines)• No official standard
www.hazelcast.com@noctarius2k
MessagePack• Extremely concise• Available in almost every language• UTF-8 and binary strings• No map keys datatype restriction• Schemaless• Binary
• 32bit size limitation for certain data types• Specification is concise too (~450 lines)• No official standard
1:02:0DrawDrawDrawDraw
2:12:22:3
www.hazelcast.com@noctarius2k
JSON• Human readable• Official standard (ECMA-404)• Subset of JavaScript (strong support)• Schemaless• Kind of type safe
• Human readable• Text-based• Map keys can be strings only• Large bytesize
www.hazelcast.com@noctarius2k
• Human readable• Official standard (ECMA-404)• Subset of JavaScript (strong support)• Schemaless• Kind of type safe
• Human readable• Text-based• Map keys can be strings only• Large bytesize
1:0Draw2:0DrawDraw
2:12:22:32:4
JSON
www.hazelcast.com@noctarius2k
• Binary• Available in a lot of languages• Derived from JSON, interoperability 1+• Schemaless• Type safe• Supports in-place updates
• No official standard• Extremely concise specification• Map keys can be strings only• Large bytesize
BSON
www.hazelcast.com@noctarius2k
• Binary• Available in a lot of languages• Derived from JSON, interoperability 1+• Schemaless• Type safe• Supports in-place updates
• No official standard• Extremely concise specification• Map keys can be strings only• Large bytesize
Draw1:02:0DrawDraw3:0
3:13:23:33:4
BSON
www.hazelcast.com@noctarius2k
• Human readable• Official standard (W3C REC-xml)• Strong commercial support (e.g. IBM)• Supported everywhere
• Type safe or schemaless• Human readable• Text-based• Very verbose• Large bytesize
XML
www.hazelcast.com@noctarius2k
• Human readable• Official standard (W3C REC-xml)• Strong commercial support (e.g. IBM)• Supported everywhere
• Type safe or schemaless• Human readable• Text-based• Very verbose• Large bytesize
1:0Draw2:03:0
3:13:23:33:43:5
XML
www.hazelcast.com@noctarius2k
More alternativesText-based:
• bencode• YAML• …
Binary:• Amazon ION• Smile• Apache Thrift• Apache Avro• Protobuf• …
ASCII based encoding, unaffected by endianess, type safecommonly for configuration, superset of JSON, type safe
skip-scan parsable, type safe, commercial support (Amazon)based on JSON, property-name back-referencesDSL, schema-bound, developed by Facebook, RPC servicesDSL, schema-bound, primarily used by HadoopDSL, schema-bound, developed by Google, RPC services
www.hazelcast.com@noctarius2k
More information:• http://cbor.io/• http://bsonspec.org/• http://www.json.org/• https://www.w3.org/TR/REC-xml/• http://msgpack.org/ • https://github.com/noctarius/borabora
• https://en.wikipedia.org/wiki/Bencode• https://en.wikipedia.org/wiki/YAML• https://amznlabs.github.io/ion-docs/• https://en.wikipedia.org/wiki/Apache_Thrift• https://en.wikipedia.org/wiki/Apache_Avro• https://en.wikipedia.org/wiki/Protocol_Buffers• https://en.wikipedia.org/wiki/Smile_(data_interchange_format)
Thank You!