Tour de Jacksonfrom Basics to Forgotten Features

Tour De JacksonJackson is quickly becoming the de facto standard tool

for processing JSON on Java platform, due to its extensive feature set and high configurability.

But learning all the available power can be challenging. This talk tries to shine light on some of

lesser-known features of Jackson.

About AuthorTatu Saloranta aka @cowtowncoder● Programming since 1983 (VIC-20 FTW!)● for Public Benefit since ~1988 (Amiga

SoundTracker Packer)● Open source since 1998 (Java VT-100 emu)● for Money, too; most recently:

○ Sun (2001-2005)○ Amazon (2005-2009)○ Ning (2009-2012)○ Salesforce (2012-)

About Author, Open SourceNotable Open Source events:● Java UUID Generator (JUG): 2002● Woodstox XML processor (Stax): 2004● StaxMate, helper for Stax: 2004● Jackson JSON processor: 2007● Aalto XML parser (non-blocking): 2008● @Ning: LZF-compress, tr13, jvm-

compressor-benchmark: 2010● Other: ClassMate, CacheMate, java-merge-

sort

Jackson BioJackson JSON (*) processing library● Born in 2007 -- initially (0.5) skimpy

streaming parser, generator● Data-binding added in 2008● New versions every 3 - 6 months● 3 core components (annotations, streaming,

databind), ~20 extension modules● of many, many Java JSON libs

○ Most popular: 10,000/day via Maven Central○ Fastest (google or ask for links)

● (*) ... and not just JSON any more...

Jackson Basic FeaturesSince beginning of time (Jackson 1.0)● Read JSON

○ as token stream (streaming) -- JsonParser○ into POJOs (databind) -- ObjectMapper/-Reader○ as JSON Trees (tree model) -- JsonNode

● Write JSON○ from POJOs -- ObjectMapper/-Writer○ from Trees○ or directly using token stream -- JsonGenerator

● POJO/Tree access built on top of streaming model (layering)

Jackson Basic FeaturesAdditions over time● More configurable data-binding (annotate)● Convert values, update (merge) POJOs

○ shortcut for "write-X-as-JSON, read-Y-from-JSON" == "x -> y"

○ Base64 for binary● Modular extensibility (modules)

○ support for 3rd party data types, formats● JSON Schema support (mostly, generation)● Convenient conversions between models:

tree <-> POJO <-> Streaming

Jackson Advanced FeaturesObject Serialization requires handling:1. Polymorphism (@JsonTypeInfo, "default

typing"), type metadata handlinga. Type id: class name or custom type nameb. Inclusion: as-property, wrapper-object, wrapper-

array, external-propertyc. Resolution mechanism (type name to/from class)

2. Object Identity (@JsonIdentityInfo)a. Id generators (sequence, UUID, from-property)b. Property name (no other inclusion methods)

Extensibility via ModulesDatatypes

● 3rd-party libs: Joda, Guava, org.json● frameworks: Hibernate (lazy-loading)● JVM language-specific types: Scala

Data formats● Smile (binary JSON) -- more on this later● XML -- move over JAXB!● CSV● YAML, BSON (mongoDB, external), Avro● future: Protobuf, Thrift?

Extensibility via ModulesJAX-RS (Jersey, RESTeasy, CXF), DropWizard

● Providers (MessageBodyReader/-Writer)○ JSON○ Smile (binary JSON)○ XML○ ... add others (CSV?) as requested

Other● Mr Bean (more later on)● Afterburner: Mach3 with bytecode gen● JAXB annotation support● JSON Schema generator (Jackson 2.2)

Forgotten Features: overview1. Mr Bean: "abstract type materialization"2. Value conversions3. Smile! (binary JSON)4. Mix-in Annotations5. ObjectReader / ObjectWriter6. ObjectReader.readValues()7. @JsonValue8. @JsonUnwrapped9. Dyna-beans (@JsonAnySetter/-Getter)

10. Lots more (misc other) -- just pointers

Forgotten Feature: Mr BeanEasy to write monkey code, but it's a chore… so why do we keep doing it this? ->

How about we just define:

public interface Name { public String getFirst(); public String getLast();}

and use it like: Name n=m.readValue(json);

public class NameImpl{ private String first, last;

public Name() { }

public void setFirst(String f){first=f; } public void setLast(String l){last=l;}

public String getFirst(){return first;} public String getLast(){return last;}}

Forgotten Feature: Mr BeanAnd so we should! ObjectMapper mapper = new ObjectMapper(); mapper.registerModule(new MrBeanModule()); Name n = mapper.readValue(in, Name.class);Brilliant! But how?

1. Take interface/abstract class, discover properties2. Implement getters AND setters (ASM)3. Define new class4. Rest works based on auto-discovery!

Works for generic and transitive types too.Can even be used outside of Jackson...

Forgotten Feature: Value ConversionsConvert any structurally compatible types: List<String> keys = ...; String[] keyArray = mapper.convertValue( keys, String[].class); Map<String,Integer> map = mapper.convertValue( pojo, new TypeReference<String,Integer>(){});

or compatible scalar types: Date date = mapper.convertValue( "2013-04-19T23:09Z", Date.class);

or even binary data with Base64! byte[] data = mapper.convertValue( "TWFuIGlzIGRpc3", byte[].class);

Forgotten Feature: Smile!● Virtues of binary formats (protobuf, Thrift,

MsgPack, Avro, BSON) often overrated -- esp. speed; compression for data size

● Still: binary typically more compact, MAY be faster to process

● But is it necessary to give up all benefits of JSON – self-descriptiveness, open content (no mandatory schema), extensibility, interoperability – to get there?

Forgotten Feature: Smile!● What if there was “binary JSON” format that

was:○ 100% compatible (same model, types) with JSON

(bit like "Fast Infoset" for XML)○ More compact than JSON○ Strictly faster to process – fast writes, not just reads○ Auto-detectable, so with proper parser/generator

can be used “just like JSON” from app perspective● Wouldn't this be useful?● http://wiki.fasterxml.com/SmileFormatSpec

defines such format: "Smile"

Forgotten Feature: Smile!● Simple usage: extends JsonFactory● ObjectMapper m = new ObjectMapper

(new SmileFactory());● … and it “Just Works”● Use cases:

○ “Big Data” – logging, Hadoop data○ Search indexing -- used by Elastic Search○ Internal (for external JSON may be better)

● Efficient transcoding to/from JSON – best of both!

● Public tests show efficiency (jvm-serializers)

Forgotten Feature: mix-in annotations● Databinding highly configurable with

Annotations, but you can not always modify value classes (or don't want to)

● How to externalize such configuration?● What if you could "mix in" annotations, from

outside?● Due to the way Java handles Annotations,

always associated with class/interface● Mix-in annotations: define "mix-in"

class/interface, add annotations to target

Forgotten Feature: mix-in annotationspublic class Rectangle { private int w, h; public Rectangle(int w, int h) { this.w = w; this.h = h; } public int getW() { return w; } public int getH() { return h; } public int getSize() { return w * h; }}

● Property names "w","h"?● Shouldn't serialize "size"● How to construct?● Mix-ins for the win!

abstract class RectangleMixIn extends Rectangle // optional{ @JsonCreator // important! MixIn(@JsonProperty("width") int w w, @JsonProperty("height") int h) {}

@JsonProperty("width") abstract int getW(); // rename @JsonProperty("height") abstract int getH(); // rename @JsonIgnore abstract int getSize(); // remove}

Forgotten Feature: mix-in annotationspublic class MyModule extends SimpleModule{ public MyModule() { super("ModuleName", Version.unknownVersion()); } @Override public void setupModule(SetupContext context) { // Mix-ins so that annotations from Target context.setMixInAnnotations(Rectangle.class, RectangleMixIn.class); }}

Forgotten Feature: ObjectReader, ObjectWriterMost code uses ObjectMapper, which works fine but:

● Can not be reconfigured (config-then-use)● Does not contain all functionality

Instead, be Jackson Pro, and use:1. ObjectReader for deserialization2. ObjectWriter for serialization

so that you can:● define per-call JSON View, Features● use new functionality (readValues())

Forgotten Feature: ObjectReader, ObjectWriterTo serialize using JSON View, indented: mapper.writerWithView(View1.class) .withDefaultPrettyPrinter() .writeValue(pojo, outputStream);or read instance of certain type, view: MyType value = mapper.reader(MyType.class) .withView(View2.class) .readValue(source);

Forgotten Feature: ObjectReader.readValues()● Large data sets often sequences of objects:

{ "x":1,"y":3 }{ "x":5,"y":2 }...

● Can read using JsonParser + ObjectMapper● But there is a simpler way:

ObjectReader r = mapper.reader(Point.class);MappingIterator<Point> it = r.readValues(in);while (it.hasNextValue()) { Point p = it.nextValue(); // ... process}

Forgotten: @JsonValueProvide your own String representation: public class MyStuff { @JsonValue public String toString() { } }

or anything Jackson knows how to serialize: @JsonValue public Map<String,Object> toJson() { }

Perfect match for custom constructors: @JsonCreator public MyStuff(Map<String,Object> map) {}

Forgotten: @JsonUnwrapped● Used to "flatten" POJOs● Can add prefix/suffix public class Box{ @JsonUnwrapped(prefix="topLeft") Point tl; @JsonUnwrapped(prefix ="bottomRight") Point br;}public class Point { int x, y;}

● would produce/consume JSON like:

{ "topLeft.x" : 0, "topLeft.y" : 0, "bottomRight.x" : 100, "bottomRight.y" : 80 }● instead of { "tl": { "x":0, "y":o }, "br" : { "x":100, "y":80 } }

Forgotten: Dyna-beans● Sometimes want to handle set of additional "extra"

properties (key/value pairs)

public class NamedBeanWithStuff{ protected final String name; // mandatory property // and then dynamic properties protected Map<String,Object> other = new HashMap<String,Object>();

@JsonCreator public NamedBeanWithStuff@JsonProperty("name") String name) { this.name = name; } public String getName() { return name; }

Forgotten Feature: Dyna-beanspublic class NamedBeanWithStuff// continued... public Object get(String name) { // non-Bean accessor return other.get(name); } // "any getter" needed for writing out dynamic props @JsonAnyGetter public Map<String,Object> any() { return other; } // "any setter" for reading JSON, adding in bean @JsonAnySetter public void set(String name, Object value) { other.put(name, value); }}

● Can be used for storing unknown properties

Forgotten Feature: OtherObjectReader.updateValue()

● Update existing (root) value● Shallow merge● Good for configuration overrides, default

valuesData-format auto-detection (alas, too long for example!)

● Detectable formats: JSON, Smile, XML, YAML (maybe CSV in future)

● Feed an InputStream, get parser/reader

Forgotten Feature: Other@JacksonInject: injecting values from context

● Indicate logical name of value to inject with annotation

● Provide mapping from name to value, per deserialization call

● Can provide value defaultingSupport for "Builder" pattern

● @JsonDeserialize(builder=Builder.class)● by default, "withX" methods (overridable)

That's All Folks!Questions? Comments?

Follow me (@cowtowncoder), or email (tsaloranta@gmail.com / tatu@fasterxml.com)

ps. FasterXML (http://fasterxml.com) -- the Benevolent Corporate Entity behind Jackson et al -- offers support, training, for Jackson and friends (Woodstox, Aalto)