![Page 1: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/1.jpg)
© 2014 SpringOne 2GX. All rights reserved. Do not distribute without permission.
Efficient Client Communication with Differential Synchronization and JSON Patch
By Brian Cavalier and Craig Walls
![Page 2: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/2.jpg)
What’s the problem with REST?
![Page 3: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/3.jpg)
Nothing, REST is awesome
![Page 4: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/4.jpg)
What’s the problem with how we typically use REST in practice?
![Page 5: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/5.jpg)
Agenda
• Motivation • Differential Synchronization (DS) • JSON Patch • DS + JSON Patch • DS w/Spring and JavaScript • The future
![Page 6: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/6.jpg)
What’s the problem with how we typically use REST in practice?
![Page 7: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/7.jpg)
Isn’t “typical REST” good enough?
• “Typical REST”: 1 request per entity per operation type • create 2 entities, update 3, delete 1 = 6 requests
• Expensive for mobile: connection latency, battery, data ($$$) • Doesn’t exploit return payload
![Page 8: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/8.jpg)
Motivation: Goals
• More efficient data transfer • More efficient use of mobile radios, networks and batteries • Take advantage of WebSocket & messaging • Data synchronization with multiple clients • Offline / disconnected
![Page 9: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/9.jpg)
How can we get data from a Spring back-end all the way to
the pixels more efficiently?
![Page 10: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/10.jpg)
Differential Synchronization
![Page 11: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/11.jpg)
Differential Sync
• Algorithm for syncing N copies of a document • Potentially supports any kind of document/data
• As long as diff & patch algorithms are available • Text, DOM, JSON
• Nice properties • Efficient: Transfer only differences • Symmetrical: same algorithm at each node • Recoverable: disconnected/offline clients, lost messages
• Published by Neil Fraser in 2009 • https://neil.fraser.name/writing/sync/
![Page 12: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/12.jpg)
Differential Sync
doc docshadow
diff
diff
patch
patch
live edits
live edits
![Page 13: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/13.jpg)
Differential Sync
doc shadow docshadow
diff
diff
patch
patch
live edits
live edits
![Page 14: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/14.jpg)
Differential Sync
• Cooperative synchronization loop • Distributed or local • Uses diff & patch algorithms
![Page 15: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/15.jpg)
JSON Patch
![Page 16: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/16.jpg)
JSON Patch
• JSON-formatted Patch for structured documents • RFC 6902, plus related JSON Pointer RFC 6901
• https://tools.ietf.org/html/rfc6902 • https://tools.ietf.org/html/rfc6901
• Suitable for sending via HTTP Patch • Defines operations, format, algorithm, and mime type
• application/json-patch+json • Can coexist w/handlers at same url via Content-Type routing • Does not define diff algorithm • Sensible requirement: patch(diff(a, b), a) === b
![Page 17: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/17.jpg)
JSON Patch
json![{"value":"a"},{"value":"b"},{"value":"c"}]!
+ patch!
[{"op":"add","path":"/3","value":{"value":"d"}},!{“op":"remove","path":"/1"}]!
= new json!
[{"value":"a"},{"value":"c"},{"value":"d"}]
![Page 18: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/18.jpg)
JSON Patch
json!{"name":{"first":"Brian","last":"Cavalier"},"occupation":"JavaScript
Engineer"}!
+ patch!
[{"op":"replace","path":"/occupation","value":"JavaScript Ranger"}]!
= new json!
{"name":{"first":"Brian","last":"Cavalier"},"occupation":"JavaScript Ranger"}
![Page 19: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/19.jpg)
JSON Patch
• Moves the operation type inside request payload • create 2 entities, update 3, delete 1 = 1 request
• Moves the identifier inside request payload • Potentially patch many entity types in a single request
• Patches are atomic • If any part of a patch fails, whole patch must fail (as per RFC) • Think: data integrity
![Page 20: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/20.jpg)
“Typical REST”
POST /todos {“description”: “Try JSON Patch”, “complete”: false} PUT /todos/1 {“description”: “…”, “complete”: true} PATCH /todos/2 {“complete”: true} DELETE /todos/3
![Page 21: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/21.jpg)
JSON Patch
PATCH /todos [ {“op”: “add”, “path”: “-“, “value”: { “description”: “Try JSON Patch”, “complete”: true}}, {“op”: “replace”, “path”: “/1”, “value”: { “description”: “…”, “complete”: true}}, {“op”: “replace”, “path”: “/2/complete”, “value”: true}}, {“op”: “remove”, “path”: “/3”}} ]
![Page 22: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/22.jpg)
“Typical REST”
POST /person { “name”: “Brian” } PUT /todos/1 { “description”: “…”, “complete”: true } PATCH /meetings/2 { “agenda”: “…” } DELETE /todos/3
![Page 23: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/23.jpg)
Pie in the sky JSON Patch
PATCH / [ {“op”: “add”, “path”: “/person/-“, “value”: { “name”: “Brian” }}, {“op”: “replace”, “path”: “/todos/1”, “value”: { “description”: “…”, “complete”: true }}, {“op”: “replace”, “path”: “/meetings/2/agenda”, “value”: “…” }}, {“op”: “remove”, “path”: “/todos/3” }} ]
![Page 24: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/24.jpg)
JSON Patch
• Reduce requests • “Typical” REST:
• # Requests = Entity type x Operation type • JSON Patch + HTTP Patch
• # Requests = 1 • Reduce payload size: Transfer only deltas
![Page 25: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/25.jpg)
Demo
![Page 26: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/26.jpg)
Introducing jiff.js
• JavaScript library for JSON diff & patch • https://github.com/cujojs/jiff
• Diffs arbitrary JSON or JavaScript object graphs • inc. objects containing arrays containing objects containing…
• Patches arbitrary JSON or JavaScript object graphs atomically • Supports advanced features: inverse patches, rebase, contextual
(“smart”) patching
![Page 27: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/27.jpg)
jiff.js
var jiff = require(‘jiff’); var rest = require(‘rest’); !var changedData = jiff.clone(data); !// … data changes via user interactions !var patch = jiff.diff(data, changedData); !rest({ method: ‘PATCH’, entity: patch });
![Page 28: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/28.jpg)
Hmmmm …. Differential synchronization requires diff and patch algorithms
JSON Patch defines a patch format and algorithm
![Page 29: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/29.jpg)
Differential Sync + JSON Patch What if we put these two things together to synchronize
structured data?
![Page 30: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/30.jpg)
Differential Sync
data model shadow data
modelshadow
diff
diff
patch
patch
changeschanges
JSON Patch
JSON Patch
Spring ServerClient (web browser, phone, etc.)
![Page 31: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/31.jpg)
Demo
![Page 32: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/32.jpg)
Differential Synchronization in Server-Side Spring
![Page 33: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/33.jpg)
Applying JSON Patch to the Java-based Domain
![Page 34: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/34.jpg)
Path to SpEL
JSON Patch Path SpEL
/0 [0]
/complete .complete
/1/description [1].description
/clients/3/address/zip .clients[3].address.zip
![Page 35: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/35.jpg)
Challenges with JSON Patch and Java
• How do you “remove” or “move” a property? • How do you “move” a list item to a different index? • How do you “add” a list item to a specific index? • How do you avoid saving the entire list when patching a list? • How do you delete an item as the result of a “delete” op? • What if a “remove” is only intended for a particular view?
![Page 36: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/36.jpg)
Introducing Spring Sync
• GitHub: http://github.com/spring-projects/spring-sync • Maven/Gradle: org.springframework.sync:spring-sync:0.5.0.BUILD-
SNAPSHOT • JsonPatch
• Applies a JSON Patch to a Java object graph
• DiffSync • Applies Differential Synchronization algorithm (leveraging JsonPatch)
• DiffSyncController • Handles PATCH requests for “application/json-patch+json” • Returns a JSON Patch to update client
• @EnableDifferentialSynchronization
![Page 37: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/37.jpg)
Enabling Spring Sync
@Configuration @EnableDifferentialSynchronization public class DiffSyncConfig extends DiffSyncConfigurerAdapter { ! @Autowired private PagingAndSortingRepository<Todo, Long> repo; @Override public void addPersistenceCallbacks( PersistenceCallbackRegistry registry) { ! registry.addPersistenceCallback( new JpaPersistenceCallback<Todo>(repo, Todo.class)); ! } }
![Page 38: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/38.jpg)
Differential Synchronization in JavaScript
![Page 39: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/39.jpg)
Network
Differential Sync in JavaScript
DOM User Interface
JavaScript objects, arrays, etc
Spring Server
patchpatch
patch patch
![Page 40: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/40.jpg)
Differential Sync in JavaScript
• Synchronize from the Spring data model to the pixels • Decouple change frequency from communication frequency:
• fast sync = responsive, but network/resource intense • slow sync = slower UI updates, but less network/resource
intense • Current Status
• Incubator JavaScript implementation based on cujojs/jiff.js
![Page 41: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/41.jpg)
When you have a system based on patches, you can do some interesting things
![Page 42: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/42.jpg)
Streaming changes w/WebSocket
42
![Page 43: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/43.jpg)
Demo
![Page 44: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/44.jpg)
Patch Algebra
44
![Page 45: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/45.jpg)
Patch Algebra
• Inverse patches • think Undo & Redo with no application specific logic
• Merge and Rebase • Apply parallel changes from multiple parties without locking
• jiff.js supports inverse, rebase
![Page 46: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/46.jpg)
Demo
![Page 47: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/47.jpg)
Challenges
• DS in an Entity-oriented world • What constitutes a document?
• Each participant must maintain a shadow copy of the document • Lists and arrays are tricky • Conflict resolution (not a big deal in practice?)
• Hypothesis: Conflicts no more likely to occur than REST • DS in itself does not solve conflict resolution, but neither does
REST
![Page 48: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/48.jpg)
Today
• Spring Sync • http://github.com/spring-projects/spring-sync • org.springframework.sync:spring-sync:0.5.0.BUILD-SNAPSHOT
• JSON Patch + diff in JavaScript • https://github.com/cujojs/jiff
• Experimental Spring support for Differential Sync and JSON Patch over HTTP Patch
• https://github.com/royclarkson/spring-rest-todos
![Page 49: Efficient Client-Server Communication with Differential Synchronization and JSON Patch](https://reader031.vdocuments.mx/reader031/viewer/2022020207/559f78241a28abab5c8b45c9/html5/thumbnails/49.jpg)
The Future
• Continue evolution of Spring Sync and JavaScript DS implementations
• Further Integrate with Spring ecosystem • Messaging & WebSocket • Smart patching
• Offline/disconnected client support • Guidance (when to use it, how to tune it, etc.)