how we built offline data access & sync in loopback
DESCRIPTION
As more and more users are primarily using their mobile devices to access the web, it’s becoming very important for the web apps to provide good experience even when the device is not connected. The common solution is to store data in a browser local storage and synchronize them with the server when there is network access available. In this talk, tell the story of how we added support for offline data access to LoopBack, and explain the technology that makes this possible. A spoiler: we use browserify to run the same model code both on the server and in the browser, so users of the LoopBack framework don't have to do a lot of work to support offline sync for the models they define. There's more to it of course, which is outlined in great detail during the talk. LoopBack is a backend framework built specifically for servicing mobile clients, making it easy to connect your mobile app all the way through to heterogeneous data sources like SQL, NoSQL or even SOAP services.TRANSCRIPT
How we built offline data access & sync in LoopBack
Miroslav Bajtoš
What is LoopBack?
Data access & ORM
● Own querying language inspired by MongoDB
● Connectors for SQL, NoSQL and more
● In-memory database
Todo.find({
where: { completed: false },
limit: 10
}, function(err, list) { /*...*/ });
REST API
Built-in data-access methodsTodo.find => GET /todos?filter=:filter
Todo.create => POST /todos
Todo.findById => GET /todos/:id
(and so on)
Custom methodsTodo.stats => GET /todos/stats
LoopBack on the server
Todomodel
RESTAPI
datasource
Mongo
MySQL
SOAP
The trouble with offline mode
Different tools & APIs
Online● REST/HTTP calls● full power of back-end &
database
GET /todos?where=
{"completed":false}
&limit=10
Offline● local storage/index db● key/value store
JSON.parse(
localStorage.todos
).filter(function(t){
return
!t.completed;
}).slice(0,10);
● What has been changed locally?
● What has changed on the server?
● How to detect conflicts?
Synchronization
Single API for offline & online
● LoopBack in the browser● LocalStorage as a database● REST server as a database
Synchronization
● Change replication between data-sources
The LoopBack recipe
Single API for online & offline
Node.js
Browser
Welcome to the browser
Todomodel
datasource
Localstorage
LoopBack server
Remoting metadata
Todo.find(filter, cb) { … }
Todo.remoteMethod('find', {
accepts: [{ arg: 'filter', type: 'object' }], returns: [{ arg: 'data', type: 'array',
root: true }], http: { verb: 'get', path: '/' }});
Todo.sharedClass.http = { path: '/todos' }
Remoting implementation - server
app.get('/todos', function(req, res, next) { Todo.find(
req.param['filter'], function(err, data){ if (err) next(err)
else res.send(data); }
);
}
Remoting implementation - client
Todo.find = function(filter, cb) { request.get( {
path: '/todos', query: { filter: filter } },
function(err, res) {
if (err) cb(err)
else cb(null, res.body); });
}
The result
One API to rule them all: ● server● online browser● offline browser
Todo.find({
where: { completed: false },
limit: 10
}, function(err, list) { /*...*/ });
Data synchronization
Change tracking
Change tracking record:modelName "Todo"
modelId "42"
revision "SHA of data"
checkpoint 2
● Updated immediately by loopback writes
● Manual update for external writers (scheduled at a regular interval)
The replication algorithm
1. Find local changes
2. Compare remote and local changes,detect conflicts
3. Resolve conflicts
4. Perform a bulk update
5. Create a new checkpoint
Browser
Synchronizing client
localTodo
model
datasource
Localstorage
LoopBack server
change tracking & replication
remoteTodo
model
datasource
DEMO!
LoopBackhttp://loopback.io
Slideshttp://bit.ly/sync-loopback
Get in touchhttp://twitter.com/bajtoshttp://linkedin.com/in/bajtos
THANK YOU!