html5 javascript storage for structured data
DESCRIPTION
HTML5 JavaScript Storage for Structured Data. Andy Gup, Esri www.andygup.net @ agup. IndexedDB and the mobile web. Why attend this session?. Have storage needs > 5MBs Want to store data types other than Strings Don’t want to manually serialize / deserialize - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/1.jpg)
HTML5 JavaScript Storage forStructured Data
Andy Gup, Esriwww.andygup.net
@agup
![Page 2: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/2.jpg)
IndexedDBand the
mobile web
![Page 3: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/3.jpg)
Why attend this session?
Have storage needs > 5MBs
Want to store data types other than Strings
Don’t want to manually serialize/deserialize
Looking into offline JavaScript
![Page 4: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/4.jpg)
Agenda
Intro to IndexedDB coding patterns
Fitting IndexedDB into overall application
Performance
Wet your appetite! Can’t cover it all
![Page 5: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/5.jpg)
Who am I?Andy Gup, Esri Sr. Developer – JS and native Android
www.andygup.netgithub.com/andygup [email protected]@agup
![Page 6: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/6.jpg)
Structured data?
JSON Objects (not serialized)
Complex Objects (difficult serialize/deserialize)
Binary data (e.g. images, files)
Arrays
![Page 7: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/7.jpg)
Offline JavaScript Demo
![Page 8: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/8.jpg)
IndexedDB browser support?
Source: Caniuse.com
![Page 9: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/9.jpg)
IndexedDB browser support?
Source: Caniuse.com
![Page 10: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/10.jpg)
How does IndexedDB work?
Key-Value pairsSearch IndexesNoSQL CursorsAsynchronous via callbacksNotifications via DOM events
![Page 11: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/11.jpg)
Key/Value pairs?
KeyString, Number, Date, Array
Value (partial list)StringObjectArrayBlobArrayBufferUint8ArrayFile
![Page 12: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/12.jpg)
Will IndexedDB work offline?
YES!Can also be used with Application Cache.
![Page 13: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/13.jpg)
How to use? Six basic steps
1. Include shim (if necessary)2. Set vendor prefixes3. Validate functionality4. Open (or upgrade)5. Add, update, retrieve or delete data6. Capture events via callbacks
![Page 14: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/14.jpg)
How to use? IndexedDBShim.js
Required for Safari 7 on iOS and MacSafari 7 only comes with Web SQL
<script src=“IndexedDBShim.js"></script>
![Page 15: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/15.jpg)
How to use? Set Prefixes
window.indexedDB = window.indexedDB || window.mozIndexedDB ||
window.webkitIndexedDB || window.msIndexedDB;
var transaction = window.IDBTransaction || window.webkitIDBTransaction;
![Page 16: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/16.jpg)
How to use? Validate functionality
this.isSupported = function(){
if(!window.indexedDB && !window.openDatabase){ return false; }
return true;};
![Page 17: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/17.jpg)
How to use? Validate functionalitythis.isSupported = function(callback){
if(!window.indexedDB && !window.openDatabase){ return callback(false); } else{
testFunctionality(function(success,result){ return callback(success,result); });
}};
![Page 18: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/18.jpg)
How to use? Validate functionalityfunction testFunctionality(callback){ . . . var objectStore = transaction.objectStore("table1"); var request = objectStore.put(myBlob,”test1key”); request.onsuccess = function(event) { callback(true,event.target.result); };
request.onerror = function(error) { callback(false,error); };}
![Page 19: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/19.jpg)
How to use? Open databasevar db = null;
var request = indexedDB.open(“customerDB”, /*version*/ 2);
request.onsuccess = function(event){db = event.target.result;
callback(true); }
request.onerror = function(error){callback(false,error);
}
STEP 1
STEP 2
![Page 20: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/20.jpg)
How to use? Transactions
Handles ALL reading and writing
var transaction = db.transaction(["customerDB"],
"readwrite");
Provides eventsThree modes
“readonly”“readwrite”“versionchange”
![Page 21: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/21.jpg)
How to use? Transaction requestsvar transaction =
db.transaction(["customerDB"], "readwrite");
var objectStore = transaction.objectStore(“customerDB”);
var request = objectStore.put({test:1},”myKey”);
// Did transaction request work or not?request.onerror = function(error){ . . . }request.onsuccess = function(event){ . . . }
STEP 1
STEP 2
STEP 3
STEP 4
![Page 22: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/22.jpg)
How to use? Write data
add(anyValue, /*optional*/ key) Write only unique valuesDuplicate entries fail with error
put(anyValue, /*optional*/ key)Overwrites existing entries with same key
![Page 23: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/23.jpg)
How to use? Write data
request = objectStore.put({test:1},”myKey”);
request.onsuccess = function(event){callback(true,event.target.result);
}
request.onerror = function(error){callback(false,error);
}
![Page 24: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/24.jpg)
How to use? Get data
request = objectStore.get(“aUniqueKey”);
request.onsuccess = function(event){callback(true,event.target.result);
}
request.onerror = function(error){callback(false,error);
}
![Page 25: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/25.jpg)
Read/Write performance
Know thy data!
Not all data types perform equally
Pre- and post-processing is expensive
![Page 26: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/26.jpg)
DEMO
Use case: Write-once, read-manyTest: data-type read performance Data: 319KB JPEG
https://github.com/andygup/indexeddb-typetest-js
![Page 27: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/27.jpg)
Chrome 35.0.1916 – MacBook319KB JPEG
![Page 28: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/28.jpg)
Safari 7.0.3 – MacBook (Shim)319KB JPEG
Whoa!!
![Page 29: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/29.jpg)
Safari 7.1.1 iPad 3 (Shim)319KB JPEG
![Page 30: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/30.jpg)
Safari 7 iPhone 5 (Shim) 319KB JPEG
Whoa!!
![Page 31: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/31.jpg)
Safari 8 iPhone 5S (no shim!)319KB JPEG
![Page 32: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/32.jpg)
Firefox 29.0.1 Desktop – MacBook319KB JPEG
![Page 33: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/33.jpg)
Android 4.4.4 – Nexus 4319KB JPEG
![Page 34: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/34.jpg)
Pre- and Post-processing
What final data type does your app need?
Pre-processWork done before ‘writing’ data to DB
Post-processWork done after ‘getting’ data from DB
![Page 35: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/35.jpg)
Pre- and Post-processing
Potentially huge performance differences
Examples- FileReader- Canvas- Bitwise conversion- Blob to objectURL
![Page 36: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/36.jpg)
Pre-processing
1. Retrieve data from <input> or server2. Convert data (if needed)3. Write to database
WriteConvert
![Page 37: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/37.jpg)
Post-processing
1. Retrieve data from database2. Convert data (if needed)3. Display/use data
Get Display
![Page 38: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/38.jpg)
Pre- and Post-processingXMLHTTPRequest response types
Value Response data type
“” String (Default)
“arraybuffer” ArrayBuffer
“blob” Blob
“document” Document
“json” Object
“text” String
![Page 39: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/39.jpg)
Pre- and Post-processing (Example)var uInt8Array = new Uint8Array(arrBuffer);var blob = new Blob(
[uInt8Array],{type:"image/jpeg"});
var image = document.createElement("img");var urlCreate = window.URL || window.webkitURL;var url = urlCreate.createObjectURL(blob); image.src = url;image.height = 30;image.onload = function(e){
callback(image);URL.revokeObjectURL(this.src);
}
OUTPUT
INPUT
![Page 40: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/40.jpg)
DEMO
Test different ways to process images:FileReader APICanvas APIBitwise manipulation
Use three different image types: PNG, GIF and PNG
https://github.com/andygup/image-parsing-test-js
![Page 41: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/41.jpg)
Convert ArrayBuffer to Image*
GIF
PNG
JPG
* Chrome 35
![Page 42: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/42.jpg)
Convert ArrayBuffer to Image*
* Chrome 35
![Page 43: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/43.jpg)
Convert ArrayBuffer to Image
PNG
JPG
![Page 44: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/44.jpg)
DEMO
Offline mapping Storing large amounts of images
https://github.com/Esri/offline-editor-js
![Page 45: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/45.jpg)
![Page 46: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/46.jpg)
Database size – how big??
General suggestion (smartphones/tablet):- Less than 100 MB
Why? Depends on:- Remaining free memory- How much memory used by browser- How many tabs are already open- If other applications already running
![Page 47: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/47.jpg)
Can I use too much memory?
YES!The device OS will shutdown the browser
Greater potential for data corruption/loss• If shutdown occurs during a ‘write’ operation
![Page 48: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/48.jpg)
Summary - IndexedDB
Designed to work with complex data.
High performance/Asynchronous
Compatible with many offline workflows
Browser support continues to improve
![Page 49: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/49.jpg)
References
https://github.com/andygup/indexeddb-typetest-js
https://github.com/andygup/image-parsing-test-js
https://github.com/Esri/offline-editor-js
https://github.com/axemclion/IndexedDBShim
http://developers.arcgis.com/javascript
![Page 50: HTML5 JavaScript Storage for Structured Data](https://reader030.vdocuments.mx/reader030/viewer/2022012917/56816241550346895dd27b37/html5/thumbnails/50.jpg)
Questions?Andy Gup, Esri Sr. Developer – JS and native Android
www.andygup.netgithub.com/andygup [email protected]@agup