couchconf london syncpoint
TRANSCRIPT
![Page 1: CouchConf London Syncpoint](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558c8db8d8b42a0b268b456f/html5/thumbnails/1.jpg)
1
Connecting Millions ofMobile Devices to the Cloud
Marty Schoch@mschoch
![Page 2: CouchConf London Syncpoint](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558c8db8d8b42a0b268b456f/html5/thumbnails/2.jpg)
2
THE WATER CYCLE
![Page 3: CouchConf London Syncpoint](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558c8db8d8b42a0b268b456f/html5/thumbnails/3.jpg)
3
The water cycle
Fun fact: In 20 minutes, one thunderstorm can drop 125,000,000 gallons of water.
![Page 4: CouchConf London Syncpoint](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558c8db8d8b42a0b268b456f/html5/thumbnails/4.jpg)
4
11101011010110100010101011101010101011101010010101001010101010001010101011110101101011010001010101110101010101110101001010100101010101000101010101111010110101101000101010111010101010111010100101010010101010100010101010111101011010110100010101011101010101011101010010101001010101010001010101011110101101011010001010101110101010101110101001010100101010101000101010101111010110101101000101010111010101010111010100101010010101010100010101010111101011010110100010101011101010101011101010010101001010101010001010101011110101101011010001010101110101010101110101001010100101010101000101010101111010110101101000101010111010101010111010100101010010101010100010101010111101011010110100010101011101010101011101010010101001010101010001010101011110101101011010001010101110101010101110101001010100101010101000101010101111010110101101000101010111010101010111010100101010010101010100010101010111101011010110100010101011101010101011101010010101001010101010001010101011110101101011010001010101110101010101110101001010100101010101000101010101111010110101101000101010111010101010111010100101010010101010100010101010111101011010110100010101011101010101011101010010101001010101010001010101011110101101011010001010101110101010101110101001010100101010101000101010101111010110101101000101010111010101010111010100101010010101010100010101010111101011010110100010101011101010101011101010010101001010101010001010101011110101101011010001010101110101010101110101001010100101010101000101010101
11101011010110100010101011101010101011101010010101001010101010001010101011110101101011010001010101110101010101110101001010100101010101000101010101111010110101101000101010111010101010111010100101010010101010100010101010111101011010110100010101011101010101011101010010101001010101010001010101011110101101011010001010101110101010101110101001010100101010101000101010101111010110101101000101010111010101010111010100101010010101010100010101010111101011010110100010101011101010101011101010010101001010101010001010101011110101101011010001010101110101010101110101001010100101010101000101010101111010110101101000101010111010101010111010100101010010101010100010101010111101011010110100010101011101010101011101010010101001010101010001010101011110101101011010001010101110101010101110101001010100101010101000101010101111010110101101000101010111010101010111010100101010010101010100010101010111101011010110100010101011101010101011101010010101001010101010001010101011110101101011010001010101110101010101110101001010100101010101000101010101111010110101101000101010111010101010111010100101010010101010100010101010111101011010110100010101011101010101011101010010101001010101010001010101011110101101011010001010101110101010101110101001010100101010101000101010101111010110101101000101010111010101010111010100101010010101010100010101010111101011010110100010101011101010101011101010010101001010101010001010101011110101101011010001010101110101010101110101001010100101010101000101010101
The water data cycle
![Page 5: CouchConf London Syncpoint](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558c8db8d8b42a0b268b456f/html5/thumbnails/5.jpg)
5
11101011010110100010101011101010101011101010010101001010101010001010101011110101101011010001010101110101010101110101001010100101010101000101010101111010110101101000101010111010101010111010100101010010101010100010101010111101011010110100010101011101010101011101010010101001010101010001010101011110101101011010001010101110101010101110101001010100101010101000101010101111010110101101000101010111010101010111010100101010010101010100010101010111101011010110100010101011101010101011101010010101001010101010001010101011110101101011010001010101110101010101110101001010100101010101000101010101111010110101101000101010111010101010111010100101010010101010100010101010111101011010110100010101011101010101011101010010101001010101010001010101011110101101011010001010101110101010101110101001010100101010101000101010101111010110101101000101010111010101010111010100101010010101010100010101010111101011010110100010101011101010101011101010010101001010101010001010101011110101101011010001010101110101010101110101001010100101010101000101010101111010110101101000101010111010101010111010100101010010101010100010101010111101011010110100010101011101010101011101010010101001010101010001010101011110101101011010001010101110101010101110101001010100101010101000101010101111010110101101000101010111010101010111010100101010010101010100010101010111101011010110100010101011101010101011101010010101001010101010001010101011110101101011010001010101110101010101110101001010100101010101000101010101
11101011010110100010101011101010101011101010010101001010101010001010101011110101101011010001010101110101010101110101001010100101010101000101010101111010110101101000101010111010101010111010100101010010101010100010101010111101011010110100010101011101010101011101010010101001010101010001010101011110101101011010001010101110101010101110101001010100101010101000101010101111010110101101000101010111010101010111010100101010010101010100010101010111101011010110100010101011101010101011101010010101001010101010001010101011110101101011010001010101110101010101110101001010100101010101000101010101111010110101101000101010111010101010111010100101010010101010100010101010111101011010110100010101011101010101011101010010101001010101010001010101011110101101011010001010101110101010101110101001010100101010101000101010101111010110101101000101010111010101010111010100101010010101010100010101010111101011010110100010101011101010101011101010010101001010101010001010101011110101101011010001010101110101010101110101001010100101010101000101010101111010110101101000101010111010101010111010100101010010101010100010101010111101011010110100010101011101010101011101010010101001010101010001010101011110101101011010001010101110101010101110101001010100101010101000101010101111010110101101000101010111010101010111010100101010010101010100010101010111101011010110100010101011101010101011101010010101001010101010001010101011110101101011010001010101110101010101110101001010100101010101000101010101
Increasingly, it starts and ends with a mobile device
Capture Consume
Collect
![Page 6: CouchConf London Syncpoint](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558c8db8d8b42a0b268b456f/html5/thumbnails/6.jpg)
6
Actually, it starts and ends with a LOT of mobile devices
Mobile-broadband subscriptions have grown 45% annually over the last four years and today there are twice as many mobile-broadband as fixed-broadband subscriptions.
The number of mobile-connected devices will exceed the world’s population in 2012; and monthly global mobile data traffic will surpass 10 exabytes in 2016.
Fun Fact: By the end of 1999, the sum of human-produced information (including all audio, video recordings and text/books) was about 12 exabytes of data, according to Berkeley study.
![Page 7: CouchConf London Syncpoint](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558c8db8d8b42a0b268b456f/html5/thumbnails/7.jpg)
7
PUTTING DATA TO WORK
![Page 8: CouchConf London Syncpoint](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558c8db8d8b42a0b268b456f/html5/thumbnails/8.jpg)
8
Make farming data work
![Page 9: CouchConf London Syncpoint](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558c8db8d8b42a0b268b456f/html5/thumbnails/9.jpg)
9
Make intelligence data work
![Page 10: CouchConf London Syncpoint](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558c8db8d8b42a0b268b456f/html5/thumbnails/10.jpg)
10
Make industrial data work
![Page 11: CouchConf London Syncpoint](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558c8db8d8b42a0b268b456f/html5/thumbnails/11.jpg)
11
Make business data work
![Page 12: CouchConf London Syncpoint](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558c8db8d8b42a0b268b456f/html5/thumbnails/12.jpg)
12
Connecting millions of mobile devices to the cloud
otherfarms
Type 1backup and sync
my stuff
Type 2sync relevant portion
of a database
Type 3collect machine-generated data
analysis
![Page 13: CouchConf London Syncpoint](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558c8db8d8b42a0b268b456f/html5/thumbnails/13.jpg)
13
We’ll focus on Type 2
Type 2sync relevant portion
of a database
Apple iC
loud
Smar
tMet
er N
etwork
Most applications combine patterns.
![Page 14: CouchConf London Syncpoint](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558c8db8d8b42a0b268b456f/html5/thumbnails/14.jpg)
14
CHALLENGES … SOLUTIONS
![Page 15: CouchConf London Syncpoint](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558c8db8d8b42a0b268b456f/html5/thumbnails/15.jpg)
15
Type 2 Mobile Sync
The ultimate home of the data for a business application is not on the device, but is a central data store in the cloud.
![Page 16: CouchConf London Syncpoint](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558c8db8d8b42a0b268b456f/html5/thumbnails/16.jpg)
16
Challenges
Which data – temporal, spatial, user?
Bandwidth conservation
Scalability of consolidated store
Conflict detection and resolution
Battery and memory conservation
Selection performance
Deletion propagation
Schema evolution
Referential integrity on interruption
New user provisioning
Security
Hard at “enterprise scale.” Excruciating at “web scale.”
![Page 17: CouchConf London Syncpoint](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558c8db8d8b42a0b268b456f/html5/thumbnails/17.jpg)
17
A multi-tier, scalable type 2 sync architecture
C-channelF-channel
Load
bal
ance
Sync
poin
t
Couc
hbas
e
Elas
tic S
earc
h
App
Serv
er
M
CHANNEL MEMBERSHIP
DOCUMENTS (TO INDEX)
DOCUMENTS SYNC METADATA
DOCUMENTS
![Page 18: CouchConf London Syncpoint](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558c8db8d8b42a0b268b456f/html5/thumbnails/18.jpg)
18
Details Subject to
Change
![Page 19: CouchConf London Syncpoint](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558c8db8d8b42a0b268b456f/html5/thumbnails/19.jpg)
19
ElasticSearch Queries and Syncpoint Channel
{ "range": { "abv": { "from": 7, "to": 10 } }}
Register ElasticSearch query named strongBeer for percolation
Then define a Syncpoint channel with strongBeer as a member query
![Page 20: CouchConf London Syncpoint](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558c8db8d8b42a0b268b456f/html5/thumbnails/20.jpg)
20
The architecture, in more detail
Couchbase Cluster
ElasticSearch Cluster Syncpoint Cluster
DocumentMutation
TAPStream
• Q1• Q2• Q3
PercolationResults
• C1 (Q1, Q2)• C2 (Q2)• C3 (Q1, Q2, Q3)
![Page 21: CouchConf London Syncpoint](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558c8db8d8b42a0b268b456f/html5/thumbnails/21.jpg)
21
Syncpoint Client API
Mobile Client Syncpoint Cluster
Establish Session
• Pluggable Authentication• Pairs user with device• Establishes control database
Subscribe to Channel(s)
• Creates a local installation of this channel• Bi-directional sync of channel documents
![Page 22: CouchConf London Syncpoint](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558c8db8d8b42a0b268b456f/html5/thumbnails/22.jpg)
22
TouchDB – Foundation for Syncpoint
Native Library
![Page 23: CouchConf London Syncpoint](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558c8db8d8b42a0b268b456f/html5/thumbnails/23.jpg)
23
TouchDB Betas – Works with CouchDB Today
TouchDB for iOSTouchDB for
Android
Not a complete product but a functional open source projectswith production deployments.Open source, and available fromCouchbase labs.
![Page 24: CouchConf London Syncpoint](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558c8db8d8b42a0b268b456f/html5/thumbnails/24.jpg)
24
JOIN THE COMMUNITY
http://groups.google.com/group/mobile-couchbase