Download - Prophet: a path out of the Cloud
![Page 2: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/2.jpg)
You may know me from...
RT (Request Tracker)
Jifty
SVK
Hiveminder
Perl 6
Shirts
2
![Page 3: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/3.jpg)
I’ve been hacking on an open source database
called “Prophet”
3
![Page 4: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/4.jpg)
It has an API like Amazon SimpleDB or Google App Engine’s...
4
![Page 5: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/5.jpg)
It’s designed for “team-scale” apps
5
![Page 6: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/6.jpg)
It’s built for P2P replication and
disconnected use
6
![Page 7: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/7.jpg)
App #1 is the canonical “offline bug tracker”
7
![Page 8: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/8.jpg)
App #2 will probably be a BBS you can sync
over sneakernet
8
![Page 9: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/9.jpg)
But first, a brief digression...
9
![Page 10: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/10.jpg)
...about cloud computing
10
![Page 11: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/11.jpg)
Living in the cloud =
sharecropping
11
![Page 12: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/12.jpg)
(That’s bad)
12
![Page 13: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/13.jpg)
This is a rant
13
![Page 14: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/14.jpg)
The bad old days:
14
![Page 15: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/15.jpg)
Pic of sharecroppers
15
![Page 16: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/16.jpg)
You farmed land you didn’t own...
16
![Page 17: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/17.jpg)
...with tools you couldn’t really afford
17
![Page 18: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/18.jpg)
You paid for it with part of your harvest...
18
![Page 19: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/19.jpg)
It sounded like apretty sweet deal...
19
![Page 20: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/20.jpg)
...until things got bad
20
![Page 21: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/21.jpg)
(Things always got bad)
21
![Page 22: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/22.jpg)
In a bad year, you got further in debt tothe land owner
22
![Page 23: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/23.jpg)
23
![Page 24: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/24.jpg)
The (more recent)bad old days:
24
![Page 25: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/25.jpg)
pic of mainframes
25
![Page 26: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/26.jpg)
You ran code you didn’t own on hardware you
didn’t own
26
![Page 27: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/27.jpg)
Things got a little better:
27
![Page 28: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/28.jpg)
Pic of PCs
28
![Page 29: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/29.jpg)
Things weren’t all rosy:
29
![Page 30: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/30.jpg)
Pic of BSOD
30
![Page 31: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/31.jpg)
Sometimes new versions of software
killed features...
31
![Page 32: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/32.jpg)
...so you were locked in to old versions
32
![Page 33: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/33.jpg)
pic of win 31?
33
![Page 34: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/34.jpg)
Things got ‘better’:
34
![Page 35: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/35.jpg)
rmsche
35
![Page 36: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/36.jpg)
Now, things are getting worse again...
36
![Page 37: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/37.jpg)
37
![Page 38: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/38.jpg)
What happens when your favorite service
goes down?
38
![Page 39: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/39.jpg)
pic of twitter being down
39
![Page 40: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/40.jpg)
...or stops accepting new signups?
40
![Page 41: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/41.jpg)
41
![Page 42: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/42.jpg)
...or gives all your data to the secret police?
42
![Page 43: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/43.jpg)
Pic of yahoo.cn
43
![Page 44: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/44.jpg)
...or starts making arbitrary choices about what’s ‘safe’ content?
44
![Page 45: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/45.jpg)
45
![Page 46: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/46.jpg)
You don’t own the services you use
46
![Page 47: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/47.jpg)
When the service provider cuts you off, that’s it. No recourse.
47
![Page 48: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/48.jpg)
Not so secret shame:I’m a really bad zealot
48
![Page 49: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/49.jpg)
My calendar lives at google.com
49
![Page 50: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/50.jpg)
50
![Page 51: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/51.jpg)
I make a web 2.0 tasklist service called
Hiveminder.com
51
![Page 52: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/52.jpg)
pic of hiveminder
52
![Page 53: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/53.jpg)
Using hosted apps is going to hurt you
53
![Page 54: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/54.jpg)
Data access is important
54
![Page 55: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/55.jpg)
APIs are great
55
![Page 56: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/56.jpg)
...but easy access to a service just makes it
easier to get locked in
56
![Page 57: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/57.jpg)
What about Google Gears, Adobe Air, etc?
57
![Page 58: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/58.jpg)
Great. now you can use your word processer while you’re offline!
58
![Page 59: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/59.jpg)
Pic of wordperfect
59
![Page 60: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/60.jpg)
Real offline apps shouldn’t need servers
60
![Page 61: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/61.jpg)
Real offline appsshould sync like you do
61
![Page 62: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/62.jpg)
I might be a nut job
62
![Page 63: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/63.jpg)
...but smart people seem to agree with me
63
![Page 64: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/64.jpg)
If we want people to have the same degree of user autonomy as we've come to expect from the world, we may have to sit down and code alternatives to Google Docs, Twitter, and EC3 that can live with us on the edge, not be run by third parties.
- Danny O’Brienhttp://www.oblomovka.com/entries/2008/07/16
64
![Page 65: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/65.jpg)
Back to that database thing...
65
![Page 66: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/66.jpg)
Jesse Vincent
66
![Page 67: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/67.jpg)
Chia-liang Kao
67
![Page 68: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/68.jpg)
We work together
68
![Page 69: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/69.jpg)
CL lives in TaipeiJesse lives in Boston
69
![Page 70: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/70.jpg)
Sometimes we needto work face to face
70
![Page 71: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/71.jpg)
TPE - BOS:TPE - HNL:BOS - HNL:
9410 mi5,095 mi5,069 mi
71
![Page 72: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/72.jpg)
Step 1: Go to Hawaii for “work”Step 2: ???Step 3: Prophet!
Our Plan
72
![Page 73: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/73.jpg)
The Plan Backfired
We were there for 8 days
We wrote 8000 lines of Perl
We figured out step 2
73
![Page 74: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/74.jpg)
Step 2:
Build a Disconnected Syncable Database
74
![Page 75: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/75.jpg)
Prophet
75
![Page 76: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/76.jpg)
Prophet
http://syncwith.us/prophet
SD
http://syncwith.us/sd
Getting Prophet
76
![Page 77: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/77.jpg)
A grounded, semirelational,
peer to peer replicated,
disconnected, versioned,
property database with
self-healing conflict resolution
Prophet
77
![Page 78: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/78.jpg)
What do all thosebuzzwords mean?
78
![Page 79: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/79.jpg)
grounded
Runs here
79
![Page 80: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/80.jpg)
grounded
Not here
80
![Page 81: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/81.jpg)
grounded
Runs at the edge
Doesn’t need to run in the cloud
Syncs with services you already use
(We call the adaptors “Foreign Replicas”)
81
![Page 82: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/82.jpg)
Joins are expensive
(They’re still possible)
semirelational
82
![Page 83: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/83.jpg)
Update any replica
Pull from any replica
Push to any replica
Publish a replica
Changes will propagate
peer-to-peer replicated
83
![Page 84: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/84.jpg)
Real-time replication is hard to scale
It only “works” with constant connectivity
I don’t have constant connectivity
Neither do you
Prophet sync can happen whenever
disconnected
84
![Page 85: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/85.jpg)
Every update is recorded as a change set
Change sets don’t lose any data
(so you can use them to go backwards)
All history is introspectable
Replication just replays changesets
versioned
85
![Page 86: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/86.jpg)
Atomic operations
CREATE, READ, UPDATE, DELETE, SEARCH
Record types can have optional validation and canonicalization
Records of the same type do not need to have the same properties
Add and remove properties at will
property database
86
![Page 87: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/87.jpg)
Remembers all conflict resolutions
Syncs all resolutions with your peers
Detects identical conflicts
Uses your peers’ resolutions to “vote” for the winner of a conflict
self-healing conflict resolution
87
![Page 88: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/88.jpg)
Working with Prophet
88
![Page 89: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/89.jpg)
RESTy API
GET /records.json
GET /records/Cars.json
GET /records/Cars/716499-5F9-4AC4-827.json
GET /records/Cars/716499-5F9-4AC4-827/wheels.json
POST /records/Cars.json
POST /records/Cars/716499-5F9-4AC4-827.json
POST /records/Cars/716499-5F9-4AC4-827/wheels.json
89
![Page 90: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/90.jpg)
RESTy API
Yes, we should be using PUT and DELETE
Yes, you can have a commit bit and help us fix it :)
90
![Page 91: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/91.jpg)
Native API(Yes, the core is Perl.)
my $cli = Prophet::CLI->new();
my $cxn = $cli->app_handle->handle;
my $record = Prophet::Record->new( handle => $cxn, type => 'Person' );
my $uuid = $record->create( props => { name => 'Jesse', age => 31 } );
$record->set_prop( name => 'age', value => 32 );
my $people = Prophet::Collection->new( handle => $cxn, type => 'Person' );
$people->matching( sub { shift->prop('species') ne 'cat' } );
91
![Page 92: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/92.jpg)
What could you build with Prophet?
92
![Page 93: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/93.jpg)
A bug tracker: “simple defects”
• id. Status, Summary
• (Arbitrary other properties too)
•History
•Comments
•Attachments
sd
93
![Page 94: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/94.jpg)
./bin/sd ticket create -- summary="Can't sync sd with Google Code" status=new
Created ticket 5 (93BF979E-08C1-11DD-94C3-D4B1FCEE7EC4)
Create
94
![Page 95: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/95.jpg)
./bin/sd ticket search --regex publish
29 } new the online help doesn't describe publish
34 } new publish a static html view of records
35 } new publish should create a static rss file
List and Search
95
![Page 96: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/96.jpg)
./bin/sd ticket update --uuid 93BF979E-08C1-11DD-94C3-D4B1FCEE7EC4 -- status=resolved
Updates
96
![Page 97: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/97.jpg)
Bugs on my laptop aren’t interesting.
97
![Page 98: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/98.jpg)
Jesse
sd publish --to fsck.com:public_html/sd/
CL
sd pull --from http://my.com/~jesse/sd
Sync!
98
![Page 99: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/99.jpg)
My project has a bug tracker
99
![Page 100: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/100.jpg)
Actually, mine use two:
• RT
• hiveminder.com
My project has a bug tracker
99
![Page 101: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/101.jpg)
Foreign Replicas
Prophet makes Foreign Replicas easy
SD gets them "for free"
100
![Page 102: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/102.jpg)
(Using only the public REST API)
It took an afternoon
Mirror an RT instance into SD
Share it with your peers using prophet
Sync changes back from your peers to RT
Supports Comments and Attachments
Wrote an RT Replica for SD
101
![Page 103: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/103.jpg)
(Using only the public REST API)
...and one for Hiveminder
102
![Page 104: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/104.jpg)
I can sync my bugs with RT or Hiveminder
103
![Page 105: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/105.jpg)
Actually, it’s better
104
![Page 106: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/106.jpg)
I can sync between RT and Hiveminder
105
![Page 107: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/107.jpg)
I can sync between two different RTs, too
106
![Page 108: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/108.jpg)
• Trac
• Launchpad
• Google Code
• SourceForge
• Bugzilla
• Jira
• GForge
• debbugs
• GNATS
• todo.txt
• Lighthouse
• Redmine
• FogBugz
• What else?
We need more replica definitions:
107
![Page 109: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/109.jpg)
What else can you use Prophet for?
108
![Page 110: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/110.jpg)
All your “social” databases
109
![Page 111: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/111.jpg)
•CRM
•Bug tracking
•Sales orders
•Phone book
•Blog
•Trading Card Database
•Ideas?
All the databases you want while offline.
110
![Page 112: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/112.jpg)
How about a P2P BBS?
Prophet doesn’t need a server.
You can sync over sneakernet.
“Private” Social Networks
111
![Page 113: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/113.jpg)
A look inside Prophet
112
![Page 114: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/114.jpg)
Anatomy of a Prophet Replica
113
![Page 115: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/115.jpg)
The bits and pieces
Database UUID
Replica UUID
Record Store
Changeset Store
Resolution Database
Configuration metadata
114
![Page 116: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/116.jpg)
The Record Store
Stores individual records by type
Not guaranteed to have all old versions
115
![Page 117: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/117.jpg)
The Changeset Store
Stores every change to a set of records
Guaranteed to have all old changesets
Replaying all changesets will create an exact clone of the replica
116
![Page 118: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/118.jpg)
Replica Backends
117
![Page 119: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/119.jpg)
Filesystem
Readable
Flat files
Compact
Fast
(Not yet fully atomic)
118
![Page 120: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/120.jpg)
HTTP
Designed to let you “publish” databases
Flat-files, Currently read-only.
Same format as the filesystem replica type.
119
![Page 121: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/121.jpg)
Subversion (DEPRECATED)
Slow
Steady
Robust
Supports remote sync
Requires Subversion Perl Bindings
120
![Page 122: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/122.jpg)
Backends are pluggable!
The filesystem is cheap and easy
The filesystem is portable
Help us write new backends:
CouchDB, SQLite, MySQL, Postgres, S3, AppEngine, $YOUR_FAVORITE_DB
121
![Page 123: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/123.jpg)
Prophet is designed to sync with “other” databases and systems
They don’t need to support all of Prophet’s features - Prophet knows how to interpret mumbo-jumbo from the Cloud
Foreign Replicas will usually be app specific
All current examples are for SD
Foreign Replicas
122
![Page 124: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/124.jpg)
Synchronization
123
![Page 125: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/125.jpg)
Publish
Serialize and export all of a replica's resolutions and changesets
124
![Page 126: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/126.jpg)
Pull
Integrate unseen resolutions and then unseen changesets from a replica
125
![Page 127: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/127.jpg)
Push
Integrate new resolutions and changesets into a replica
126
![Page 128: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/128.jpg)
Conflicts
127
![Page 129: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/129.jpg)
Figures out the best resolution
“Nullifies” the conflict so the changeset can be cleanly integrated
Integrates the conflicting changeset
Records the resolution as a new changeset
Records the resolution decision in the resolution database
Resolving Conflicts
128
![Page 130: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/130.jpg)
Prophet has clever ways to figure out the best resolution.
If there are previous resolutions for the same conflict and a majority agree, use that
If the merger has specified a “prefer this side” choice, use that
Prompt the user to make a decision, giving them info about previous decisions for this conflict
“The Best Resolution”
129
![Page 131: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/131.jpg)
Scaling
130
![Page 132: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/132.jpg)
Scaling to giant clusters is boring
(Can I play the “They’re not Green” card here?)
Scales to many weakly coonnected peers
You are not Google.
Does anyone here work for Google?
Current target is databases of O(50k) records
How does it scale?
131
![Page 133: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/133.jpg)
We have a political agenda.
Cloud computing is not Open Source.
APIs for “export” are not good enough.
You should always have full control.
You probably don’t need to store 10 billion records in one database.
Why not, then?
132
![Page 134: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/134.jpg)
Do you have 10 billion bugs, customer contacts
or sales orders?
133
![Page 135: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/135.jpg)
That said, we'd love to see a scalable, high
performance prophet replica store
134
![Page 136: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/136.jpg)
Getting Involved
135
![Page 137: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/137.jpg)
Project Status
Simple, well-defined Perl API
RESTy web API (with microserver)
Fast, lightweight backend
Small, active dev community
Great test coverage
...less than great documentation coverage
136
![Page 138: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/138.jpg)
Better ergonomics
Improved search and indexing
(Including full-text indexing)
Client libraries for other languages
Proper security model
More apps
Our Plans
137
![Page 139: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/139.jpg)
Prophet
6937 lines of code and doc
1952 lines of tests
sd
2121 lines of code and doc
973 lines of tests
Codebase
138
![Page 140: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/140.jpg)
Prophet is very young
Prophet designed in April
Prophet core implemented in April
SD designed in April
SD built in June and July
139
![Page 141: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/141.jpg)
We need your help!
Kick-ass functional and text indexing
Backend data store improvements
Slick GUIs for syncing
More Foreign Replicas for SD
Documentation improvements
A clever logo
New applications
140
![Page 142: Prophet: a path out of the Cloud](https://reader037.vdocuments.mx/reader037/viewer/2022110307/55583238d8b42ac6078b4614/html5/thumbnails/142.jpg)
Prophet
http://syncwith.us/prophet/download
SD
http://syncwith.us/sd/download
Getting Prophet
141