developing ii: advanced document design
TRANSCRIPT
When considering how to model data for a given application, you should…•Think of a logical container for the data•Think of how data groups together
You may notice that…•From a software development standpoint, this maps better to the way you structure the data•It’s a more natural way of representing the entities the application will be handling
Modeling Data
• Key selection: start with the username
• Use this as a common prefix for related data
• When building user’s session information, fetch these items
• Create related documents by prefixing with the same username
• Can extend this concept further
• Other data records for this user, building out object graph, etc.
Example: Data Profile for a User
{ “_id”: “auser_profile”, “user_id”: 7778 “password”: “a1004cdcaa3191b7”, ”common_name”: ”Alan User”, ”nicknames”: [”Al”, ”Buddy”], "sign_up_timestamp": 1224612317, "last_login_timestamp": 1245613101}
{ “_id”: “auser_friends”, “friends”: [ “joe”, “ingenthr”, “toru” ]}
Modeling a Blog
6
CommentComment
PostPost
CommentComment
CommentComment
CommentComment
Blog post structure:•Main content itself•User to blogs is a 1:N relationship•Set of comments, also a 1:N relationship
Representing Objects● Simply treat an object as a JSON document
● Could also represent it with serialized objects from the higher level language
● Determine the key for an object using the class name (or type) of the object and an unique ID
Representing Object Lists● Denormalization● Save an array of object IDs in this list
Representing Blog Data in Couchbase
7
• Three entities (for this simple example)• Users• Hold the user’s profile, enough to instantiate the needed data for a session
• Blog lists• A simple list of blogs written by a given user– semi normalized
• Blogs• The blogs authored by a given user
Modeling the Example Blog Data
8
User ObjectKey: ’JChris'
JSON{ “_id” : “JChris”, “nid” : 1, “name” : “Chris” “password” : “sldkfjslkdfj”}
Blog Data in Couchbase
9
Blog ObjectKey: ’blog_spoke_at_couchconf'
JSON{ “_id” :
“blog_spoke_at_couchconf”, “user_id” : 1 “content-md” : “blah de bla
h”}
UserBlog ListKey: ’JChris_BlogList'
JSON{ “_id” : “Player1_Plantlist”, “blogtitles” : [“spoke_at_couchconf”, “(didn’t)_miss_sxsw”, “next_to_couch_st”]}
● No need to “ALTER TABLE”
● Add new “fields” all objects at any time– Specify default value for missing fields– Increased development speed
● Using JSON for data objects● This will allow future capabilities with Couchbase Server 2.0● Offers the ability to query and analyze arbitrary fields with views
Schema-less Modeling of Data
10
Creating a new blog post
// Create the new blogBlog aBlog = new Blog(”ready_for_lunch");// do more
cbclient.set(aBlog.getId(), aBlog);
// Update the user’s bloglistUser theUser = user.fetch(cbclient.get(username);
// Add the plant to the playertheUser.addPost(aBlog);
// Store the player's new plant listcbclient.set(username + “_bloglist", theUser.getBlogList());
Modifying Blog Data in Couchbase
11
• User profile• Main pointer into the user data:• Blog entries• Badge settings, like a twitter badge
• Blog posts• Contains the blogs themselves• Blog comments• Comments from other users
Adding Comments to a Blog
{
“_id”: “jchris_Hello_World”,
“author”: “jchris”,
“type”: “post”
“title”: “Hello World”,
“format”: “markdown”,
“body”: “Hello from [Couchbase](http://couchbase.com).”,
“html”: “<p>Hello from <a href=\“http: …
“comments”: [
[ “format”: “markdown”, “body”:”Awesome post!” ],
[ “format”: “markdown”, “body”:”Like it.” ]
]
}
Blog Document Sample
Blog Document Sample, Broken up
{
“_id”: “jchris_Hello_World”,
“author”: “jchris”,
“type”: “post”
“title”: “Hello World”,
“format”: “markdown”,
“body”: “Hello from [Couchbase](http://couchbase.com).”,
“html”: “<p>Hello from <a href=\“http: …
“comments”: [
“comment1_jchris_Hello_world”
]
}
{“_id”: “comment1_jchris_Hello_World”,“format”: “markdown”, “body”:”Awesome post!” }
You can imagine how to take this to a threaded list
Threaded Comments
Blog First comment Reply to
comment
More Comments
ListList
Advantages:• Only fetch the data when you need it• For example, rendering part of the page, jQuery style• Spread the data (and thus the load) across the entire cluster
• Simulate the way federated systems work:
• Examples: checking accounts, credit card transactions
• Create a record per transaction
• Leverage views to reconcile the results of the transaction
• If results don’t reconcile, there is a missing transaction or a flaw in the business logic
Exchanging Virtual Currency
{ “From”: “matt”, “to”: “jchris”, “coins”: 30 }
{ “From”: “matt”, “to”: “perry”, “coins”: 30 }
{ “From”: “jchris”, “to”: “perry”, “coins”: 30 }
{ “From”: “matt”, “to”: “jchris”, “coins”: 30 }
{ “From”: “matt”, “to”: “perry”, “coins”: 30 }
{ “From”: “jchris”, “to”: “perry”, “coins”: 30 }
Exchanging Virtual Currency
{ “From”: “matt”, “to”: “jchris”, “coins”: 30 } { “From”: “matt”, “to”: “perry”, “coins”: 30 } { “From”: “jchris”, “to”: “perry”, “coins”: 30 }
Mapperfunction(transaction) { emit(transaction.from, transaction.amount *-1); emit (transaction.to, transaction.amount); }
Reducefunction(keys, values) { return sum(values);}
Query balance view with key == user, get the balance for
the user.
Query the sum of the entire view, value should be 0.
Q: How can I implement transactions across multiple documents?
A: Transactions in this distributed system can be re-imagined with views. (example to follow…)
Q: Can I write more complex aggregation logic? For example apply a transform or a filter when reducing?
A: While the built in reduce functions are often enough and highly tuned, Couchbase Server can execute arbitrary javascript functions.
Common Questions
• Player items• Find all the weapons owned by an individual player• Leader board view• Showing who has the highest level in the system
Demo: Game Simulator Views