getting started with mongodb and mongoose
DESCRIPTION
MongoDB and Mongoose IntroductionTRANSCRIPT
Getting Started WithMongoDB{ author: “Ynon Perek” }
Thursday, January 31, 13
Whoami
Ynon Perek
http://ynonperek.com
Thursday, January 31, 13
Agenda
MongoDB Overview
Mongo Test Drive
Mongo Data Model
CRUD Operations
Working With Files
Thursday, January 31, 13
MongoDB Overview
Data Store for JSON Objects
Thursday, January 31, 13
MongoDB Overview
Data Store for JSON Objects
{ “Name” : “Rose Tyler” }
Thursday, January 31, 13
JSON Objects
A JSON Object is a collection of key/value pairs
Keys are simple strings
Values can be: Numbers, Strings, Arrays, Other Objects, and more
Thursday, January 31, 13
JSON Examples
{ “name”: “The Doctor”, “age”: 900 }
{ “race”: “human”, “body parts” : [“head”, “legs”, “arms”, “eyes”]}
Thursday, January 31, 13
MongoDB Overview
A Document Oriented Database (No SQL)
Thursday, January 31, 13
Keeping It Simple
Thursday, January 31, 13
Keeping It Simple
No Transactions
No Joins
Thursday, January 31, 13
Application Architecture
DBSERVER
Thursday, January 31, 13
Application Architecture
DBSERVER
DB
DB
Thursday, January 31, 13
What Can Mongo Do For You
Create and store objects
Arrange them in collections
Retrieve them later
Thursday, January 31, 13
Q & A
Thursday, January 31, 13
Mongo Test DriveCreate MongoLab Account And Start Using The DB
Thursday, January 31, 13
Install mongo Client
Download mongo from:http://www.mongodb.org/downloads
Extract zip file
Run mongo
Thursday, January 31, 13
Install mongo Client
Choose production release for your architecture
Thursday, January 31, 13
Install mongo Client
Note: Still using Windows XP ? You’ll have to use the previous 2.0 version
Thursday, January 31, 13
Meet MongoLab
Thursday, January 31, 13
Database Dashboard
Thursday, January 31, 13
Create New Database
Choose database name
Choose provider
Choose plan (free is good)
Create a DB user
Thursday, January 31, 13
Database Dashboard
Thursday, January 31, 13
Connecting To The DB
There are two options to work with your new DB
You can use the web console
You can use the command line console
Let’s start with the web.
Thursday, January 31, 13
Demo: Creating Documents
Create a few documents on the web console
Update the data
Delete some of them
Search by fields
Thursday, January 31, 13
Mongo Data Model
Let’s model A blog post in a blog app
What’s The Data ?
How Should You Save It ?
Thursday, January 31, 13
Cool MongoDB Design
{“title”: “Mongo 101”,“author” : “ynonp”,“comments” : [ { “author” : “...”, “content” : “...” }, { “author” : “...”, “content” : “...” }],“tags” : [ “funny”, “informative”],“content” : “...”
}
Thursday, January 31, 13
Q & A
Thursday, January 31, 13
Lab
Create a DB for musical info
Create a collection called albums
Add info for 3 albums you like, including:
Album Name, Artist, Tracks, Release Date, Genres
Tracks is an array of objects
Genres is an array of strings
Thursday, January 31, 13
CRUD OperationsCreate, Read, Update and Destroy Data
Thursday, January 31, 13
Mongo CRUD
Create is called insert
Read is called find
Update is called update
Destroy is called remove
Thursday, January 31, 13
Mongo CRUD
From a developer’s perspective, MongoDB operations are the same through the driver and through the console
In both cases, operations look like function calls or method invocations
We’ll use mongo shell for the rest of this chapter
Thursday, January 31, 13
Inserting Data
Use the command insert or save to insert a new object
db.collection.insert( obj );
db.collection.insert( array );
Thursday, January 31, 13
Inserting Data
Inserting to a new collection creates the collection
Inserting an object with an _id key, it is used as the object’s id (and must be unique).
Thursday, January 31, 13
find and findOne perform read operations
Both take a query
find returns a cursor
findOne returns an object
db.collection.find( <query>, <projection> )
Reading Data
Optional: Fields to fetch
Thursday, January 31, 13
Query Document
An empty (or missing) query document returns everything
db.collection.find({})
db.collection.find()
Thursday, January 31, 13
Query Document
Each key/value pair in the query document imposes a condition on the results (objects that match).
db.movies.find({ “genre” : “indie” });
db.books.find({“pages” : { “$gt” : 100 }});
Thursday, January 31, 13
Query Document
Each key/value pair in the query document imposes a condition on the results (objects that match).
db.movies.find({ “genre” : “indie” });
db.books.find({“pages” : { “$gt” : 100 }});
Query Object
Thursday, January 31, 13
Query Document
A compound query means a logical AND on the conditions.
db.inventory.find( { “type” : “snacks”, “available” : { “$lt” : 10 } });
Thursday, January 31, 13
Quiz: What Is Returned
{ “publisher” : “DC”}
from alterego publisher
Earth Bruce Wayne DC
Earth Peter Parker Marvel
Krypton Clark Kent DC
Thursday, January 31, 13
Quiz: What Is Returned
{ “publisher” : “DC”, “from” : “Earth”}
from alterego publisher
Earth Bruce Wayne DC
Earth Peter Parker Marvel
Krypton Clark Kent DC
Thursday, January 31, 13
More Queries
You can use “$or” to have an OR expression
{ “$or” : [ { “type” : “food” }, { “type” : “drinks” } ]}
Thursday, January 31, 13
Sub Documents
If your document has sub-documents, it’s possible to query by a full sub document or look for a partial match
Full sub-document query means subdocument is exactly as specified in the query
Example:
{ ISBN : { “ISBN-10” : “1906465592”, “ISBN-13” : “978-1906465599” }}
Thursday, January 31, 13
Sub Documents
A partial query matches all objects that have at least the required field (but may contain more)
Example:{ “language.primary” : “english”}
Value of language is an object, and it has a field called primary
Thursday, January 31, 13
Arrays
You can use an exact array match by providing the full array in the query
Example:{ tags : [ “funny”, “cute”, “cats” ]}
Thursday, January 31, 13
Arrays
You can query for an array that has at least one element matching the query
Example:
{ “tags” : “funny” }
Thursday, January 31, 13
Arrays
If you have a subdocument as the element of an array, it’s possible to query by its fields using the dot notation.
Examples:
{ “tracks.4.name” : “Rose Mary Stretch” }
{ “tracks.name” : “Rose Mary Stretch” }
Thursday, January 31, 13
Query Operators
Complex queries are performed with special operators.
These are reserved words starting with a $
Some of them: $gt, $gte, $lt, $lte, $ne, $in, $nin, $all, $or, $not
Thursday, January 31, 13
Comparator Queries
Value for key a is greater than 10{ “a” : { “$gt” : 10 }}
Value for key b is not 7{ “b” : { “$ne” : 7 }}
Value for key name is greater (dictionary sort) than ‘bird’{ “name” : { “$gt” : “bird” }}
Thursday, January 31, 13
Queries: $in, $nin
Use $in to specify a choice from multiple options
Value for grade is 85, 90 or 100{ “grade” : { “$in” : [ 85, 90, 100 ] } }
Value for fruit is neither apple nor banana{ “fruit” : { “$nin” : [“apple”, “banana” ] } }
Thursday, January 31, 13
Quiz: What Is Selected
{ “reads” : { “$gt” : 10 }, “author” : { “$nin” : [“admin”, “manager”, “boss” ] } }
author reads title
admin 99 How To Use Mongo
Joe 120 How To Make Money
Jim 8 Windows Manual
Thursday, January 31, 13
Queries: $all
Select objects with array containing all elements
Example:{ “tags” : { “$all” : [ “funny”, “cats” ] } }
Thursday, January 31, 13
More Query Operators
“$size” - array has a specific number of elements
“$exists” - field present or missing
Example:
{ “friends” : { “$size” : 7 } }
{ “producer” : { “$exists” : false } }
Thursday, January 31, 13
Aggregation
count() - returns how many objects found
distinct() - returns all distinct values for a key
Example:
db.posts.distinct( “tags” )
Thursday, January 31, 13
Resources
Queries Cheat Sheethttp://www.10gen.com/sites/default/files/downloads/mongodb_qrc_queries.pdf
Thursday, January 31, 13
Q & A
Thursday, January 31, 13
Lab
Using the previously defined musical DB. Query for:
Albums released after/before 2008
Albums with 7 tracks
Albums by a specific genre
Albums by a specific track name
Display ALL different genres in the DB
Thursday, January 31, 13
Update
Update operations modify existing data in the DB
Mongo supports two update commands: update() and save()
Update is the more general (and complex)
Thursday, January 31, 13
Update
The general form for update is:
db.collection.update( <query>, <update>, <options> )
Which Entries to update
What to do with them
Thursday, January 31, 13
Update
The second argument to update() is an operator object
It tells update what to do with the data
Some keys you can use: “$set”, “$inc” “$push”, “$pushAll”, “$addToSet”, “$pop”, “$pull”, “$pullAll”
Thursday, January 31, 13
Update: set
$set modifies a value or add a new value
Example:
db.posts.update( { title: “Why Is Your Cat Unhappy” }, { $set : { “archived” : true } });
Thursday, January 31, 13
Quiz: $set
What happens here ?
db.cats.update( { color: “white” }, { “$set” : { “owners” : [“John”, “Jim”] } });
Thursday, January 31, 13
Quiz: $set
Update owners array of the first cat with white color
If you want to update all objects, use multi
db.cats.update( { color: “white” }, { “$set” : { “owners” : [“John”, “Jim”] } } { multi : true });
Thursday, January 31, 13
Update: inc
$inc increases a numeric value
Example:
{ “$inc” : { “age” : 11 } }
Thursday, January 31, 13
Quiz: $inc
What happens here ?
db.songs.update( { “title” : “Killing Lies” }, { “$inc” : { “plays” : 1 } });
Thursday, January 31, 13
Update: push and pushAll
push() and pushAll() add items to an existing array
If they array did not exists, it is created
Example:
db.creatures.update( { name: “The Doctor” }, { “$push” : { companions : “Rose Tyler” } })
Thursday, January 31, 13
Update: addToSet
The $addToSet adds a new item only if it wasn’t already in the array
Example:
{ “$addToSet” : { “tags” : “funny” } }
Thursday, January 31, 13
Update: pop
pop removes items of an array
Use a value of 1 to remove the last element
Use a value of -1 to remove the first element
Example:
{ “$pop” : { “companions” : 1 } }
Thursday, January 31, 13
Update: pull
Remove a specific item from an array.
Can use $pullAll to remove all matching elements
Example:
{ “$pull” : { “companions” : “Rose Tyler” } }
Thursday, January 31, 13
Updating with save()
The second update operation is save()
takes a document:
If the document has an id - update it
If not, insert it to the DB
Thursday, January 31, 13
Deleting Data
remove() deletes objects from a collection
Takes a query and possibly a <justOne> arguments
Examples:
db.posts.remove({ “author” : “Father Angelo” })
db.music.remove({ “genres” : “pop” })
db.posts.remove({ “tags” : “funny” }, 1 );
Thursday, January 31, 13
Q & A
Thursday, January 31, 13
Lab
From the previous music database:
Add a new album with 4 tracks
Add a new track to that new album
Set property “plays” on all albums to 6
Increase it by 4 only for “indie” albums
Delete all “indie” music
Thursday, January 31, 13
MongooseMongoDB + Node.JS
Thursday, January 31, 13
What’s That
An Object Relational Mapper for Node.JS
Handles gory details so you don’t have to
Fat Models
Thursday, January 31, 13
Agenda
Hello Mongoose
Schema and Data Types
Custom Validators
Querying Data
Poor Man’s Joins (Populate)
Mongoose Plugins
Thursday, January 31, 13
Online Resources
http://mongoosejs.com/
https://github.com/LearnBoost/mongoose
http://www.youtube.com/watch?v=4fQsDiioj3I
irc: #mongoosejs on freenode
Thursday, January 31, 13
Hello Mongoose
var mongoose = require('mongoose');mongoose.connect('localhost', 'test');
var schema = mongoose.Schema({ name: 'string' });var Cat = mongoose.model('Cat', schema);
var kitty = new Cat({ name: 'Zildjian' });kitty.save(function (err) { if (err) // ... console.log('meow');});
Thursday, January 31, 13
Mongoose Objects
Schema
Model Model
Schema
Model
Thursday, January 31, 13
Mongoose Objects
{ name: String}
Cat
kitty
var mongoose = require('mongoose');mongoose.connect('localhost', 'test');
var schema = mongoose.Schema( { name: 'string' });
var Cat = mongoose.model( 'Cat', schema);
var kitty = new Cat( { name: 'Zildjian' });
Thursday, January 31, 13
Schema Definitions
A schema takes a description object which specifies its keys and their types
Types are mostly normal JS
new Schema({ title: String, body: String, date: Date, hidden: Boolean, meta: { votes: Number, favs: Number }});
Thursday, January 31, 13
Schema Types
String
Number
Date
Buffer
Boolean
Mixed
ObjectId
Array
Thursday, January 31, 13
Nested Objects
Creating nested objects is easy
Just assign an object as the value
var PersonSchema = new Schema({ name: { first: String, last: String }});
Thursday, January 31, 13
Array Fields
Array fields are easy
Just write the type as a single array element
var PersonSchema = new Schema({ name: { first: String, last: String }, hobbies: [String]});
Thursday, January 31, 13
Schema Use Case
Let’s start writing a photo taking app
Each photo is saved in the DB as a Data URL
Along with the photo we’ll save the username
var PhotoSchema = new Schema({ username: String, photo: String, uploaded_at: Date}); var Photo = mongoose.model( 'Photo', PhotoSchema);
Thursday, January 31, 13
Creating New Objects
Create a new object by instantiating the model
Pass the values to the ctor
var mypic = new Photo({ username: 'ynon', photo: 'foo', uploaded_at: new Date()});
Thursday, January 31, 13
Creating New Objects
After the object is ready, simply save it mypic.save();
Thursday, January 31, 13
What Schema Can Do For You
Add validations on the fields
Stock validators: required, min, max
Can also create custom validators
Validation happens on save
var PhotoSchema = new Schema({ username: { type: String, required: true }, photo: { type: String, required: true },
uploaded_at: Date});
Thursday, January 31, 13
What Schema Can Do For You
Provide default values for fields
Can use a function as default for delayed evaluation
var PhotoSchema = new Schema({ username: { type: String, required: true }, photo: { type: String, required: true }, uploaded_at: { type: Date, default: Date.now }});
Thursday, January 31, 13
What Schema Can Do For You
Add methods to your documents
var EvilZombieSchema = new Schema({ name: String, brainz: { type: Number, default: 0 }}); EvilZombieSchema.methods.eat_brain = function() { this.brainz += 1;};
Thursday, January 31, 13
Custom Validators
It’s possible to use your own validation code
var toySchema = new Schema({ color: String, name: String}); toySchema.path('color').validate(function(value) { return ( this.color.length % 3 === 0 );});
Thursday, January 31, 13
Schema Create Indices
A schema can have some fields marked as “index”. The collection will be indexed by them automatically
var PhotoSchema = new Schema({ username: { type: String, required: true, index: true }, photo: { type: String, required: true }, uploaded_at: { type: Date, default: Date.now }});
Thursday, January 31, 13
Schemas Create Accessors
A virtual field is not saved in the DB, but calculated from existing fields. “full-name” is an example.
personSchema.virtual('name.full').get(function () { return this.name.first + ' ' + this.name.last;});
personSchema.virtual('name.full').set(function (name) { var split = name.split(' '); this.name.first = split[0]; this.name.last = split[1];});
Thursday, January 31, 13
Q & A
Thursday, January 31, 13
Querying Data
Use Model#find / Model#findOne to query data
// executes immediately, passing results to callbackMyModel.find({ name: 'john', age: { $gte: 18 }}, function (err, docs) { // do something with data // or handle err});
Thursday, January 31, 13
Querying Data
You can also chain queries by not passing a callback
Pass the callback at the end using exec
var p = Photo.find({username: 'ynon'}). skip(10). limit(5). exec(function(err, docs) { console.dir( docs );});
Thursday, January 31, 13
Other Query Methods
find( cond, [fields], [options], [cb] )
findOne ( cond, [fields], [options], [cb] )
findById ( id, [fields], [options], [cb] )
findOneAndUpdate( cond, [update], [options], [cb] )
findOneAndRemove( cond, [options], [cb] )
Thursday, January 31, 13
Counting Matches
Use count to discover how many matching documents are in the DB
Adventure.count({ type: 'jungle' }, function (err, count) { if (err) .. console.log('there are %d jungle adventures', count);});
Thursday, January 31, 13
Lab
Create a Schema called “Album”
Add fields: artist, year, tracks
Create a model and a document
Add validator for year
Save it in the DB
Thursday, January 31, 13
Lab
Create 5 albums from years 2008, 2009, 2010, 2011, 2012
Query the 3 newest albums
Print the artist name and the number of tracks
Print the artist who has the most albums
Thursday, January 31, 13
Populating Collections
Mongo has no joins
Let’s Fake Them
Thursday, January 31, 13
Start With Relationships
Execute the following to create an initial relationshiphttps://gist.github.com/4657446
Watch the data in the DB:
Album.artist = ObjectId("5106b6e6fde8310000000001")
Thursday, January 31, 13
Use Query#populate
query#populate sends another query for the related object
Album.findOne().exec(function(err, doc) { // prints undefined console.log( doc.artist.name );}); Album.findOne().populate('artist').exec(function(err, doc) { // prints Pink Floyd console.log( doc.artist.name );});
Thursday, January 31, 13
Use Query#populate
Full method signature:
Query#populate( path, [fields], [model], [cond], [options] )
cond is a query condition object ( i.e. { age: { $gte: 21 }}
options is a query options object ( i.e. { limit: 5 }
Helps when populating arrays
Thursday, January 31, 13
Mongoose Plugins
A plugin connects to the Schema and extends it in a way
Thursday, January 31, 13
Mongoose Plugins
A mongoose plugin is a simple function which takes schema and options
Demo: lastModifiedPluginhttps://gist.github.com/4657579
Thursday, January 31, 13
Mongoose Plugins
find or create plugin:
https://github.com/drudge/mongoose-findorcreate
Thursday, January 31, 13
Mongoose Plugins
Hashed password field plugin:https://gist.github.com/4658951
Thursday, January 31, 13
Mongoose Plugins
Mongoose troops is a collection of useful mongoose plugins:https://github.com/tblobaum/mongoose-troop
Thursday, January 31, 13
Thank You
Photos from: http://123rf.com
Slides available at: http://ynonperek.com
Thursday, January 31, 13