tdd a rest api with node.js and mongodb
TRANSCRIPT
Developing and Testing a MongoDB and Node.js REST API
Valeri KarpovNode.js Engineer, MongoDBwww.thecodebarbarian.com
github.com/vkarpov15@code_barbarian
*
What is this talk about?
•Briefly introduce MongoDB and Node.js
•MongoDB is great for storing web/mobile app data
•So let’s build a REST API using Node.js!
•+ learn a bit about test-driven dev with Node.js
•+ learn two MongoDB schema design principles
•Server-side only
•Upcoming EdX course
*
What is MongoDB?
•Document database - store objects, not columns
•Often better* performance• Better data locality than RDBMS for 1 query vs 5 JOINs
*
What is MongoDB?
•Easier ops• No need to set up databases, tables, schemas, etc.
• Run one executable, you now have a working db
• MongoDB Cloud Manager and Ops Manager
•Developer sanity• In my experience, easiest database to get working
• Minimal data transformation between DB and server
• Company has dedicated “Developer Experience” team
*
Overview
•Part 1: Shopping Cart Application• Search for products
• Add them to your cart
• Check out with Stripe
•Part 2: Using Node.js and the Mongoose ODM
•Part 3: Schema Design
•Part 4: Building an API with the Express framework
•Part 5: Testing with Mocha + Superagent
*
Part 1: What Does the App Do?
*
What Does the App Do?
*
What Does the App Do?
*
App Structure
•"Bad programmers worry about the code. Good programmers worry about data structures and their relationships." - Linus Torvalds
•3 schemas for 3 collections:
•Products
•Categories
•Users
*
Schema Relationships
•Product belongs to one or more categories
•Users can have multiple products in their cart
•Representing relationships in MongoDB is tricky• MongoDB doesn’t have a JOIN equivalent
•But that’s what mongoose is for• Mongoose provides pseudo-JOINs (separate queries)
*
Part 2: Using the Mongoose ODM
•“Object document mapper” (like ORM, but for MongoDB)
•“MongoDB object modeling designed to work in an asynchronous environment”
•Written for Node.js
•Provides schema validation, pseudo-JOINs, etc.
*
Brief Overview of Node.js
•require()
● Async I/O
*
What Does Async Mean?
Register eventhandler
In node.js, you don’t execute I/O imperatively.
You register a callback to execute when the I/O is done
Single thread - handler can’t be interrupted
Prints before“done reading”
*
Why Async?
•Most web app/mobile app backends are I/O bound• Multi-threading doesn’t help if CPU is just waiting for I/O
• Mostly just waiting for database to respond
•Nowadays there’s a REST API for everything• Analytics and tracking: Segment
• Transactional email: Vero
• Mailing lists: MailChimp
•Threads are cumbersome for I/O bound programs
*
Using Your Schema
Create User
Save user to MongoDB
Load user from MongoDB
Print user to stdout
*
Part 2 Takeaways
•Mongoose provides several neat features• Model part of MVC
• Default values
• Schema validation and declarative schema design
*
Part 3: Schema Design
•3 schemas:• Product
• Category
• User
•Going to use mongoose to define schemas
•Will use a couple key schema design principles
*
Product Schema
*
Product Schema in Action
*
Category Schema
*
Category Schema Queries
•What categories are descendants of “Electronics”?
•
•
•What categories are children of “Non-Fiction”?
•
•What categories are ancestors of “Phones”?
*
Product + Category Schemas
*
Category Schema Takeaways
•Queries in MongoDB should be simple
•Strive for minimal data transformation by server
•“Store what you query for”
•“If you need [the aggregation framework in a heavily used API endpoint], you're screwed anyway, and should fix your program.” - Linus Torvalds
•Good for performance and developer sanity
*
User Schema
User’s cart as an arrayof ObjectIds...
*
Principle of Least Cardinality
•Product and user = many-to-many relationship
•Don’t necessarily need a mapping table
•User won’t have 1000s of products in cart
•Can represent relationship as array in user since one side is small
•If one side of many-to-many is bounded and/or small, it is a good candidate for embedding
•Arrays that grow without bound are an antipattern!• 16mb document size limit
• network overhead
*
Part 4: The Express Framework
•Most popular Node.js web framework
•Simple, pluggable, and fast
•Great tool for building REST APIs
*
Your First Express App
Route Parameter
*
What is REST?
•Representational State Transfer
•HTTP request -> JSON HTTP response
•Business logic on top of MongoDB schemas• Access control, emails, analytics, etc.
*
Structuring Your REST API
*
GET /category/id/:id
Find Category
Error handling
Output JSON
*
GET /category/parent/:id
*
GET /product/category/:id
*
Adding Products to User’s Cart
•Recall cart is an array of products
*
Adding Products to User’s Cart
Get cart from HTTP request
Overwrite user’s cart
Let mongoose handlecasting and validating data
*
PUT /me/cart Takeaways
•Mongoose lets you be lazy
•Access control using subdocs
*
Bonus: Stripe Checkout
Create a Stripe charge with the npm ‘stripe’ module
*
Bonus: Stripe Checkout
Error handling
Empty user carton success
*
Part 4 Takeaways
•Express REST API on top of mongoose• Access control
• Business logic
• Define what operations user can take on database
•Mongoose casting and validation for APIs
*
Part 5: Test-Driven Development
•Building an API is tricky
•Lots of different error conditions
•Express has a lot of magic under the hood
*
NodeJS Concurrency and Testing
•Node.js runs in an event loop
•Single threaded
•Can run client and server on same thread!• Client sends HTTP request
• Client registers a callback awaiting the result
• Server’s “on HTTP request” event handler is triggered
• Server sends response, continues waiting for events
• Client’s callback gets fired
•Test server end-to-end
*
Superagent
•NodeJS HTTP client
•Isomorphic: runs in both browser and NodeJS
•Same author as Express
*
Mocha
•Testing Framework for NodeJS
•Same author as Express
•BDD-style syntax• describe() -> test suite
• it() -> individual test
*
Setting Up Category API Tests
*
Testing GET /category/id/:id
*
Part 5 Takeaways
•NodeJS concurrency makes testing easy
•Not just unit tests - full E2E for your REST API
•Can manipulate database and make arbitrary HTTP requests
*
•Upcoming EdX Video Course
•Slides on http://www.slideshare.net/vkarpov15
•Looking for beta testers! Sign up for notifications• http://goo.gl/forms/0ckaJ4YvJN
•Interested in learning about AngularJS?• Professional AngularJS on Amazon
•More NodeJS+MongoDB content at:• www.thecodebarbarian.com
• Twitter: @code_barbarian
Thanks for Listening!