sc5 meetup: node.js night with jaakko manninen

Node.js at Rocket PackLessons learned; Patterns to know and love

Jaakko ManninenRocket Pack / Disney Interactive

Wednesday, May 15, 13

Rocket Pack

• Makes awesome HTML5 games and the tools to build them

• Engine, Tool chain, Services 100% JS

• Acquired by Disney in 2011

• Started using Node.js in 2010, version 0.1

• Full stack, cloud hosted setup

• Very early days for HTML5 games: first toolchain


What is Node.js

• JavaScript engine with IO library & APIs

• Google’s V8 + libuv + core APIs

• V8 is Chrome’s very fast JS engine

• One thread of execution

• Asynch, non-blocking, event driven IO

• Great for servers, real-time messaging, but also for web and standalone apps


Road to Node

• First as HTTP front end for Scala backend

• Parse request, deliver to AMQP message broker

• Games ran engine code in Rhino/JVM

• Ran it in Node and soon ported everything to Node

• V8 is very fast :)

• Most packages we have used, we have had to patch :(

• Today package quality improving but still poor :/


Rocket Pack version 1

• Cloud: Scalable, stateless, modular

• Top to bottom JavaScript, all the way from MongoDB to the client side runtime engine

• Messaging to tie the elements together

• First RabbitMQ & node-amqp

• Then wrote our own Messaging API


Messaging• Wrote load-balancing, p2p messaging API

using github.com/einaros/ws

• Careful about fanout style exchanges, where processing capability is limited to one core only

• Central discovery: if you have a multicast group, think about node-mdns

• Socket.IO messages quickly routed to services by FE

• Sticky lb; no Store yet; no engine.io yet

• Use engine.io with eg. a Redis Store


Processing woes

• An evolving platform needs to run data migrations

• Memory limited to 1.2GB (fixed only in 2012)

• Use cluster to task workers on other cores

• Or discover nodes in cluster (mdns, roll-your-own)


Lessons part 1• One-thread-only is your friend in scaling

• Many packages are buggy and you will have to patch them (but submit a pull request!)

• Use domain to capture out-of-stack errors

• Where did that socket hang up come from? domain will tell you

• Domain itself is experimental

• Never throw - just return errors

• Never use uncaughtException


Part 2 - Local Tools

• Brought dev tools to the dev’s machine

• Windows support; offline support

• Plugins: declare and depend on named interfaces

• Excellent testability, modularity, composability

• CommonJS, npm, package.json everywhere


• npm is an awesome package manager• Large dependency trees may cause rare issues

• It doesn’t have to be JavaScript

• Tens of thousands of packages

• Private repository recommended

• Our repo mirrors npm; has our private packages

• Not trivial to set up, but worth it


browserify

• github.com/substack/node-browserify wraps your CommonJS modules in a bundle loadable in the browser

• connect + browserify = JIT compilation

• Manage your site/app’s dependencies via npm

• Most Node APIs available

• Even our UI is a browserified npm plugin


when.js

• github.com/cujojs/when is a lightweight CommonJS Promises/A+ library with other async goodies (iterators etc)

• We now use Promises everywhere

• used to use github.com/caolan/async

• Bubbling up errors: handle errors in one place

• Also wraps Node functions in Promises

• nodefn.call(fs.readFile, ‘/etc/passwd’).then()


return longTask().then(function(result) {// do stuff with result

}).otherwise(function(error) {})

longTask(function(err, result) {if (err) return err;// do stuff with result

})

Without Promises:

Bubble up Promises:


vfs

• github.com/c9/vfs is a Virtual File System abstraction that works over WS (with msgpack), HTTP and locally in Node

• brings the same FS API to the whole stack; lets us access the fs the same way everywhere; browser to server

• uses github.com/c9/smith, an RPC agent system that gives us streams and processes in the browser

• output logs from long-running processes

• EMFILE: file watching by interval not by fd; github.com/isaacs/graceful-fs


Lessons part 2• Always use jshint in your editor

• Forget callbacks, use and return Promises

• Use browserify and npm to assemble your software

• Design around one thread per process and partition around ~3500m/s type of issues

• Build with grunt; test with mocha and jsdom

• jsdom is fully fledged DOM implementation


Native applications

• github.com/rogerwang/node-webkit lets you build native apps with Node.js and WebKit

• Linux, Mac, Windows all supported

• Even WebGL is supported

• TPOLM demo time! (wrapped in node-webkit)

• http://tpolm.com/demos/inedible_candy


http://tpolm.com/demos/inedible_candy

http://tpolm.com/demos/inedible_candy

Thanks!

[email protected]@gmail.com

github.com/rocketpack

@kschzt


mailto:[email protected]












sc5 meetup: node.js night with jaakko manninen

Technology