introduction to node.js: perspectives from a drupal dev

Introduction to Node.jsPerspectives from a Drupal dev

Mike Cantelon, Vancouver Drupal Users Group, Nov. 25, 2010

Thursday, November 25, 2010

I am:

A longtime PHP dev

Devop for The Georgia Straight

Experimenting with JS/SSJS and HTML5

http://mikecantelon.com

http://github.com/mcantelon


http://drupal.org/node/372396




WHY NODE.JS?


wordsquared.com: Real-time HTML/JS Scrabble MMO


The real-time web

Real-time applications are an interesting web subset

Examples: Twitter, Etherpad, games, monitoring

Real-time paradigm will spawn new types of apps


Three barriers to real-time

Conventional languages can be slow

Conventional web servers can be slow

Conventional databases are slow


Node.js eliminates two

Conventional languages can be slow

Conventional web servers can be slow



http://www.slideshare.net/Vodafonedeveloper/nodejs-vs-phpapache

Node.js vs PHP/Apache




So what exactly is Node.js?

Server-side Javascript (SSJS) implementation

“Asynchronous” (more on that later)

Built using V8 (JS engine used in Chrome Browser)


Advantages over PHP

Javascript is a cleaner language

Asynchronous execution increases performance

Node.js is suitable for writing TCP/IP apps


“Asynchronous”?

Also known as “event-based”

Think of it as defining actions triggered by events

CPU spends less time waiting around


More benefits of SSJS

Requires less mental context switching for devs

Allows sharing of logic between client and server side

Server-side JQuery for screen scraping? Yes!


What about that last barrier?

Conventional languages are slow

Conventional servers are slow



MongoDB is one solution

MongoDB is generally faster than SQL databases

MongoDB queries are written in Javascript, not SQL

MongoDB works well with node.js


And Drupal?

Drupal is great for managing content

Node.js has no killer CMS

Node.js can pull data from Drupal (details later)


This looks conceptually like...


SSJS HELLO WORLD


Node.js isn’t hard to learn

Knowing Javascript is the hard part

Node.js is accessible

Express is a framework that makes Node.js easier


Hello World in Node.js shell

var sys = require('sys');sys.puts('Hello world');


Node.js server Hello World

var http = require('http');

http.createServer(function (req, res) { res.writeHead(200, {'Content-Type': 'text/plain'}); res.end('Hello World\n');}).listen(8124, "127.0.0.1");


Hello World in Expressvar express = require('express');var app = express.createServer();

app.get('/', function(req, res){ res.send('Hello World');});

app.listen(8124);


Why use Express?

Handles routing and redirection

Nice templating support

Provides sessions


Express Templating Support

Supports EJS, HAML, SASS, Jade (HAML variant)

Extend with app-specific helper functions

Provides “partials” (subtemplates)


app.get('/', function(req, res){ res.render('index.haml', { locals: { title: 'My Site' } });});

Templating Example


SETTING UP NODE.JS


Installing Node.js

OS X: build from source or brew install node

Linux: build from source

Build instructions at http://nodejs.org/#download


Installing NPM

NPM lets you easily install node.js modules

OS X: brew install npm

Instructions for manual install:http://mikecantelon.com/npm


http://mikecantelon.com/story/installing-npm-nodejs-package-manager

http://mikecantelon.com/story/installing-npm-nodejs-package-manager

Installing Express

npm install express

Now the command express will set up an example Express app for you


Installing MongoDB

OS X: build from source or brew install mongodb

Linux: http://www.mongodb.org/display/DOCS/Quickstart+Unix

npm install mongodb


http://www.mongodb.org/display/DOCS/Quickstart+Unix




DRUPAL -> NODE.JS


Drupal as a datastore

One way Node.js and Drupal can work together is having Drupal be the storehouse for important content requiring long-term management


Something like...


Serve data, not just pages

For realtime apps you’re going to want to serve chunks of data to the browser rather than whole pages

Whole pages take longer to render and use more bandwidth to transmit

Leverage the browser as much as possible


Serving a page...

1. Browser requests a URL2. Server grabs some data from a database and dresses it up as a page using a template.3. Whole page gets sent back to browser


...vs Serving Data

1. Browser requests some data2. Server sends back data3. Browser templates data and outputs HTML into the existing page


Where to serve data from

Pulling Drupal data directly to the browser is quick to implement, but puts more stress on your Drupal stack

The alternative is to pull Drupal data to node.js and cache using a datastore like MongoDB

Data can then be relayed via AJAX or Socket.io


DRUPAL DATA SHARING


Javascript For Data Sharing

JSON is the duct tape of the web

drupal_to_js turns any chunk of data into JSON

Drupal Views can output JSON via Views Datasource


What JSON looks like

{ 'drupal': { 'language': 'PHP', 'license': 'GPL', 'developed_by': { 'individuals', 'organizations', 'companies' } }}


Drupal pumping out JSON


Example: RSS to JSON via Views


What our example will do

Use the Aggregator module to pull data from an RSS feed (specifically “Drupal Planet”)

Use Views, via the Views JSON module, to publish the aggregator items as JSON data


Install Views Datasource


Add View


Add Page Display and Path


Add Paging


Add Fields


Set Style to JSON


Set Feed/Category ID


Save View and Check Path


Demo use of Drupal/JSON...

[demo]

http://github.com/mcantelon/Drupalurk




Hack for Paging (v6 beta2)

function mytheme_preprocess_views_views_json_style_simple(&$vars) { global $pager_total, $pager_page_array; $element = $vars['view']->pager['element']; $vars['rows']['pager'] = array( 'total' => $pager_total[$element], 'current' => $pager_page_array[$element] );}

http://gist.github.com/581974

Views Datasource needs theme tweak to make paging work

Stick the snippet below into your theme’s template.php


http://gist.github.com/gists/581974/edit

http://gist.github.com/gists/581974/edit

Hack for Paging (v6 beta2)

This enables you to add &page=<page number starting at 0> to JSON calls

You can then implement your own JS paging


Possible JSON View Uses

Pull front-page content

Pull content by taxonomy

Pull recent comments

Whatever else you can do with a view


Pulling JSON into Node.js

var sys = require('sys'), rest = require('restler-aaronblohowiak'), item, node

rest.get('http://mikecantelon.com/jsontest/News') .addListener('complete', function(data) {

for(item in data.nodes) { node = data.nodes[item].node sys.puts(node.Title) sys.puts(node.Body) }})


restler module allows easy HTTP JSON requests


http://mikecantelon.com/jsontest/News'

http://mikecantelon.com/jsontest/News'



TALKING TO MONGODB


Things to remember

Can create databases and schema on-the-fly

Queries are Javascript

A “collection” is similar to an SQL DB’s “table”

MongoDB has a shell so easy to experiment


Doing something to data

1. Open server DB connection2. Open database3. Open collection4. Do something to the collection


Opening server DB connection

var mongo = require('mongodb');var port = mongo.Connection.DEFAULT_PORT;var db = new mongo.Db( 'classroom', new mongo.Server('localhost', port, {}), {});


Opening database

db.open(function(err, db) { // stuff gets done in here});


Opening a collection

db.collection( 'students', function(err, collection) { // collection operations here });


Removing from a collection

collection.remove( function(err, collection) { // records are now gone! });


Adding some records

// make four students with random agesvar names = ['Rick', 'Jane', 'Bob', 'Lisa'];

var index;for (index in names) { collection.insert({ 'name': (names[index]), 'age': Math.round(Math.random(4)*10)+18 });}


Displaying data from records

// display namescollection.find({}, {}, function(err, cursor) { cursor.toArray( function(err, students) { var index; for(index in students) { sys.puts(students[index].name); }; } ); });


DEPLOYING NODE.JS


Node.js isn’t a daemon

Like PHP, node.js doesn’t run as a daemon

This means that naive deployment would require keeping a terminal open

There are a number of ways to deal with this...


Upstart (Ubuntu/some other distros)

Make a config file for each app in /etc/init

Name config file my_app.conf and set as executable

Start app using: sudo start my_app


Installing Upstart

Ubuntu: apt-get install upstart

Linux: http://upstart.ubuntu.com/download.html


http://upstart.ubuntu.com/download.html

http://upstart.ubuntu.com/download.html

System V init script (the right way)

rc.local/custom script (quick and dirty)

Gnu screen (for temporary deployment)

No Upstart? (Centos/some other distros)


Requires adding a line to /etc/rc.local and a script

Example line in rc.local triggers script at startup:su - mike -c "/home/mike/drupalchat/RUN.sh" &

Script should set NODE_PATH if you’re using npm

rc.local/custom script


Example script

#!/bin/bashcd /home/mike/drupalchatexport NODE_PATH=/home/mike/root/usr/local/bin/node server.js


Hotnode will restart node.js when you change a file

Start app with hotloading: hotnode my_app.js

Installation: npm install hotnode

Hotnode


BONUS! WEBSOCKETS


What are WebSockets?

WebSockets are an “HTML5” technology for two-way TCP/IP textual communication

Modern browsers, like Chrome and Firefox 4, offer a Javascript WebSockets interface


Why not just use AJAX?

AJAX is a hack that uses HTTP for communication

HTTP request/response cycle has a lot of overhead

WebSockets increase speed, lessen bandwidth use


Implementing with Socket.io

Socket.io is a client/server websocket solution, using Node.js for the server side

The client library tries WebSockets, but falls back to other mechanisms when talking to old browsers


Questions? Ideas?


Flickr Credits

http://www.flickr.com/photos/sidehike/459483568/http://www.flickr.com/photos/wysz/86758900/http://www.flickr.com/photos/generated/291537716/http://www.flickr.com/photos/gsfc/3720663082/http://www.flickr.com/photos/skreuzer/354316778/http://www.flickr.com/photos/ubookworm/455760111/http://www.flickr.com/photos/batega/1596898776/http://www.flickr.com/photos/batigolix/3778363253/http://www.flickr.com/photos/wwworks/4472384764/http://www.flickr.com/photos/19779889@N00/4398186065/http://www.flickr.com/photos/theplanetdotcom/4878809615/http://www.flickr.com/photos/estherase/177188677/


http://www.flickr.com/photos/sidehike/459483568/

http://www.flickr.com/photos/sidehike/459483568/

http://www.flickr.com/photos/wysz/86758900/

http://www.flickr.com/photos/wysz/86758900/

http://www.flickr.com/photos/generated/291537716/

http://www.flickr.com/photos/generated/291537716/

http://www.flickr.com/photos/gsfc/3720663082/

http://www.flickr.com/photos/gsfc/3720663082/

http://www.flickr.com/photos/skreuzer/354316778/

http://www.flickr.com/photos/skreuzer/354316778/

http://www.flickr.com/photos/ubookworm/455760111

http://www.flickr.com/photos/ubookworm/455760111

http://www.flickr.com/photos/batega/1596898776/

http://www.flickr.com/photos/batega/1596898776/

http://www.flickr.com/photos/batigolix/3778363253/

http://www.flickr.com/photos/batigolix/3778363253/

http://www.flickr.com/photos/wwworks/4472384764/

http://www.flickr.com/photos/wwworks/4472384764/

http://www.flickr.com/photos/19779889@N00/4398186065/

http://www.flickr.com/photos/19779889@N00/4398186065/

http://www.flickr.com/photos/theplanetdotcom/4878809615/in/photostream/

http://www.flickr.com/photos/theplanetdotcom/4878809615/in/photostream/

http://www.flickr.com/photos/estherase/177188677/

http://www.flickr.com/photos/estherase/177188677/

Resources

Node.js, Express.js and MongoDBhttp://nodejs.org/http://expressjs.com/http://www.mongodb.org/

Socket.iohttp://socket.io/

This presentationhttp://mikecantelon.com/drupal-nodejs


http://nodejs.org

http://nodejs.org

http://expressjs.com

http://expressjs.com

http://www.mongodb.org

http://www.mongodb.org

http://socket.io

http://socket.io

http://mikecantelon.com/drupal-nodejs

http://mikecantelon.com/drupal-nodejs

introduction to node.js: perspectives from a drupal dev

Technology

express var express

express npm

slow conventional databases

slow conventional servers

slow conventional web

server hello world var

command express

accessible express