intro2 nodejs 2pm

1

Welcome!

Register to Cloud Foundry:Go to http://www.cloudfoundry.com • Click Register - link• Use Promo-code: cloudtoday

Download Node.js• Go to http://nodejs.org/

Getting Started:Go to http://www.cloudfoundry.com • Click “Get Started” menu

http://www.cloudfoundry.com/

http://nodejs.org/


© 2009 VMware Inc. All rights reserved

Node.js Basics: An Introductory training

Raja Rao DVCloud Foundry Developer Advocate (Node.js)@rajaraodv

www.cloudfoundry.com

3

Agenda

1. About Node.js• Internal working of Node.js• Buzz around Node.js• Who is using it• What kind of apps are being built

2. Coding in Node.js• Sync v/s Async coding (Callbacks)• Classes & Modules (CommonJS)• npm & package.json• Node.js EventEmitters

3. Node.js & Cloud Foundry (w/ demo)• Hello World app in Cloud Foundry • Using Sticky Sessions • CloudFoundry Module & connecting to Redis, MongoDB etc. • Express.js (RESTful) app• Socket.io + Express.js (Real-time) app

4

What is Node.js

Node.js is a platform to build fast and scalable network applications. It is built on Google Chrome’s v8 engine & implements event-driven, non-blocking I/O model.

- It is ~80% C/C++ & ~20% JS (APIs)- Uses CommonJS module system.- Executes JavaScript on the server- Built by Ryan Dahl - Sponsored by Joyent

Ryan Dahl(Node.js creator)

5

What is Node.js

6

What is the biggest advantage of Node.js?

Biggest thing Node.js brings to the table (other than JS, of course) is savings in I/O cost

7

The cost of I/O

http://blog.mixu.net/2011/02/01/understanding-the-node-js-event-loop/

8

So how does Node.js save I/O cost?

Node.js saves I/O cost by implementingevent driven, Non-blocking I/O model

9

Event-driven, non-blocking I/O platform/server

Multi-threaded blocking server v/s

Event-driven, non-blocking server

What exactly is a event-driven, non-blocking server? How is it different from a multi-threaded server?

10

Multi-threaded server - Threads are spawned for every connection

Multi threadedserver

T1

T2

T3 T4 T5

T6 T7 T8 T9User4 refreshes 3 times

User1

User2

User3Refreshes 2 times

i/o request

i/o request

DB

BlockingI/O

FS

T

Because every I/o is blocking, server spawns a thread in a thread-pool to support multiple requests

Thread

11

Non-blocking & Evented I/O (Node.js server)

Node.js

User4 refreshes 3 times

User1

User2

User3Refreshes 2 times

i/o requestDB

Non-blockingI/O

FS

POSIXAsync

Threads

t3

t1t2

t4

T1V8

Event loop

Singlethreadserves all users

i/o request

delegate i/o tolibeio

T1V8

T1V8

T1V8

T1V8

T1V8

T1V8

T1V8

V8 Thread running JS code (Single threaded)

t3

t1t2 POSIX threads doing

async I/O (multi-threaded)

i/o result returned 2 EL after x time

Everything except your (JS) code is runs in parallel (by libuv)

T1V8

T1V8

12

Event-driven, non-blocking I/O server

Multi-threaded blocking server (Apache) v/s

Event-driven, non-blocking server (Nginx)

Real-world example of the two designs?

13

Apache V/s Nginx: performance

Ref: http://blog.webfaction.com/a-little-holiday-present

Reqs/sec v/s concurrent connections

At ~4000 concurrent connections, - Nginx can serve ~9000 reqs/sec - Apache can serve ~3000 reqs/sec

14

Apache V/s Nginx: Memory usage

Ref: http://blog.webfaction.com/a-little-holiday-present

Memory v/s concurrent connections

At ~4000 concurrent connections, - Nginx uses 3MB memory- Apache uses 40MB memory

15

Saving I/O is great, what else is happening w/ Node.js?

Let’s look at community, libraries, buzz around Node.js

16

Other things going on for Node.js

2nd most popular watched on git

17


8,000+ libraries/modules/servers

Web frameworksRoutersStatic file serversMicroframeworksFrameworksMiddlewareJSGIConnectOther middlewareOtherDatabaseMS SQL ServerPostgreSQLMySQLSQLite

OracleNoSQL and Key/ValueMongoHiveRedisCouchDBOther NoSQL implementationsMiscellaneous and multiple DBTemplatingCSS EnginesContent Management SystemsBuild and DeploymentPackage Management SystemsModule LoaderOpenSSL / Crypto / HashingSMTPTCP / IP

Multiple protocolsHTTPFTPE-mailXMPPOther networkingRPCWeb Sockets & AjaxMessage QueuesClass systemsTesting / Spec FrameworksWrappersParsersJSONXML

Command Line Option ParsersParser GeneratorsOther ParsersDebugging / Console UtilitiesCompressionGraphicsSoundPayment GatewaysAPI clientsControl flow / Async goodiesI18n and L10n modulesBoilerplatesContinuous Integration ToolsDDD, CQRS, EventSourcingDesktop application relatedJavaScript threadsOther

High-level library categories

https://github.com/joyent/node/wiki/modules



18


Node in Production!• LinkedIn, Yahoo!, Yammer, eBay, Twitter etc.• >1000 other companies/startups are using it in production

All kinds of interesting apps:End-user apps:• Real-time apps • Mobile apps• Web sites • Hardware programming (robots!)

Platform apps (Servers / Services):• Node-http-proxy - Node.js implementation of reverse proxy like nginx• LdapJS.org – - Node.js implementation of LDAP server itself• SMTP – Node.js implementation of SMTP server itself• XMPP, SSH, RPC, many more.

19

Agenda – part 2




20

Let’s look at the code..

Synchronous codev/s

Asynchronous Code

How does async code differ from sync(regular) code?

21

Callbacks – Control flow

Things to note:1. Async code doesn’t directly ‘return’

anything2. Instead, it takes a

function(callback) & calls that function when result becomes available

Use case: Let’s say we have an item’s id and want to get its name from DB and print it

//Synchronous & blocking codefunction getItemNameById(id) { //blocks or waits for DB

return db.get(id); //step 2}

var name = getItemNameById(100); //step 1

//print name in step 3console.log(name); //step 3

//Async & non-blocking code function getItemNameById(id, callback) {

db.get(id, callback); //step 2//nothing is returned here

}

//step 3Some internal function calls the callback w/ result

//You create a callback helper function function displayHelperCallback(name) {

console.log(name); //step 4}

//pass callback function to consume the resultgetItemNameById(100, displayHelperCallback); //step 1

22

Callbacks – Control flow (detailed version in Node.js)

//INTERNALS OF A DB LIBRARY (HIGH LEVEL)

function db() { this.dbConnection = net.connection(); // connects to DB}db.protorype.get = function(id, callback) { var self = this; //step 3 & //step4 is dbConnection.read (below) this.dbConnection.read(id, function(result, callback) { self. receiveFromDB(result, callback);//step 101 }); }

db.protorype.receiveFromDB = function(result, callback) { callback(result); //Execute callback step step 102}

//YOUR APPvar db = require(‘db’);function getItemNameById(id, callback) {

db.get(id, callback); //step 2}

//You create a callback helper function function displayHelperCallback(name) {

console.log(name); //step 103}

//pass callback function to consume the resultgetItemNameById(100, displayHelperCallback); //step 1

//step 5V8 is free to run other functions in the event-loop.

//step 5, step 6 ..step 100Say v8 notices 95 other things to do (in the event loop), it starts executing them one by one.

At some point b/w step 3 and step 100, returns result & asks to run dbConnection.write’s callback.

This event goes to the back of the queue as step 101

Step 5

23

Node.js Programming

Classes & CommonJS module

How can I better organize my code?

24

JavaScript Classes (util.inherits)

Node.js provides handy util.inherits function to inherit a class.- This also provides ‘subclass.super_’ to access super class’ functions

var require(‘util’); //import util module

//Super Classfunction Automobile(license, model) { this.license = license; this.model = model;}

Automobile.prototype.getModel = function() { return model;}

//Sub classfunction Car(license, model) { Automobile.call(this, license, model);}

util.inherits(Car, Automobile);

25

CommonJS modules

//Automobile.js filefunction Automobile(license, model) { this.license = license; this.model = model;}

Automobile.prototype.getModel = function() { return model;}exports.Automobile = Automobile;

//Car.js filevar util = require('util');var module = require('./Automobile');var Automobile = module.Automobile;

function Car(license, model) {Automobile.call(this, license, model);

}

util.inherits(Car, Automobile);

console.log(new Car("1232", "BMW").model); //prints BMW

Things to note:1. Allows keeping JS code in

separate files

2. Use “exports.<name>” to export something

3. Use require(‘path/to/module’) to import it

4. use require(‘module’).<name> to access things inside module

26

CommonJS modules: Exporting multiple things

//myModule.js fileexports.myFunction = function () { return ‘hi there’;}exports.myArray = [‘foo’, ‘bar’];

exports.myVariable = ‘I’m a variable’;

//app.js filevar myModule = require('./myModule');

console.log(myModule.myFunction()); //prints ‘’hi there’console.log(myModule.myArray[1]); //prints ‘bar’console.log(myModule.myVariable); //prints I’m a variable’

Things to note:1. You can directly export function,

arrays, variables

2. You can export multiple things from one file using ‘exports’

27

CommonJS modules: ‘exports’ v/s ‘module.exports’

//myModule.js filemodule.exports = function () { return ‘hi there’;}

//app.js filevar myFunction = require('./myModule');

console.log(myModule.myFunction()); //prints ‘’hi there’

Things to note:If you want to export only one class/function.. so that it can be used directly by the recipient, you can use:module.exports = <something>;

Warning:If you use both module.exports and exports.bla, exports.bla will NOT be exported(ignored)

28

Installing external modules – npm (Node Package Manager)

Use npm (Node Package Manager) to install modulesnpm install <moduleName>e.x. npm install express

Modules are copied into ./node_modules folder /myapp/myapp/node_modules/express

Things to note:1. npm = Node Package Manager

2. It is a CLI to install modules from http://search.npmjs.org

3. LOCAL: npm install express1. It installs in myapp/node_modules/express

4. GLOBAL: npm install express -g 1. It installs in /usr/local/lib/node_modules/ (default)2. Installs executable files in /usr/local/.bin (default)

5. Use GLOBAL when the library has some shell script & want to reuse it for different apps

http://search.npmjs.org/

29

package.json – Describe app & dependencies in a file

//package.json

{ "name": ”MyApp", "description": ”My awesome twitter app", "version": "2.5.8", "author": ”Raja <[email protected]>", "dependencies": { "express": “2.3.4”, "mime": "", "connect-redis": ">= 0.0.1" }}

Things to note:1. If you use package.json, you can

simply do: npm install (w/o any module names)

2. Keep package.json in root directory

3. Using package.json is preferred over individual npm install <module>

4. You need to have all the modulespre-installed (i.e. npm

install)before uploading your app

to Cloud Foundry

30

Node.js EventEmitter (A utility class that allows emitting events)

Node.js EventEmitter

EventEmitter class implements Observer pattern and provides on and emit APIs - It is used when creating (not using) an async library.

31

Events – Node.js EventEmitter (A node.js utility class that allows emitting events)

//Simplified EventEmitter (Observer pattern)

function EventEmitter() { //store events and callbacks like {event1: [callback1, callback2] , event2 : [cb3, cb4]…} this.eventNameAndCallbackList = {}; }

//Allow others to add a callback(function) for a event name(string)EventEmitter.prototype.on = function(eventName, callback) { //add eventName and callback to eventNameAndCallbackList };

//When an event is emitted, call each callbacks in a loopEventEmitter.prototype.emit = function(eventName) { for(var i =0; i < currentCallbacks.length ; i++) {

currentCallbacks[i](); //call each callback }};

exports.EventEmitter = EventEmitter;

32

Events – Node.js EventEmitter (continued)

//myIOModule.jsvar util = require('util');var events = require('events');

//myIOClass is a subclass of events.EventEmitter classvar MyIOClass = function () { events.EventEmitter.call(this);};util.inherits(MyIOClass, events.EventEmitter);

MyIOClass.prototype.readFromDB = function(query){// <--reads data code here -->this.emit('data', "some data");

}exports.MyIOClass = MyIOClass; //export the class

Say you are writing an I/O library & writing readFromDB function but don’t know how to handleasync DB result.

You can solve it by..1. Inheriting your class from EventEmitter 2. Then you can use its ‘emit’ function to an event when data arrives (asynchronously)3. You ask people who’ll be using your library to implement ‘on’ function

33

Events – Node.js EventEmitter (continued)

//app.jsvar myIOModule = require('./myIOModule');

var myIOClass = new myIOModule.MyIOClass();myIOClass.on('data', function (data) { console.log(data);});

myIOClass.readFromDB('select * from users');

Say you are an end-user trying to use DB library to read result from DB..

1. You’ll have to implement ‘on’ function for the given event name (‘data’) and set a callback2. DB libraries internal function will call your callback when the result comes back

34

Events – A library can emit multiple events

I/O libraries usually emit multiple events.. connected, disconnected, error, ready, data, result etc.

//So you can listen to all of them..function myFunction() { db.on(‘error’, function(e) { console.error(e); }); db.on(‘connect’, function() { //db is connected db.query(user); }); db.on(‘disconnect’, function(){ console.log(‘db disconnected’); });

db.connect(‘127.0.0.1’, ‘100’);}

35

Events – Error/Exception handling

//Say there was an exception trying to connect to db. Function () { try { db.connect(‘127.0.0.1’, ‘4000’); // failed to connect; connectionException } catch (e) { console.error(e); } }

Above try/catch won’t handle it because the act of connection itself is an i/o

//Say there was an exception trying to connect to db. Function () { //Typically I/O libraries triggers ‘error’ event (or callback). We’ll need to listen to that event db.on(‘error’, function(e) { console.error(e); });

db.connect(‘127.0.0.1’, ‘100’); // failed to connect; connectionException}

36

Agenda – part 3




37

Cloud Foundry

Cloud Foundry – Open Platform as a Service

38

Cloud Foundry open Platform as a Service

The PaaS of choice for the Cloud era

Simple• Let’s developers focus on their code and not wiring middleware

Open• Avoid lock-in to specific cloud, frameworks or service

• Completely open source from day one

Flexible and Scalable• Self service, deploy and scale your applications in seconds

• Extensible architecture to “digest” future cloud innovation

39

Cloud Foundry open PaaS - Choice of frameworks

OSS community

40Application Service Interface

Data Services

Other Services

Msg Services

Cloud Foundry open PaaS - Choice of application services

vFabric Postgres

vFabric RabbitMQTM

Additional partners services …

41Application Service Interface

Data Services

Other Services

Msg Services

Cloud Foundry open PaaS - Choice of clouds

Private Clouds

PublicClouds

MicroClouds

.COM

PartnersCl

oud

Prov

ider

Inte

rface

Avoid

Lock-in

42

Node.js & Cloud foundry (Demos and Examples)

43

Hello World App on Cloud Foundry

//Simple http server on localhostvar http = require('http');http.createServer(function (req, res) { res.writeHead(200, {'Content-Type': 'text/plain'}); res.end('Hello World\n');}).listen(3000, '127.0.0.1');console.log('Server running at 127.0.0.1:3000');

//Simple http server on Cloud Foundryvar http = require('http');var host = process.env.VCAP_APP_HOST || ‘localhost’;var port = process.env.VCAP_APP_PORT || ‘3000’;http.createServer(function (req, res) { res.writeHead(200, {'Content-Type': 'text/plain'}); res.end('Hello World\n');}).listen(port, host);console.log('Server running at ’ + host + “:” + port);

Things to note:1. Cloud Foundry will provide host and port info in process.env variable

2. You can use https to access your app (up to nginx; http w/in CF)

3. Save your app as app.js or server.js to automatically use that as main node.js

44

“cloudfoundry” module & Connecting to MongoDB, Redis

Connecting to services

45

“cloudfoundry” NodeJS helper module

npm install cloudfoundry

var cloudfoundry = require(‘cloudfoundry’);

cloudfoundry.cloud //is running on Cloud Foundry?cloudfoundry.host // hostcloudfoundry.port // port

//Example: Say you are using ‘test-mongodb’ MongoDB service, you can get its info:cloudfoundry.mongodb['test-mongodb'].credentials.hostnamecloudfoundry.mongodb['test-mongodb'].credentials.portcloudfoundry.mongodb['test-mongodb'].credentials.dbcloudfoundry.mongodb['test-mongodb'].credentials.usernamecloudfoundry.mongodb['test-mongodb'].credentials.password

Things to note:1. Cloudfoundry module (built by ‘igo’) provides easy access to environment variables

2. For more, please go through https://github.com/cloudfoundry-samples/cloudfoundry

https://github.com/cloudfoundry-samples/cloudfoundry


46

“cloudfoundry” NodeJS helper module

npm install cloudfoundry

var cloudfoundry = require(‘cloudfoundry’);

cloudfoundry.cloud //is running on Cloud Foundry?cloudfoundry.host // hostcloudfoundry.port // port

//Example: Say you are using ‘test-mongodb’ MongoDB service, you can get its info:cloudfoundry.mongodb['test-mongodb'].credentials.hostnamecloudfoundry.mongodb['test-mongodb'].credentials.portcloudfoundry.mongodb['test-mongodb'].credentials.dbcloudfoundry.mongodb['test-mongodb'].credentials.usernamecloudfoundry.mongodb['test-mongodb'].credentials.password

Things to note:1. Cloudfoundry module (built by ‘igo’) provides easy access to environment variables

2. For more, please go through https://github.com/cloudfoundry-samples/cloudfoundry



47

MongoDB – Example of inserting a user (using mongodb-native module)

var mongodb = require('mongodb').Db;var conn; // holds connection

//connect to db and get connection obj//connectionUrl looks like mongodb://username:pwd@host:port/dbNamemongodb.connect(connectionUrl, function(err, connection) {

conn = connection;});

//add a user function addUser (userObj, callback) {

//Get the collection that holds usersconn.collection('users', function (err, userCollection) { //insert user to this collection

userCollection.insert(userObj, {safe:true}, function(err) {callback(userObj);

}); });

}

//PS: Error handling is not shown

48

Demo app

Things to note:1. Simple MongoDB demo app, adds random users and pulls existing users

2. https://github.com/rajaraodv/mongoapp1

https://github.com/rajaraodv/mongoapp1

https://github.com/rajaraodv/mongoapp1

49

ExpressJS

50

Hello World App using ExpressJS

var host = process.env.VCAP_APP_HOST || 'localhost';var port = process.env.VCAP_APP_PORT || '3000';

var express = require('express');var app = express.createServer();

app.listen(port, host);

Things to note:1. ExpressJS is Ruby Sinatra

inspired web framework

2. It is built on top of ‘Connect’ – which itself is a wrapper for http-module

3. It provides support for:1. Dev/Prod Configurations2. Routes3. Templating4. Sessions5. Many, many other

features

51

Hello World App using ExpressJS (Middlewares)


//Middlewaresapp.use(express.logger()); //logs requestsapp.use(express.static(__dirname + ‘/public’)); //sets location of public filesapp.use(express.bodyParser()); //parses HTTP POST body

Things to Note:1. Middlewares are functions that helps in common tasks involved

building in web applications2. They are actually connect-module functions but exposed by

ExpressJS for simplicity

52

Hello World App using ExpressJS (Environments)


app.configure('development', function() { //On error, print exceptions to console & to the web-page itself app.use(express.errorHandler({ dumpExceptions: true, showStack: true }));});

app.configure('production', function() { //On error, this simply says ‘Internal error Occurred’

app.use(express.errorHandler({ dumpExceptions: true, showStack: false }));

app.use(express.logger()); //logs requests});

Things to Note:1. Express uses NODE_ENV environment variable to set working

environment2. On CF, to toggle b/w ‘development’ and ‘production’, you can use.. vmc env-add <appName> NODE_ENV ‘production’

53

Hello World App using ExpressJS (Routing)

var express = require('express');var app = express.createServer();app.use(express.bodyParser());

//Receive HTTP GETapp.get('/user', function(req, res) { //Don’t let people call to /user throw new Error(“I’m private. Call me w/ a user id");});

app.get('/user/:id', function(req, res){ res.send('user ' + req.params.id);});

//Receive HTTP POSTapp.post(‘’/register”, function(req, res) { //(Use bodyParser middleware for this) var body = req.body; db.save(body.user, body.password);}

Things to note:1. You can use Routing to

listen to requests to call different functions

2. You can listen to HTTP POST, PUT etc.

54

Hello World App using ExpressJS (Sessions)

Things to note:1. To create sessions, use cookieParser & session middlewares

2. By default Express uses MemoryStore to store sessions

3. You can use Redis to store sessions


//Middlewaresapp.use(express.static(__dirname + ‘/public’)); //sets location of public filesapp.use(express.bodyParser()); //parses HTTP POST bodyapp.use(express.cookieParser()); //Parses cookies headersapp.use(express.session({secret: 'your secret here}));

55

ExpressJS (Sticky sessions for multi instances)

Things to note:1. Sticky Session is a reverse proxy / load balancer feature to help persistent

connection

2. When Sticky Session is on, request goes from Nginx to the same instance no matter how many instances of your app is running .

3. Cloud Foundry’s Nginx provides Sticky Sessions on ‘jsessionid’ cookie

4. W/o setting this requests are randomly sent to different instances & you’ll have to use external store like Redis to fetch session data (recommended).


//Middlewaresapp.use(express.static(__dirname + ‘/public’)); //sets location of public filesapp.use(express.bodyParser()); //parses HTTP POST bodyapp.use(express.cookieParser()); //Parses cookies headersapp.use(express.session({secret: 'your secret here’, key: ‘jsessionid’ }));

56

ExpressJS (demo)

57

ExpressJS demo app screenshots (routing, sessions & sticky sessions)

For more: https://github.com/rajaraodv/express1

Demo Explains how session, sticky sessions, routing etc. works

https://github.com/rajaraodv/express1



58

Socket.io on Cloud Foundry

Socket.io

59

Socket.io on Cloud Foundry (server side)

var sio = require('socket.io');var express = require('express');var app = express.createServer();

var io = sio.listen(app);//listen to expressio.configure(function() { io.set('log level', 1); io.set("transports", ["xhr-polling"]); //Currently CF doesn’t support websockets});

Things to Note:1. Socket.io is mainly used to build real-time apps

2. Socket.io provides a single interface to switch b/w various transport techniques like xhr-polling, websocket, JSONP etc

3. In addition, it provides heartbeats, reconnection, timeouts etc. that are vital for real-time apps.

4. It works seamlessly with ExpressJS

60

Socket.io on Cloud Foundry (server side continued)

//After listening to express..wait for connection from browser

io.sockets.on('connection',function(socket) { // When the client/browser emits 'sendchat', this listens and executes socket.on('sendchat', function(data) { // We will echo it back to ALL sockets io.sockets.emit('updatechat’, data); });});

61

Socket.io on Cloud Foundry (client side)

<script src="/socket.io/socket.io.js"></script>//socket.io serves this file from server

var socket = io.connect(document.location.href); //connect to the server

// on connection socket.on('connect', function() {

console.log("client connected"); });

// Whenever the server emits 'updatechat', this updates the chat body socket.on('updatechat', function (data) { $('#conversation').append(data); // append it to my list });

//When the user enter some data, send it to server function sendchat() { var message = $('#chatField').val();

// Emit or tell server to execute 'sendchat’ socket.emit('sendchat', message);

}

62

Socket.io (+ ExpressJS) Demo app screenshots

For more: https://github.com/rajaraodv/socketio1

https://github.com/rajaraodv/socketio1



63

Summary




64

Next Steps

Register to Cloud Foundry:Go to http://www.cloudfoundry.com • Click Register - link• Use Promo-code: cloudtoday

Download Node.js• Go to http://nodejs.org/

Getting Started:Go to http://www.cloudfoundry.com • Click “Get Started” menu


http://nodejs.org/


65

Questions?

@rajaraodv (github.com/rajaraodv)@cloudfoundry (github.com/cloudfoundry)

Questions?

intro2 nodejs 2pm

Technology