Your first app: node.jsA tutorial for real-world application development using Node.js,AngularJS, Express and MongoDB

Jim Schubert

This book is for sale at http://leanpub.com/yfa-nodejs

This version was published on 2017-11-29

This is a Leanpub book. Leanpub empowers authors and publishers with the Lean Publishing process. LeanPublishing is the act of publishing an in-progress ebook using lightweight tools and many iterations to getreader feedback, pivot until you have the right book and build traction once you do.

© 2013 - 2017 Jim Schubert

Tweet This Book!Please help Jim Schubert by spreading the word about this book on Twitter!

The suggested tweet for this book is:

I just bought Your First App: Node.js from leanpub.com!

The suggested hashtag for this book is #yfa-nodejs.

Find out what other people are saying about the book by clicking on this link to search for this hashtag onTwitter:

#yfa-nodejs

Contents

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Preview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3Who this book is for . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3About the author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6Terminal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8JavaScript Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8Node.js Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14Non-blocking I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

PrefaceYou’re reading the first draft of Your First App: node.js. As reader questions and comments come in, I’ll beupdating the contents of the book.

I look forward to reader feedback. Please email me at yfanodejs@gmail.com with questions or comments.

PreviewThis is a preview of “Your First App: Node.js”. If you’d like to purchase a full copy, please visithttps://gumroad.com/jimschubert.

IntroductionIf you want to learn how to write a quality application using node.js and various related tools, you’re lookingat the right book. This book aims to be a complete introduction to writing a fully functional application usingnode.js. It is not meant to be an in-depth romp through JavaScript or node.js internals. If that’s the type ofbook you’re looking for, then this is the wrong book.

You’ll find numerous books on the subject of node.js, but few (if any) provide start-to-finish details regardingtrue application concerns. For example, youmay find a book aboutwriting a node.js applicationwithmongodbor a book about a node.js application using Backbone.js, but these books only focus on a narrow subset of theapplication lifecycle. Rather than being another 200-page blog article compiled into book format, this bookaims to get you on your way in the application life cycle within a very specific technology stack. In otherwords, knowing how to write code using a specific technology is only part of the real-world concerns ofcreating and maintaining an application. This book attempts to address some of these real-world concernsthat are commonly left out of other similar texts.

Because of the nature of this book, focusing on the process of writing a node.js application, therewill be many times where you’ll need filler information.

Just as you would in a professional environment, you’ll need to research any missing content. I’lldo my best explain the basics, then point you to helpful resources for further reading.

Who this book is for

This book is intended for professionals or students who have a comfortable working knowledge of JavaScript,HTML, CSS, and the stateless nature of HTTP. If you’re not comfortable with JavaScript, I highly recommendreading both JavaScript: The Definitive Guide1 and JavaScript Garden2. Although I’ll quickly go over somenecessary fundamentals for beginners, it will be difficult to become a ‘node.js developer’ without a solidfoundation in JavaScript. HTML, CSS, and client-side JavaScript will be necessary for the presentation of yourapplication, while understanding the stateless nature of HTTP will make some of the techniques discussed inthis book easier to learn.

I like to believe that developers who pick up a book like this tend to learn well from examples and hands-oncoding. Hell, I’m writing this book for people like me. I make a lot of assumptions about your technologicalcapability since this book is written for new and experienced software engineers. With that in mind, if anexample says something like ‘download Twitter Bootstrap,’ I assume you know how to do this (or can figureit out) and I won’t write out the steps. I don’t know you, but since we’ve already established that you andI are essentially the same, I’ll skip some pretty basic stuff. I know you don’t want 50 pages of installationinstructions that you can just read in 3 paragraphs on a vendor’s site. You’re here for content and experienceand that’s what I’ll give you.

1http://oreil.ly/18GKUWk2http://bit.ly/18GLa7O

Introduction 4

About the author

My name is Jim Schubert and I am a Software Engineer currently working at IBS, Inc. in Richmond, VA.My wife and I had a beautiful baby boy named Jack on November 12, 2013 (yes, 11/12/13!). We moved backto Virginia after a year and a half in Seattle, where I worked at Expedia on a node.js API. I enjoy API andfront-end development. Prior to Expedia I worked at Enghouse Interactive on a product called CallRex, whichwas a call center call recording software suite. I was the maintainer of the SDK and documentation, workedon the API (C++ and WCF services), rewrote the installers using WiX, and created an MSBuild framework ontop of VMWare’s VIX API to automate the installation, packet playback, and testing of the product.

I ‘began coding’ in 1988 when my mother brought home an old desktop machine (possibly running on a486DX). This machine only had a text-based console interface. It also had a 5.25” floppy of BASIC. I wroteout some spaghetti code, learning only from the BASIC documentation. I loved that I could write code whichcould execute and do something fun or valuable, so I’ve been doing that as a hobby ever since.

Around 1995, my family bought our first desktop computer and connected to the internet with AOL. I spent alot of time doing what I called ‘making web pages’, which just meant learning how to display text and imageswith HTML. Eventually, I began incorporating CSS and some JavaScript. In 1998 or 1999, I was running myown PHP-nuke portal to host a personal blog, image gallery, and a custom module I wrote for displayingtutorials about programming or image editing. At some point around this same time, I began learning VisualBasic and Delphi (Kylix). I was somewhat enthralled by the ‘hacker scene’ back then and decided to solve aproblem with AOL’s IM chat: enabling away messages. I used a pretty popular module called dos32.bas tocreate an away messaging system in AOL 4.0/5.0. The app wasn’t hugely popular, but I had quite a few peopleusing the app and requesting features. Later, an online friend and I wrote an HTTP server and FTP server inVisual Basic following the RFC of each protocol as our requirements. That was also a lot of fun.

In 2005, I decided to attend Virginia Commonwealth University in Richmond so I could turn my hobby intoa career. I did just that.

In my senior year of VCU, I did a semester internship with a company that was acquired halfway through theinternship by GE Healthcare. Youmay have seen commercials for the software I worked on; Agent Smith fromTheMatrix walks around an empty hospital explaining how software turns a ‘waiting room’ into just a ‘room’.That application was my deep-dive into changing my coding hobby into a career. I learned a lot in those fewmonths about ASP.NET, C#, SQL Server, HL7 compliance, Agile/Scrum, and a lot of other technologies. Theapplication itself coordinated the technological processes of maintaining complete ‘awareness’ of patients,staff, and equipment using RFID. Although learning how to write actual code was useful, I believe what Igained the most out of the experience was understanding how people and processes play a major factor in theapplication equation. Starting my career with this very basic insight has actually given me a leg up on otherengineers of my level in the workforce; I’ve been told I’m able to see the “big picture” a little more clearly.This is something I hope to convey with this book.

More recently, I’ve been working on multiple internal APIs at Expedia. Two of these APIs are written innode.js with a clear goal of being scalable and highly available. I’ve played a large part on my team in drivingprocesses and best practices to improve code quality and performance for all of the APIs I work with. Oneof the node.js APIs acted as a consumable facade on top of a legacy system. I also lead efforts on a team torework the legacy system into an application stack of Scala and AngularJS, which was meant to eventuallyeither replace or better support the node.js facade. During that time, I also drove team training on git workflowas well as Agile processes. It was ultimately a life changing experience.

Introduction 5

I’ve also forked a book by TJ Holowaychuk called Mastering Node and contributed quite a bit to it. You cansee my changes on github at jimschubert/masteringnode3. You can also download the text or read it online athttp://jimschubert.github.io/masteringnode/4. I abandoned work on the Mastering Node book because I hadwritten so much and changed the build process so drastically without my changes being merged upstream,and I didn’t want to have two such drastically divergent versions of the same text. I had also heard that TJwas working on a node.js book to be released by Manning.

I decided to take a break from writing and investigate different book publishing platforms for either freely-available or inexpensive technology books. There are technologies like git scribe5, sphinx6, or pure DocBook7.I have a love for markdown, and I was toying around with the idea of writing a markdown-to-ebook utilitywhen I found Leanpub8. Leanpub offers a lot more than I was looking for:

1. Markdown text2. Iterative pubishing strategy3. A marketplace4. Source code includes and other markdown extensions5. Multiple formats: pdf, ebook, mobi

This is my first publishing experience on Leanpub. I welcome all questions and comments about the book atnodejs@ipreferjim.com.

Code

Code examples in the book will resemble the following example.

This code is an Animal!

1 var Animal = function Animal() {

2 this.species = null;

3 this.immortal = false;

4 this.move = null;

5 this.sound = null;

6 }

The above code is inline, meaning it is code that is neither intended to be a self-contained executable examplenor is it part of a larger executable example. Some code will be available in the accompanying source code.This code will display the relative path to the code:

3http://bit.ly/196RlPp4http://bit.ly/196RrGy5https://github.com/schacon/git-scribe6http://sphinx-doc.org/7http://www.docbook.org/8https://leanpub.com/

Introduction 6

Beginner code in ruby:code/basics/beginner.rb

#!/usr/bin/env ruby

puts "What is your name?"

name = gets

puts "Hello #{name.chomp}!"

Examples

Many examples throughout this book are available as companion code. The code can be downloaded fromgithub repository: jimschubert/yfa-nodejs-code9.

To obtain a copy of the example code, you can click the Download ZIP link on the repository’s page. Or, ifyou already have git installed, you can clone the repository.

$ mkdir ~/samples && cd ~/samples

$ git clone git@github.com:jimschubert/yfa-nodejs-code.git

$ cd yfa-nodejs-code

If you don’t already have git installed, got to http://git-scm.com/10 and follow installation instructions. You’llneed git in future chapters, so it doesn’t hurt to jump ahead.

Terminal

There will be many instances in this text where you’ll need to type some commands into a terminal. Myterminal’s shell is Bash 4.2.45, so all commands should execute perfectly fine in all Bash 4.x environments.For short commands, I may ask you via inline text to “execute ls -la in your terminal”. For longer examples,the executable code will be preceded by a $ symbol to indicate the bash shell. Standard output and standarderror will be displayed in this text just as it would be displayed in your terminal. For example, rather than askyou to “execute grunt --help in a terminal” I may display the entire command and output:

$ grunt --help

grunt-cli: The grunt command line interface. (v0.1.9)

If you're seeing this message, either a Gruntfile wasn't found or grunt

hasn't been installed locally to your project. For more information about

installing and configuring grunt, please see the Getting Started guide:

http://gruntjs.com/getting-started

9https://github.com/jimschubert/yfa-nodejs-code10http://git-scm.com/

Introduction 7

When you need to execute server-side JavaScript code directly, after installing node.js you can run node

directly from the command line. You’ll be dropped into a REPL, which stands for Read-Eval-Print-Loop andis a fancy acronym for an interactive shell. In a node REPL, you can execute code and get immediate feedback.A REPL isn’t a special node.js concept, Ruby has irb, Scala has its own REPL, even C# has a REPL if you useMono.

Feedback

If you have questions or comments related to the book, there are two ways to communicate with me. You cango the old school route and email me at nodejs@ipreferjim.com. This is the preferred method if you don’t feelcomfortable with asking a question or making a comment publicly.

You can also open an issue on the GitHub repository (https://github.com/jimschubert/yfa-nodejs-code). If youdon’t have a GitHub account, I highly recommend creating one. Don’t worry, it’s free. GitHub is not only aplatform for hosting and sharing code, it’s a collaborative platform which enhances the underlying revisioncontrol system called ‘git’.

Follow the instructions at GitHub Help: Creating an issue11 for creating an issue. This article is hosted andmaintained by GitHub, so it should remain current as GitHub’s user interface or workflow changes.

âš̂ Please limit reporting issues to actual issues with the book’s text or the accompanying source code. Youcan even open an issue to say ‘Love the book!’ if you want.

X Please don’t ask questions about unrelated projects or problems you’re having with node.js code frompersonal projects unless your problem is directly related to something from the book. For general questionslike these, you should post a question on StackOverflow12. StackOverflow is a huge community of softwareprogrammers with a shared goal of improving software. Sometimes your code needs a second pair of eyes andStackOverflow can provide it. Just be sure to do a Google search on your problem or search StackOverflowto see if others have had similar issues.

11http://bit.ly/1tXEY6S12http://stackoverflow.com/

BasicsIn case you skipped the introduction, let me say again that I highly recommend reading both JavaScript: TheDefinitive Guide and JavaScript Garden13.

The goal of this book is not to teach you how to be a JavaScript developer, but I will cover some fundamentalconcepts just in case you’re trying to hit the ground running.

Now is a good time to install node.js. I am using v0.10.2.

If you haven’t yet, follow the installation instructions at http://nodejs.org14.

JavaScript Basics

JavaScript is a dynamic language. Twenty years ago when JavaScript was born, there were radically differentbest practices and standards. Thankfully, the language has grown in that time, but there are still some weirdthings to consider. This section is not meant to be a ‘how to write JavaScript’ introduction; the goal is to callout specific parts of JavaScript that will be beneficial to your understanding of node.js.

Prototypal

A lot of people seem to have issues coming to a JavaScript or node.js coding environment from a classicallanguage background. A classical language is one in which a class can derive from another class and aninstantiation of that class chain is called an object. In a classical language, the class constructor(s) or theobject itself maintain all behavioral logic of the object. In a prototypal language15, objects inherit from otherobjects. This means that an object can contain not only the behaviors of other objects, but essentially derivefrom another fully constructed object. You might think this sounds complex, but prototypal inheritance opensthe door for fun features. It also allows for some annoying bugs if implemented improperly. This forces thecommunity to create interesting standards and patterns to help us avoid those bugs. If you’re not familiarwith prototypal languages, be sure to read the Wikipedia article16 on the subject for a basic understanding.

Objects

In JavaScript, prettymuch everything is an object. Even literals (e.g. var x = 100;) are special types of objects.Functions are also special types of invokable objects.

13JavaScript: The Definitive Guide is available at http://oreil.ly/18GKUWk and JavaScript Garden is at http://bit.ly/18GLa7O.14http://nodejs.org15see Wikipedia’s article at http://bit.ly/14BvTAP and http://bit.ly/14BwhiR for more information about ‘Prototypal Inheritance in JavaScript’.16see Wikipedia’s article at http://bit.ly/14BvTAP and http://bit.ly/14BwhiR for more information about ‘Prototypal Inheritance in JavaScript’.

Basics 9

According to MDN, Every function in JavaScript is actually a Function object17. Some will arguethat all objects are functions because you have to call a constructor function to instantiate mostobjects. You don’t even have to explicitly call a constructor function. Throw the following line inthe node REPL when you get a chance:

{} instanceof Object

Don’t get hung up on arguments in the JavaScript community or you’ll get nowhere.

An object itself in JavaScript is just an associative array (or amap, if you prefer). It all boils down to a collectionof keys which each point to a value of some sort. This is often forgotten by newer members of the communitybecause of the helpful dot-notation. For example, look at the following object literal which assigns a value toa key in dot notation:

var x = {};

x.someValue = 100;

Now, consider that an object is just a container of key,value pairs. If you wanted to store the result of n2, youmay do so like this:

var x = {};

x['n ** 2'] = 4;

Realize, however, that using non-contiguous and non-alphanumeric text for keys can make the usage of yourobject difficult. For a more detailed explanation of objects in JavaScript, check out MDN: Working withObjects18.

Strict equality

JavaScript has the concept of equality (==) and strict equality (===). The standard equality operator (==) appliessome oddly complex type coercion rules to each side of the equals signs. For example, look at the followingnode.js REPL output::

$ node

> 1 == false

false

> 0 == false

true

> "0" == false

true

> "" == false

true

> "0.0" == false

17http://mzl.la/13L8pWd18http://mzl.la/18GTPai

Basics 10

true

> "false" == false

false

> false == false

true

If you’re not familiar with a REPL environment (REPL stands for ‘read-eval-print-loop’), the first line executesnode, and subsequent lines either begin with a > to indicate user input or no preceding character to indicatean evaluated result. If the above results annoy your judgement, consider the following results using strictequality::

> 1 === false

false

> 0 === false

false

> "0" === false

false

> "" === false

false

> "0.0" === false

false

> "false" === false

false

> false === false

true

In short, I recommend always using the strict equality (===) operator unless you are 100% certain the coercedvalue is what you intend and there is a comment in code to indicate this intention.

Scope/Context

Some folks will argue all day long about whether scope and context are the same thing. Rather than attemptto differentiate between these two somewhat loaded terms, I’ll discuss functional scope and execution context.

Current JavaScript environments are based on ECMAScript 262, Edition 5. To quote the documentation19:

When control is transferred to ECMAScript executable code, control is entering an executioncontext. Active execution contexts logically form a stack. The top execution context on this logicalstack is the running execution context. A new execution context is created whenever control istransferred from the executable code associated with the currently running execution context toexecutable code that is not associated with that execution context. The newly created executioncontext is pushed onto the stack and becomes the running execution context.

19http://bit.ly/18GZ8qk

Basics 11

An execution context maintains three things: variable environment, lexical environment, and the binding tothis.

Elsewere in the document, you will find that there is an internal property called [[Scope]] which refers tothe lexical environment of the execution context. This scope property is defined in Table-9 of the document:

A lexical environment that defines the environment in which a Function object is executed. Ofthe standard built-in ECMAScript objects, only Function objects implement [[Scope]].

Does all of that make your brain hurt? If so, let me attempt to state it more simply.

Whenever a function executes, it does so in a certain context which defines the variables (functions andobjects) it has created internally as well as the variables it has access to externally as a result of being nestedwithin other functions. Resolution of variables propagates all the way up to the global environment. This ishow I define functional scope.

One possible ‘gotcha’ is that functions are lexically scoped, meaning this variable resolution order isdetermined by where a function is declared and not by where a function is invoked. This is the basis forclosures; we can define a function which is returned from another function and encapsulate any variablesfrom the enclosing or outer function privately.

When people refer to context, I don’t think they’re talking about execution context as much as they are talkingabout the value of this in the current execution context. Consider the following example which demonstratesthe value of x in different contexts.

basics/context.js

1 var x = 5;

2

3 var example = {

4 x: 100,

5

6 a: function(){

7 var x = 200;

8 console.log('a context: %s, var x = %s', this.x, x);

9 },

10

11 b: function(){

12 var x = 300;

13 return function(){

14 var x = 400;

15 console.log('b context: %s, var x = %s', this.x, x);

16 };

17 },

18

19 c: function(){

20 var other = { x: 500 };

21 var execB = this.b().bind(other);

Basics 12

22 execB();

23 return execB;

24 }

25 }

26

27 console.log('example.x: ' + example.x); // 100

28 example.a(); // 100

29 example.b()(); // undefined

30 example.a.call({x:9999}); // 9999

31 var execB = example.c(); // 500

32 execB.call({x:9999}); // 500

In this code example, you may be wondering how the context has changed between functions a, b and c. Todemonstrate, I created a variable x in each lexical environment for a given function. Notice how the value 5is never displayed.

In function a, the reference bound to this is the object named example.

In function b, we indirectly change the bound context by returning a function fromwithin a function which inturn changes the encapsulating object (e.g. scope) of the final function. You may expect this resulting functionto print ‘5’, but the x variable is susceptible to the same rules here as it is in function a: the encapsulatingcontext’s this is actually bound to the global execution environment (node, in our case).

When we see a output 9999, we are using the special function .call() which allows us to manipulate thebound this context. Rather than relying on the example object as context, we can easily pass in a literal objectwith any value we want! Usually, when I saying something is ‘changing context’, this is what I’m talkingabout. There are two special functions which can change the value of this at the point of invocation: call()and apply().

Line 21 in the example demonstrates a third way to modify the this bound context. The interesting thingabout this approach to contextual modification is that bound functions can not have their context changedat runtime after being bound. This behavior can be seen on line 32. If we attempt to use call() as we did onfunction a, the context is not changed at all. This allows us to essentially lock down function context whenworking with or creating our own modules.

Constructor Functions

A special case for the scope and context discussion is the use of constructor functions. A constructor functionis a function which creates an object and, as we saw in the example object above, a new execution context.The community standard for differentiating constructor functions from normal functions is to format normalfunction names in camel case (likeThis) and constructor functions in pascal case (LikeThis). Unfortunately,the visual hint of beginning with a capital letter is only a hint.

Code analysis tools such as JSHint and JSLint, when incorporated in your build script, can beconfigured to emit warnings or errors when you don’t correctly use the community’s capitalizationstandard.

Basics 13

To actually execute a constructor function as a constructor, you must call it with the new operator, such asvar x = new FunctionName();. Take a look at what happens when you call a constructor function withoutthe new operator:

basics/constructors.js

1 var x = 100;

2

3 function Example(name){

4 var self = this;

5 self.x = 200;

6 self.name = name;

7

8 self.look = function(){

9 console.log("%s: x is %d", self.name, self.x);

10 };

11 return self;

12 };

13

14 var a = new Example('A');

15 a.look();

16 var b = Example('B');

17 b.look();

18

19 console.log(a);

20 console.log(b);

If you run the above example using node, you’ll see a lot of output you probably didn’t expect:

A: x is 200

B: x is 200

{ x: 200, name: 'A', look: [Function] }

{ ArrayBuffer: [Function: ArrayBuffer],

Int8Array: { [Function: Int8Array] BYTES_PER_ELEMENT: 1 },

Uint8Array: { [Function: Uint8Array] BYTES_PER_ELEMENT: 1 },

Uint8ClampedArray: { [Function: Uint8ClampedArray] BYTES_PER_ELEMENT: 1 },

...

setTimeout: [Function],

setInterval: [Function],

clearTimeout: [Function],

clearInterval: [Function],

setImmediate: [Function],

clearImmediate: [Function],

console: [Getter],

x: 200,

Basics 14

name: 'B',

look: [Function] }

How did object B get all that extra stuff in it? For starters, I’ve cheated a bit on line 11. If a constructor functiondoesn’t return a value, the default behavior is to either return the bound this object. Using the new operatorcreates a new this context, while skipping the new operator causes this to be bound to the global object. Thatis, when you call a constructor function without the new operator, you’re just calling a function within thecurrent execution context. In the above output, the execution context is the node global space (sometimesreferred to as the global context). Had I skipped line 11, object A would have worked exactly as it had beforebut object B would have been undefined. Some people try to avoid this issue by using a constructor guard ormarking the function itself with 'use strict';.

Constructor Guard

1 function Example(){

2 if(!(this instanceof Example)) {

3 return new Example();

4 }

5 }

‘use strict’;

1 function Example(name){

2 'use strict';

3 this.name = name;

4 }

The problemwith the constructor guard is that it can cause issues or complexities with prototypal inheritance.The 'use strict'; directive prevents accidentally leaking the internals of your object to the global scope;the value of this is undefined when the constructor is called without the new operator.

If you still have questions or concerns regarding constructor functions, read about it further on the MozillaDeveloper Network.

Node.js Basics

In 2008 or 2009, Google began shifting its gears from being an online search and marketing firm toconcentrating on a specific goal of ‘making the web faster’. Amajor part of this initiative was the improvementof web page responsiveness, load times, and traffic reduction. They created and implemented a protocolcalled SPDY which is meant to reduce the number of IO hits a page load incurs on a site. They alsocontributed to, forked, and rebranded the Chromium browser. In the rebranded Google Chrome browser,Google implemented the v8 JavaScript engine.

Google’s v8 engine is not necessarily unique when compared to other JavaScript engines. For example,Mozilla’s SpiderMonkey JavaScript engine is written in C/C++ and is open source in much the same way as

Basics 15

Google’s v8 engine. Google strives for cross-platform feature completeness and is released under a permissivelicense. Mozilla obviously strives for cross-platform completeness, but finding a single clear licesnse for theSpiderMonkey engine is not exactly easy20.

I can only really hypothesize why Ryan Dahl chose Google’s v8 engine as the basis for creating node.js. I’mnot saying v8 is any better than SpiderMonkey or other JavaScript implementations. It was likely chosen basedon personal preference as much as it was related to speed or ease of use as a dependency.

In 2009, Dahl gave an excellent presentation21 which discussed reasons why we need evented non-blockingI/O and why JavaScript is very well suited for the task. If you can track down the video presentation of node.jsas JsConf 2009, I highly recommend watching that too.

What is node.js?

Node.js is a server-side JavaScript environment which provides evented asynchronous I/O and a minimalisticset of core features. In the beginning, node.js combined v8 with libev. Unfortunately, libev is an eventedI/O library for Unix-based systems only. Eventually, libev was replaced with a higher-level abstractionlibrary called libuv which allows node.js to run on Unix-based systems and Windows. In 2011, Microsoftbegan contributing to node.js22 to bring it to the Windows platform. More recently, libuv removed libev as adependency23.

The key concept for node.js is evented I/O (mostly network-based) operations. This is handled via an eventloop. Although understanding the implementation details of event loops can be pretty complicated, the ideais fairly simple. Let’s analogize.

Try to think about an application as a fast food restaurant. A cashier is patiently waiting at the register. Acustomer walks up to the register and places an order. The cashier hands the order off to the kitchen andmaybefulfills the customer’s drink order before returning to the register. Another customer arrives and places anorder, initiating the same scenario as before. A third customer is walking to the register, but just before thecustomer reaches the register a cook dings a bell and the cashier steps away to grab the first order and handit to the customer. The cashier then begins taking the third customer’s order. Midway through the order, thesecond customer’s order is ready. Because the cashier is not rude (maybe this is in Canada?), the third ordermust be completed before the second customer’s order can be handed back to the customer. But, the cashiercan easily hand over the second customer’s order before filling the third customer’s drink order. Eventually,the third customer’s order is fulfilled and there are no more customers. The cashier waits patiently.

It may sound strange, but the above scenario is a simplification of an event loop. We see things like this everyday. New node.js developers seem to have a hard time either understanding the event loop or understandingwhere it comes from. Take the following example, for instance:

20Comparing the licenses located at http://hg.mozilla.org/mozilla-central/file/9af6265e9884/LICENSE with the information athttp://www.mozilla.org/en-US/foundation/licensing/, it might be difficult to determine whether Mozilla’s license will clash with your chosenlicense.

21http://bit.ly/18GgPDN22http://bit.ly/1bAJU7a23http://bit.ly/13nXiDA

Basics 16

basics/helloworld.js

1 function main(){

2 console.log("Hello, world!");

3 }

4

5 main();

There’s a little more going on in this example than the standard obligatory printing of “Hello, world!” I’vewrapped the output line in a function called main and subsequently called that function. If you run this file,you’ll see that it prints out “Hello, world!” and exits. There is no event loop because we haven’t told theruntime to listen to or emit any events.

There are a couple ways we can initiate an event loop. Possibly the easiest way to visualize this is to use thestandard JavaScript setInterval function to cause our application to write out “Hello, world!” every 250ms.To kill this application, you’d need to press CTRL-C.

basics/helloworld-loop.js

1 function main(){

2 console.log("Hello, world!");

3 }

4

5 setInterval(main, 250);

The above example is technically an event loop, although there aren’t really any events going on other thanthe setInterval function. Another way to initiate an event loop is to bind to or set up any event whichinteracts in some way with a file handle. In Unix, nearly everything is considered a file, so this would meanbinding to a socket, file descriptor, or event screen input:

basics/helloworld-sigint.js

1 // respond to the SIGINT event

2 process.on('SIGINT', function(){

3 process.stdout.write('\nHandling SIGINT event!\n');

4 // cleanly exit with non-error status

5 process.exit(0);

6 });

7

8 // listen to events on standard input

9 process.stdin.setEncoding('utf8');

10 process.stdin.on('data', function(chunk) {

11 process.stdout.write('Hello, world!\n');

12 });

13

14 // Start reading from stdin so we don't exit.

15 process.stdin.resume();

16 console.log('Type anything and press ENTER...');

Basics 17

In the above example, process.stdin.resume() causes node.js to continually monitor the readable standardinput stream for data. Binding a listener to watch for ‘SIGINT’ does not cause node to create an event loop onits own. Play around with this last example to get an idea of how the event loop works. If you’re interestedin how libuv creates and maintains an event loop, there is a free uvbook24 available which dives into theinternals of libuv.

Low-level

The node.js library is meant to be very low-level. Rather than a traditional web server with plugins, modulesand other froofiness, there are servers specializing in tcp, udp, and http connectivity. Instead of giving youmodules for interacting with a variety of file types (PNG, JPEG, etc.), there are readable and writable streams.This can be intimidating. Luckily, the node.js community produces and shares a large repository of opensource modular code, called npm.

The npm project provides Node Packaged Modules and is bundled together with node.js in recent versions(recent means v0.6.3 or higher).

We won’t go in depth on npm. All you need to know to get started is that npm handles package managementvia a package.json file and that you can install modules locally (within your project) or globally (accessible toyour entire system). If you’ve used other package managers like bundler, bower, NuGet, or any others, you’llpick up on npm quickly.

Don’t be put-off by the low-level nature of node.js core libraries. Many of the modules in the npm repositoryare ‘production-ready’ and allow you to build applications and modules atop a very customizable softwarestack.

Non-blocking I/O

There are two terms I will use interchangeably here: non-blocking I/O and asynchronous I/O. Conceptuallythey’re similar, but fundamentally they’re different. Non-blocking I/O is any file descriptor based operationfor which a call returns immediately with a message to indicate to the caller that the function would havebeen a blocking operation if called differently, and it is up to the invoking code base to continually poll thekernel in some way for the error or result. Asynchronous I/O operates in almost the same way, except insteadof requiring the framework to continually bother the kernel for a response, a callback function’s memorylocation is referenced in the invocation so the kernel can simply write to some memory when it is ready.

Real world scenarioLet’s relate blocking and non-blocking I/O to the real world with an office scenario.

Suppose the year is 1983 and you don’t have access to many of the high-tech conveniences you do today.Your boss drops off a stack of what appears to be 1,000 sheets of paper.

She asks you, “I need you to check the spelling of this document. How long do you think it will take?” Youreply, “No clue, but I’ll take a look.”

Annoyingly, your boss sits in your office and waits until you’re finished editing her papers. You ask what’s

24http://bit.ly/1c4n6Nm

Basics 18

gotten into her and she says that she’s unable to do any other work until this stack of papers is complete.She’s blocked waiting for you to finish. This is an example of blocking I/O– your boss provides the inputas a stack of 1,000 papers to be corrected and is literally unable to do anything else until you provide theoutput (papers with no spelling errors).

The next day, your boss provides a similarly daunting stack of papers to be corrected. You ask her, “Do youneed these right away? I have a few other things to finish.” Surprisingly, she says she has a lot of otherthings to work on and asks you to just call her office when you’re done. You churn through these pages andwhen you’re finished, you give her a ring and she picks up the papers. Your boss was not blocked by yourtask and is able to do whatever other work she has while waiting for you to finish. This is an example ofasynchronous I/O because the responsibility of notifying your boss that the output was ready was on you.If, on the other hand, your boss regularly came back to your office whenever she was free to save you a trip,that would be non-blocking I/O because she’s polling your status. In both situations, she’s able to get otherthings done will you’re chugging along (which is why I use these terms interchangeably).

What this means is that any time there is a sycnchronous operation going on, nothing else can occur. Anyrequests to do other work would have to be queued up in order to be executed when the current synchronousoperation is finished. As you can see in the blocking operations diagram, a blocking call is represented by asolid white rectangle (an activation in UML terms). The example illustrates what would happen in code whichqueries whether or not a file exists before reading the entire file contents into memory.

Diagram: Blocking operations using fs module

Basics 19

Code: Blocking operations using fs module

var fs = require('fs'),

file = '/tmp/example.txt';

if(fs.existsSync(file)) {

var contents = fs.readFileSync(file, 'utf8');

console.log(contents);

} else {

console.log('%s does not exist', file);

}

The non-blocking diagram has broken areas between the asynchronous calls to fs.exists and fs.readFile.While the operating system’s kernel performs some work for fs.exists, it’s possible for some other code(not fs.readFile) directly after this function call to execute. When the kernel has completed the operation,a function is invoked inside of which fs.readFile is executed depending on the result of the fs.exists call.

Diagram: Non-blocking operations using fs module

Code: Non-blocking operations using fs module

var fs = require('fs'),

file = '/tmp/example.txt',

o = { encoding: 'utf8' };

fs.exists(file, function(itExists) {

if(itExists) {

fs.readFile(file, o, function(err, data) {

if(err) {

console.log(err);

} else {

console.log(data);

}

Basics 20

});

} else {

console.log('%s does not exist', file);

}

});

You may have noticed that the non-blocking example’s code looks a little more complex than the blockingexample’s code. This is most likely because of how we’re taught to program. It is ingrained into our collectivedeveloper minds to write blocking code25.

Beginner code in ruby:code/basics/beginner.rb

#!/usr/bin/env ruby

puts "What is your name?"

name = gets

puts "Hello #{name.chomp}!"

Does the above snippet look familiar? You may not have initially learned coding in ruby, but the process isthe same: show text, get input, show more text. Thinking in this way is inherently easier to comprehend than:create a loop which responds to events, show text, loop?, get input, loop?, show more text only when input isreceived.

Summary

By now you should have a pretty good grasp of the JavaScript basics that I consider a requirement for node.jsdevelopment. You should also be comfortable with node.js as a low-level starting point for non-blocking andevented I/O as a server-side JavaScript environment. Be sure to bookmark the node.js documentation site(http://nodejs.org/api/).

In the next chapter, we’ll go over the initial concern of any development project: what we plan to build.

25I admit, I’m using a very similar example to the one Ryan Dahl used at jsconf ‘09.