ewd 3 training course part 42: the qewd docker appliance

Copyright © 2016 M/Gateway Developments Ltd

EWD 3 Training CoursePart 42

The QEWD Docker Appliance

Rob TweedDirector, M/Gateway Developments Ltd

Twitter: @rtweed


Docker?

• A technology that allows software to be encapsulated inside an isolated container

• According to Docker themselves:– "Docker containers wrap a piece of software in a

complete filesystem that contains everything needed to run: code, runtime, system tools, system libraries –anything that can be installed on a server. This guarantees that the software will always run the same, regardless of its environment."

– See https://www.docker.com/what-docker


QEWD Docker Container• QEWD is available from the main docker.io

repository as a pre-built Docker container– rtweed/qewd

• You can also build your own version using the Dockerfile that is included in the QEWD repository– eg, if you wanted to apply your own customisations– See https://github.com/robtweed/qewd/blob/master/docker/Dockerfile


Benefits of the QEWD Container

• Provided you've installed Docker, you don't need to do any further installation– Just run the container

• First time you do this, it will automatically download it from docker.io


Benefits of the QEWD Container

• QEWD functionality without any "moving parts" on your system– eg no Node.js dependency clashes

• You don't even need Node.js installed on your system

– You just define your handler modules• Write JavaScript files and let the QEWD container

handle them– Without Node.js having to even be present on your

system


QEWD Appliance

• Just the core QEWD engine– Master process– Worker processes

• Doesn't include the embedded database– Currently designed to work with Redis only

• Using mapped volumes– You can use Redis locally, in another

container or remotely


QEWD Appliance

• Normal QEWD Session management will take place provided a Redis database is available


QEWD Appliance

• Nothing you can do in a standard local QEWD installation that can't also be achieved by using the Dockerised version– There's even a version for the Raspberry Pi!

• rtweed/rpi-qewd


QEWD Appliance

• "Out of the box" defaults– Management password: 'keepThisSecret!'– Worker pool size: 1– Database: Redis listening on port 6379

• qewd-monitor application is included and ready to run– No other applications or APIs pre-defined


QEWD Appliance

• You can customise the configuration:– Management password– Server name (what appears in qewd-monitor's

banner)– Worker pool size


QEWD Appliance

• By using Docker volume mapping, you can make specific directories in your host system available to the QEWD Docker container

• In your host system you can therefore define interactive QEWD applications:– Static browser-side resources (HTML, JS,

CSS files)– Back-end handler modules


QEWD Appliance

• You can also define routes within your host system, to be used by the QEWD Appliance– For Web / REST service APIs


So let's try it out

• I'm going to use a Ubuntu Linux system• However, Docker can be installed on most

platforms– Windows– Linux (all versions/flavours)– Unix– MacOS– Even the Raspberry Pi!


Installing Docker on Ubuntu 16.04

• sudo apt-get update• sudo apt-get install docker.io

• You can now use Docker– By default, you'll need to run Docker

commands as sudo– This can be modified. See:– https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-16-04


Installing Docker on Raspberry Pi

• Just invoke the command:curl -sSL https://get.docker.com | sh

• You can now use Docker on your Raspberry Pi– By default, you'll need to run Docker

commands as sudo


Some Quick Docker Tests

• Try running these to confirm that Docker is properly installed and working on your system:– sudo docker version– sudo docker info


Let's start a Redis ContainerOn Linux:

• sudo docker run -d --name redis -p 6379:6379 redis

• Runs as a daemon (-d)• Exposes internal port 6379 (the default Redis port)

externally to the host as port 6379 (-p)

• First time you invoke this command, it will download and install the Redis Docker container


Let's start a Redis ContainerOn Raspberry Pi:

• sudo docker run -d --name redis -p 6379:6379 hypriot/rpi-redis

• Runs as a daemon (-d)• Exposes internal port 6379 (the default Redis port)

externally to the host as port 6379 (-p)

• First time you invoke this command, it will download and install the Redis Docker container


Test Redis• First install the Redis command line tool:

– sudo apt-get install redis-tools

• Now start the Redis command line:redis-cli

• It should respond with127.0.0.1:6379>– it's working and you're connected to the Redis container!

• Try the command:info

• Exit redis-cli by typing CTRL&C


Prepare for the QEWD Appliance

• First important step:– Create a directory for mapping the QEWD working

directory– It can be any directory you like. I'll use:

cd ~mkdir qewd

– Now make a sub-directory named modulescd qewdmkdir modules

• You don't need to put anything into this directory yet, but it should exist


Start the QEWD Appliance

sudo docker run -d -p 8080:8080 --link redis:redis -v /home/ubuntu/qewd/modules:/opt/qewd/mapped rtweed/qewd

First time you invoke this, it will download the QEWD Container fromDocker.io

-p 8080:8080 maps port 8080 in the QEWD container to your host's port 8080-d run as a daemon. You can also try -it for interactive display-link redis:redis links to the Redis container-v maps the QEWD working directory to the one we created

QEWD will now be running!

On Linux:



sudo docker run -d -p 8080:8080 --link redis:redis -v /home/pi/qewd/modules:/opt/qewd/mapped rtweed/rpi-qewd

First time you invoke this, it will download the QEWD Container fromDocker.io

-p 8080:8080 maps port 8080 in the QEWD container to your host's port 8080-d run as a daemon. You can also try -it for interactive display-link redis:redis links to the Redis container-v maps the QEWD working directory to the one we created

QEWD will now be running!

On Raspberry Pi:


Try QEWD

http://192.168.1.123:8080/qewd-monitor/index.html

Change the IP address / host name as appropriate for your host machine

You should see the qewd-monitor application!

Log in using the default password: keepThisSecret!


Let's check Redis again

• Start redis-cli again• This time enter the command:

– keys *

• You should see a load of Redis key/value pairs, representing the QEWD Session global storage used by your qewd-monitorsession


Let's build a quick application

• We'll create the files in the ~/qewddirectory that we previously made


First, stop the QEWD Container

• Use the qewd-monitor; or• Use the docker stop command

– To do that we need to first find out the QEWD container's unique Id

– sudo docker ps -aCONTAINER ID IMAGEd810c520894e rtweed/qewd7d0c18b8cec5 redis


First, stop the QEWD Container

• Use the qewd-monitor; or• Use the docker stop command

– To do that we need to first find out the QEWD container's unique Id

– sudo docker ps -aCONTAINER ID IMAGEd810c520894e rtweed/qewd7d0c18b8cec5 redis Usually the first

3 characters aresufficient


Stop the QEWD Container

• sudo docker stop d81


Create a test application

• We'll name it testapp– It will have a button which, when clicked,

sends a message to the QEWD back-end which returns an acknowledgement response

• First create a directory for the browser-side static files:

cd ~/qewdmkdir wwwcd www


Create the index.html<html> <head><title>Demo QEWD application</title>

</head> <body><script src="//ajax.googleapis.com/ajax/libs/jquery/2.2.0/jquery.min.js"></script><script src="/socket.io/socket.io.js"></script><script src="/ewd-client.js"></script><script src="app.js"></script>

<button id="testBtn">Click Me</button>

<div id="content">Content goes here

</div></body>

</html> Save as ~/qewd/www/testapp/index.html


Create the app.js

Save as ~/qewd/www/testapp/app.js

$(document).ready(function() {

EWD.log = true;

EWD.on('ewd-registered', function() {$('#testBtn').on('click', function(e) {var message = {type: 'testButton'};EWD.send(message, function(messageObj) {$('#content').text(messageObj.message.ok);

});

});});

EWD.start('testapp', $, io);});


Create the back-end module

• This needs to go into the directory we created earlier:

~/qewd/modules• We'll name the module testapp.js


Create testapp.jsmodule.exports = {handlers: {testButton: function(messageObj, session, send, finished) {console.log('*** handling the button click message!');session.data.$('foo').value = 'bar';finished({ok: 'testButton message was processed successfully!'

});}

}};

Save as ~/qewd/modules/testapp.js


How will QEWD find the module?

• The QEWD Docker Appliance will expect to find it in its node_modules directory

• We don't have access to that from the host• But we've mapped ~/qewd/modules to a

mapped volume in the Docker container:~/opt/qewd/mapped


How will QEWD find the module?

• We can use a QEWD module map to tell QEWD where to find the back-end module for our testapp application

• The QEWD Docker appliance allows us to define this in the mapped volume directory using the reserved file name: – moduleMap.js


Create moduleMap.jsmodule.exports = {testapp: '/opt/qewd/mapped/testapp'

}

Save as ~/qewd/modules/moduleMap.js



}

The module map exports as many mappings as you want




}

The module map exports as many mappings as you want

We just need this one.




}

Note: the mapped module path is to the internal pathwithin the container




}

Note: the mapped module path is to the internal pathwithin the container, not the path we created on the host

~/qewd/modules/testapp.js


That should be all we need

• When we start the QEWD Appliance again, we must also map the directory containing the browser-side files, ie– ~/qewd/www/testapp



• On Linux:

sudo docker run -d -p 8080:8080 --link redis:redis ↩ -v /home/ubuntu/qewd/modules:/opt/qewd/mapped ↩

-v /home/ubuntu/qewd/www/testapp:/opt/qewd/www/testapp ↩ rtweed/qewd



• On Raspberry Pi:

sudo docker run -d -p 8080:8080 --link redis:redis ↩-v /home/pi/qewd/modules:/opt/qewd/mapped ↩

-v /home/pi/qewd/www/testapp:/opt/qewd/www/testapp ↩rtweed/rpi-qewd


Try running testapp

It works

The console show that testapp correctly registereditself on QEWD

Try clicking the button..


Try running testapp

It works too!

A message was sentto the QEWD back-endwhich returned the expectedresponse


Check the SessionStart the qewd-monitor application and click the Session tab:


Check the SessionStart the qewd-monitor application and click the Session tab:

Where did this come from?


It came from testapp.jsmodule.exports = {handlers: {testButton: function(messageObj, session, send, finished) {console.log('*** handling the button click message!');session.data.$('foo').value = 'bar';finished({ok: 'testButton message was processed successfully!'

});}

}};

Save as ~/qewd/modules/testapp.js


QEWD Apps with no moving parts!

• We just created a QEWD app without having anything installed other than Docker

• We just defined:– the HTML and JavaScript files for the

browser– The back-end JavaScript handler module

• And mapped their directories as Docker volumes


Adding more apps• Each app you create will need its own

subdirectory under the ~/qewd/www directory– For its browser-side HTML, JavaScript, etc files– Each of these will have to be mapped to the

corresponding volume in the QEWD container• Their back-end modules, however, just go into

your mapped ~/qewd/modules folder– And then define their module mapping in

moduleMap.js


Mapping Multiple Apps

module.exports = {testapp: '/opt/qewd/mapped/testapp',mySecondApp: '/opt/qewd/mapped/mySecondApp',myThirdApp: '/opt/qewd/mapped/myThirdApp'

}

~/qewd/modules/moduleMap.js


Mapping Multiple Apps

• eg, On Linux:

sudo docker run -d -p 8080:8080 --link redis:redis ↩-v /home/ubuntu/qewd/modules:/opt/qewd/mapped ↩

-v /home/ubuntu/qewd/www/testapp:/opt/qewd/www/testapp ↩-v /home/ubuntu/qewd/www/mySecondApp:/opt/qewd/www/mySecondApp ↩-v /home/ubuntu/qewd/www/myThirdApp:/opt/qewd/www/myThirdApp ↩rtweed/qewd


What about REST APIs?

• We can use the QEWD Docker Appliance to very quickly create REST APIs, again without any moving parts other than Docker itself

• Even simpler to create than applications


Let's create an API• We'll define a URL path prefix:

– /api• And handle any lower-level URLs, eg:

– /api/test

• To do this we just need to create a handler module for the /api path. We'll call it api.js. It must be saved into the ~/qewd/modules directory


Create api.jsmodule.exports = {

restModule: true,

handlers: {test: function(messageObj, finished) {finished({foo: 'bar'});

}}

}; Save as ~/qewd/modules/api.js



restModule: true,


}}

}; Save as ~/qewd/modules/api.js

It's just a standard QEWD REST/Web Service Module



restModule: true,


}}

}; This function, test, will handleany incoming /api/test requests


QEWD needs to know how to find it

• As we're defining the handler module in a mapped directory, we need to tell QEWD where to find it and how to define it as a route.

• For this we define a reserved named file in our ~/qewd/modules directory:– routes.js


Create routes.jsmodule.exports = [{path: '/api',module: '/opt/qewd/mapped/api'

}];

Save as ~/qewd/modules/routes.js



}];

Save as ~/qewd/modules/routes.js

You export an array of route objects

We'll just define one for /api in this example,but you can define as many routes as you wish



}];

Note that the route must map to the container'sinternal mapped volume path

So any incoming requests with a URL starting/api will be handled by the api.js file we've createdin the host's mapped directory


Let's try it

• Our handler function isn't making any distinctions about the HTTP method used, so we can test it just using a browser.


Let's try it

It worked!


REST/Web Service APIs

• Of course this was a very simple example• But you can write as complex APIs as you

wish. • All the functionality described in parts 31

and 36 of this course are available in the QEWD Docker Appliance


REST APIs without moving parts

• You can now define Web/REST APIs by writing JavaScript handler module files

• No need to install Node.js


Customising QEWD Appliance

• QEWD Docker Appliance defaults:– Management password: keepThisSecret!

• Always a good idea to change this!– Server name: QEWD Docker Server

• Used by the qewd-monitor application– Worker pool size: 1

• You can change these


Customising QEWD Appliance

• Create a file named custom.js in your mapped directory (eg ~/qewd/mapped)– custom.js is a reserved name that the QEWD

Docker Appliance will look for and use for customising its configuration


Create custom.jsmodule.exports = {config: {managementPassword: 'mySecret!',serverName: 'Robs QEWD Docker Appliance',poolSize: 2

}};

Save as ~/qewd/modules/custom.js


Restart the Appliance

• eg:– sudo docker restart d81

• Try running qewd-monitor– You'll need to use your new custom login

password– It will display the new name for your server– It will show 2 worker processes


Further Customisation

• You can get access to the master process objects:– Create user defined data / objects that are

passed to worker processes– Access the ewd-qoper8 object– Access the Express object

• Define a run function. This is invoked when QEWD starts.


Create custom.jsfunction customise(config, q, intercept) {console.log('*** This is the custom function calling!');

}

module.exports = {config: {run: customise,managementPassword: 'mySecret!',serverName: 'Robs QEWD Docker Appliance',poolSize: 2

}};

Save as ~/qewd/modules/custom.js


Create custom.jsfunction customise(config, q, intercept) {console.log('*** This is the custom function calling!');

}

module.exports = {config: {run: customise,managementPassword: 'mySecret!',serverName: 'Robs QEWD Docker Appliance',poolSize: 2

}}; q is the main ewd-qoper8 object

intercept.app is the Express objectallowing you to add custom middleware


That's the QEWD Docker Appliance

• A handy and simple, low-impact way of using QEWD without worrying about interactions with your existing set-up– And without having to install Node.js or a

database• All the functionality of a standard local

implementation of QEWD is still available to you

ewd 3 training course part 42: the qewd docker appliance

Software