LNUG: Having Your Node.js Cake and Eating It Too

Download LNUG: Having Your Node.js Cake and Eating It Too

Post on 22-Jan-2018

515 views

Category:

Software

5 download

TRANSCRIPT

  • Copyright 2017 M/Gateway Developments Ltd

    Having Your Node.js Cake and Eating It Too

    Rob TweedM/Gateway Developments Ltd

    Twitter: @rtweed

    http://qewdjs.com

    mailto:rtweed@mgateway.com

  • Copyright 2017 M/Gateway Developments Ltd

    A bit of background

    Royal Marsden Hospital: 1980s Touche Ross Management Consultants

    (now Deloitte): early 1990s NHS-wide Networking Project

    Independent consultant & software developer since 1993 Specialising in Web and associated

    technologies, particularly in healthcare

  • Copyright 2017 M/Gateway Developments Ltd

    A bit of background

  • Copyright 2017 M/Gateway Developments Ltd

    A bit of background

    Node.js: Since August 2010 (v0.2 had just come out!) Gave one of the talks at LNUG inaugural

    meeting

  • Copyright 2017 M/Gateway Developments Ltd

    And So, 6 Years Later

  • Copyright 2017 M/Gateway Developments Ltd

    Node.js Architecture

    Everything executes in a single process You might have 1000's of users doing stuff

    concurrently They're all sharing the same process!

  • Copyright 2017 M/Gateway Developments Ltd

    Node.js Architecture Non-blocking I/O

    No user activity must block the process, or everyone grinds to a halt

    Event-driven, asynchronous logic Fire off the request, but don't wait for the results

    Carry on with the next task When results come back, handle them at the next

    opportunity

  • Copyright 2017 M/Gateway Developments Ltd

    Node.js = server-side JavaScript

    Ryan Dahl wasn't actually a big JavaScript fan However, he realised that it was a convenient

    language for such an environment, as it's designed for exactly the same kind of way of working in the browser

    Google's V8 JavaScript engine was Open Sourced, so he used it to provide the language for Node.js (which is actually mostly written in C++)

  • Copyright 2017 M/Gateway Developments Ltd

    Asynchronous Syntax in Other Languages?

    Node.js isn't unique in supporting asynchronous logic and non-blocking I/O

    Available also in most other modern languages Java Python, etc

    Allows efficient use of resources when making parallel requests for resources Files, external web services etc

  • Copyright 2017 M/Gateway Developments Ltd

    Asynchronous Syntax in Node.js

    Node.js is unique in that you have no option but to use Asynchronous logic

  • Copyright 2017 M/Gateway Developments Ltd

    Drawbacks of Node.js

    In every other language, zealots will encourage you to use it for everything!

    But even Node's greatest exponents will tell you not to use it for certain things

  • Copyright 2017 M/Gateway Developments Ltd

    Drawbacks of Node.js

    Don't use Node.js if you have: CPU-intensive logic Complex database manipulation

    Particularly relational database handling

  • Copyright 2017 M/Gateway Developments Ltd

    Drawbacks of Node.js

    Don't use Node.js if you have: CPU-intensive logic

    Tie up the CPU and everyone else using the process will be blocked

    Complex database manipulation Particularly relational database handling

  • Copyright 2017 M/Gateway Developments Ltd

    Drawbacks of Node.js

    Don't use Node.js if you have: CPU-intensive logic Complex database manipulation

    Particularly relational database handling Due to the limitations of asynchronous logic, eg you can't

    chain functions Limits creation of very high-level database

    abstractions

  • Copyright 2017 M/Gateway Developments Ltd

    Main criticism of newbies to Node.js

    Asynchronous logic

  • Copyright 2017 M/Gateway Developments Ltd

    Async is the New Sync?

    Node.js version 8 now supports Async/Await which greatly improves the syntax needed to handle asynchronous logic Avoids "callback hell" Very like synchronous logic

  • Copyright 2017 M/Gateway Developments Ltd

    Async/Await Doesn't solve:

    Node.js concurrency Still need to avoid CPU-intensive code

    High-level database abstractions Proper chaining of functions requires

    synchronous logic

  • Copyright 2017 M/Gateway Developments Ltd

    Usual Solutions

    Use a third-party queue to offload CPU-intensive work to some other environment RabbitMQ ZeroMQ

    Use another language such as Rails for database-intensive work

  • Copyright 2017 M/Gateway Developments Ltd

    The down-sides

    Heterogeneous environment Multiple skill-sets Multiple moving parts

    Node.jsRabbitMQ

    Rails

    Java

    Database

  • Copyright 2017 M/Gateway Developments Ltd

    What does Ryan Dahl Think?

  • Copyright 2017 M/Gateway Developments Ltd

    What does Ryan Dahl Think?

    "..within a single process we could handle many requests by being completely asynchronous. I believed strongly inthis idea at the time, but over the past couple of years, I think thats probably not the be-all and end-all idea for programming."

  • Copyright 2017 M/Gateway Developments Ltd

    What does Ryan Dahl Think?

    "..when I first started hearing about Go, which was around 2012, they had really easy to use abstractions, that make "blocking I/O", because its all in green threads at the interface between Go and the operating system.

    I think it is actually all non-blocking I/O."

  • Copyright 2017 M/Gateway Developments Ltd

    What does Ryan Dahl Think?

    "I think Node is not the best system to build a massive server web. I would definitely use Go for that.

    And honestly, thats basically the reason why I left Node. It was the realization that: oh, actually, this is not the bestserver side system ever."

  • Copyright 2017 M/Gateway Developments Ltd

    What do I think?

    I want to have my Node.js Cake and Eat it!

  • Copyright 2017 M/Gateway Developments Ltd

    I want my Node.js Cake & Eat it

    I like JavaScript I want just one language for everything

    1 skill set 1 set of moving parts

    No extra technologies, thank you Keep it simple

    And I want to be able to use Node.js for all situations

  • Copyright 2017 M/Gateway Developments Ltd

    The Problem is Concurrency

    All concurrent users in Node.js share the same process

  • Copyright 2017 M/Gateway Developments Ltd

    The Problem is Concurrency

    All concurrent users in Node.js share the same process

    As it happens, Amazon Web Services accidentally created a solution And nobody (including AWS) seems to have

    realised the consequences of what they've done

  • Copyright 2017 M/Gateway Developments Ltd

    AWS Lambda

    "Function As A Service" AKA "Serverless"

  • Copyright 2017 M/Gateway Developments Ltd

    AWS Lambda

    "Function As A Service" AKA "Serverless"

    You upload functions AWS will execute them

    You don't worry about how or on what physical machine(s)

    You pay per invocation of your function(s)

  • Copyright 2017 M/Gateway Developments Ltd

    AWS Lambda

    The first technology they supported was Node.js

  • Copyright 2017 M/Gateway Developments Ltd

    AWS Concurrency?

    Your function will be invoked in a private computation container of some sort

    Your function has that container all to itself for the duration of its execution

    No competition with other concurrent users

  • Copyright 2017 M/Gateway Developments Ltd

    Lambda Node.js Examples

    AWS examples show use of asynchronous APIs

    Node.js users of Lambda use asynchronous APIs and Async/Await

  • Copyright 2017 M/Gateway Developments Ltd

    Lambda Node.js Examples

    AWS examples show use of asynchronous APIs

    Node.js users of Lambda use asynchronous APIs and Async/Await

    But unless your Lambda function really needs to be asynchronous, why use asynchronous logic?

  • Copyright 2017 M/Gateway Developments Ltd

    Asynchronous Syntax in Other Languages?

    Available also in most other modern languages Java Python, etc

    Allows efficient use of resources when making parallel requests for resources Files, external web services etc

    But no programmer in those languages would use async logic if they didn't have to

  • Copyright 2017 M/Gateway Developments Ltd

    Async is the New Sync?

    Why would Node.js developers use asynchronous logic in Lambda functions if they don't have to? Partly because that's what you do, right?

  • Copyright 2017 M/Gateway Developments Ltd

    Async is the New Sync?

    Why would Node.js developers use asynchronous logic in Lambda functions if they don't have to? Partly because that's what you do, right? Partly because almost no synchronous APIs

    exist, particularly for: Database integration Web/REST service access

  • Copyright 2017 M/Gateway Developments Ltd

    Such APIs are possible

  • Copyright 2017 M/Gateway Developments Ltd

    Such APIs are possible

  • Copyright 2017 M/Gateway Developments Ltd

    Any other way

    To have your Node.js cake and eat it? Not everyone will want to use Lambda

    Would it be possible to create a locally-available environment where my Node.js code runs in an isolated container where concurrency isn't an issue?

  • Copyright 2017 M/Gateway Developments Ltd

  • Copyright 2017 M/Gateway Developments Ltd

    What Is QEWD?

    Essentially it's a multi-purpose Node.js-based run-time Platform

  • Copyright 2017 M/Gateway Developments Ltd

    What Is QEWD?

    Essentially it's a multi-purpose Node.js-based run-time Platform

    Creates an isolated run-time container for your message/request handler functions, allowing: CPU-intensive work Database abstractions using synchronous

    logic

  • Copyright 2017 M/Gateway Developments Ltd

    QEWD's Architecture

    Master Process Pool of Worker Processes

  • Copyright 2017 M/Gateway Developments Ltd

    QEWD's Architecture

    Master Process Handles and queues all incoming requests

    from client HTTP/REST requests via Express or Koa.js WebSocket requests via socket.io

    Returns responses to client

  • Copyright 2017 M/Gateway Developments Ltd

    QEWD's Architecture

    Pool of Persistent Worker Processes Where all the processing occurs A single queued request is dispatched to an

    available worker process Each Worker process handles a single

    request at a time

  • Copyright 2017 M/Gateway Developments Ltd

    QEWD's Architecture

    The queue / dispatcher / worker-process pool management part of QEWD is handled by a module named ewd-qoper8

  • Copyright 2017 M/Gateway Developments Ltd

    Master Node.js Process

    Queue

    Queueprocessor/dispatcher

    IncomingRequests

    QEWD Architecture

    Every incoming requestis passed from Expressand placed in a queue

    No further processingof requests occurs inthe master process

    Expressor

    Koa.js

    socket.io

    HTTPREST

    WebSocket

  • Copyright 2017 M/Gateway Developments Ltd

    QEWD Architecture Master Node.js Process

    Queue

    Queueprocessor/dispatcher

    Queue dispatcher isinvoked whenever arequest is added tothe queue

  • Copyright 2017 M/Gateway Developments Ltd

    QEWD Architecture

    Node.js Worker Process

    Master Node.js Process

    Queue

    Queueprocessor/dispatcher

    Worker processstarted if none

    available

  • Copyright 2017 M/Gateway Developments Ltd

    QEWD Architecture

    Node.js Worker Process

    Master Node.js Process

    Queue

    Queueprocessor/dispatcher

    Redis/Cache/GT.M

    QEWD &application-specific

    Modules loaded

    CustomWorkerModule

    CustomWorkerModule

  • Copyright 2017 M/Gateway Developments Ltd

    QEWD Architecture

    Node.js Worker Process

    Master Node.js Process

    Queue

    Queueprocessor/dispatcher

    CustomWorkerModule

    Node.js Worker ProcessCustomWorkerModule

    Request passedto worker

    Redis/Cache/GT.M

  • Copyright 2017 M/Gateway Developments Ltd

    QEWD Architecture

    Node.js Worker Process

    Master Node.js Process

    Queue

    Queueprocessor/dispatcher

    CustomWorkerModule

    Worker flagged as Unavailable

    Node.js Worker ProcessCustomWorkerModule

    Begin processing message

    Redis/Cache/GT.M

  • Copyright 2017 M/Gateway Developments Ltd

    QEWD Architecture Master Node.js Process

    Queue

    Queueprocessor/dispatcher

    Unavailable / processingAnother incomingrequest Node.js Worker Process

    CustomWorkerModule

    Node.js Worker ProcessCustomWorkerModule

    Redis/Cache/GT.M

  • Copyright 2017 M/Gateway Developments Ltd

    QEWD Architecture

    Node.js Worker Process

    Master Node.js Process

    Queue

    Queueprocessor/dispatcher

    CustomWorkerModule

    Unavailable / processing

    Node.js Worker ProcessCustomWorkerModule

    Node.js Worker ProcessCustomWorkerModule

    Node.js Worker ProcessCustomWorkerModule

    If worker pool size not exceeded,another worker is startedand request passed to it

    Redis/Cache/GT.M

    Redis/Cache/GT.M

  • Copyright 2017 M/Gateway Developments Ltd

    QEWD Architecture Master Node.js Process

    Queue

    Queueprocessor/dispatcher

    Unavailable / processing

    Node.js Worker ProcessCustomWorkerModule

    Node.js Worker ProcessCustomWorkerModule

    Redis/Cache/GT.M

    If entire Worker Pool is busy:

    Unavailable / processing

    Node.js Worker ProcessCustomWorkerModule

    Node.js Worker ProcessCustomWorkerModule

    Redis/Cache/GT.M

    Unavailable / processing

    Node.js Worker ProcessCustomWorkerModule

    Node.js Worker ProcessCustomWorkerModule

    Redis/Cache/GT.M

  • Copyright 2017 M/Gateway Developments Ltd

    QEWD Architecture Master Node.js Process

    Queue

    Queueprocessor/dispatcher

    If entire Worker Pool is busy:

    New requests remain

    in queue

    Unavailable / processing

    Node.js Worker ProcessCustomWorkerModule

    Node.js Worker ProcessCustomWorkerModule

    Redis/Cache/GT.M

    Unavailable / processing

    Node.js Worker ProcessCustomWorkerModule

    Node.js Worker ProcessCustomWorkerModule

    Redis/Cache/GT.M

    Unavailable / processing

    Node.js Worker ProcessCustomWorkerModule

    Node.js Worker ProcessCustomWorkerModule

    Redis/Cache/GT.M

  • Copyright 2017 M/Gateway Developments Ltd

    QEWD Architecture Master Node.js Process

    Queue

    Queueprocessor/dispatcher

    Unavailable / processing

    Node.js Worker ProcessCustomWorkerModule

    Node.js Worker ProcessCustomWorkerModule

    Redis/Cache/GT.M

    As soon as a worker is available again,a queued message can be passed to it

    Unavailable / processing

    Node.js Worker ProcessCustomWorkerModule

    Node.js Worker ProcessCustomWorkerModule

    Redis/Cache/GT.M

    Available

    Node.js Worker ProcessCustomWorkerModule

    Redis/Cache/GT.M

  • Copyright 2017 M/Gateway Developments Ltd

    QEWD Architecture Master Node.js Process

    Queue

    Queueprocessor/dispatcher

    Finished

    Node.js Worker ProcessCustomWorkerModule

    Node.js Worker ProcessCustomWorkerModule

    Node.js Worker ProcessCustomWorkerModule

    Redis/Cache/GT.M

    A user's handler function signalscompletion using the function:

    finished(responseObject);

    This returns the responseobject to the master

    process

  • Copyright 2017 M/Gateway Developments Ltd

    QEWD Architecture Master Node.js Process

    Queue

    Queueprocessor/dispatcher

    Finished

    Node.js Worker ProcessCustomWorkerModule

    Node.js Worker ProcessCustomWorkerModule

    Node.js Worker ProcessCustomWorkerModule

    And the response isreturned to the client

    that sent the original request

    (via Express/Koa/socket.io)

    Redis/Cache/GT.M

  • Copyright 2017 M/Gateway Developments Ltd

    QEWD Architecture Master Node.js Process

    Queue

    Queueprocessor/dispatcher

    Available

    Node.js Worker ProcessCustomWorkerModule

    The finished() functionalso automatically returnsthe worker process back

    to the available pool

    So it can now handlethe next queued request

    Redis/Cache/GT.M

  • Copyright 2017 M/Gateway Developments Ltd

    QEWD Architecture Master Node.js Process

    Queue

    Queueprocessor/dispatcher

    Available

    Node.js Worker ProcessCustomWorkerModule

    Worker processes, once started, are persistent

    No start-up / tear-down cost

    Redis/Cache/GT.M

  • Copyright 2017 M/Gateway Developments Ltd

    QEWD Architecture Master Node.js Process

    Queue

    Queueprocessor/dispatcher

    Node.js concurrency is handledby the master process.

    100% asynchronous logic

    The master process doesalmost nothing

    The perfect Node.js application:no CPU-intensive or long-

    running tasks, so veryhigh-performance

    MultipleConcurrentIncomingrequests

  • Copyright 2017 M/Gateway Developments Ltd

    QEWD Architecture Master Node.js Process

    Queue

    Queueprocessor/dispatcher

    Node.js concurrency is handledby the master process.

    100% asynchronous logic

    The master process doesalmost nothing

    The perfect Node.js application:no CPU-intensive or long-

    running tasks, so veryhigh-performance

    All the actual work happens in theisolated worker processes

    MultipleConcurrentIncomingrequests

  • Copyright 2017 M/Gateway Developments Ltd

    QEWD Architecture Master Node.js Process

    Queue

    Queueprocessor/dispatcher

    Available

    Node.js Worker ProcessCustomWorkerModule

    Worker processes only handlea single request at a time

    Completely isolated run-timeenvironment for handler functions

    No need for concerns aboutNode.js concurrency, so

    synchronous APIs can be used

    Redis/Cache/GT.M

  • Copyright 2017 M/Gateway Developments Ltd

    QEWD Architecture Master Node.js Process

    Queue

    Queueprocessor/dispatcher

    Available

    Node.js Worker ProcessCustomWorkerModule

    Redis/Cache/GT.M

    Long-running or CPU-intensivelogic has no direct impact on

    other worker processes

  • Copyright 2017 M/Gateway Developments Ltd

    Why not just use child_process?

    To avoid the very high performance cost of starting and tearing down each child process At least 30ms to start a child_process

    Each QEWD worker is a child_process But keeps running, and is re-used

  • Copyright 2017 M/Gateway Developments Ltd

    Why not just use Cluster?

    Cluster simply spreads the concurrent requests across a pool of persistent child_processes If one of them is CPU intensive, all other

    requests in that cluster are held up QEWD forces each child_process to

    handle a single request only Creates an isolated run-time environment for

    each request

  • Copyright 2017 M/Gateway Developments Ltd

    Why not just use RabbitMQ or ZeroMQ?

    I just want to just use a single technology: Node.js / JavaScript

    Initial tests showed no performance benefit in using ZeroMQ

  • Copyright 2017 M/Gateway Developments Ltd

    Performance?

    ewd-qoper8 module in isolation: Handles the queue/master/worker architecture

    Raspberry Pi 3 Model B 4 core CPU Readily-available commodity item Low-cost

    Easily-replicable benchmark

  • Copyright 2017 M/Gateway Developments Ltd

    Performance

    ewd-qoper8 benchmark script provided: How many workers? 3 How many messages? 500,000 Create a steady state of messages added to

    the queue as they're being processed: Add a batch of 622 messages at a time Wait 100ms between each batch

    Messages are sent to workers which echo them straight back

  • Copyright 2017 M/Gateway Developments Ltd

    Performance

    5,800 messages/second sustained throughput

    Limiting factor is master process hitting 100% CPU

    Workers only 30% CPU, so plenty of capacity to do real work at this rate

  • Copyright 2017 M/Gateway Developments Ltd

    What can QEWD be Used For?

    Full-stack platform for browser & Native mobile applications WebSocket or Ajax messaging

    Seamlessly swap between transports All automated and abstracted out of the way

    With security also automated for you too

  • Copyright 2017 M/Gateway Developments Ltd

    What can QEWD be Used For?

    API server for REST applications Optional session management

    Automated token-based authentication Server-side (persistent JSON) session storage

    Or automated JSON Web Token based: Authentication Client-side session storage

  • Copyright 2017 M/Gateway Developments Ltd

    What can QEWD be Used For?

    MicroService Platform With each MicroService supported by its own

    shared or dedicated QEWD server Using JWTs

    Shared secret on each QEWD server Using socket.io to provide high-performance,

    persistent connections between QEWD servers

    Secured over HTTPS if required

  • Copyright 2017 M/Gateway Developments Ltd

    QEWD MicroService Fabric

    ewd-qoper8queue

    Express

    Node.js

    socket.io

    CacheGT.M,YottaDBRedisNode.js

    WorkerProcess

    CacheGT.M,YottaDBRedisNode.js

    WorkerProcess

    CacheGT.M,YottaDBRedisNode.js

    WorkerProcess

    ewd-qoper8queue

    Express

    Node.js

    socket.io

    CacheGT.M,YottaDBRedisNode.js

    WorkerProcess

    CacheGT.M,YottaDBRedisNode.js

    WorkerProcess

    CacheGT.M,YottaDBRedisNode.js

    WorkerProcess

    ewd-qoper8queue

    Express

    Node.js

    socket.io

    CacheGT.M,YottaDBRedisNode.js

    WorkerProcess

    CacheGT.M,YottaDBRedisNode.js

    WorkerProcess

    CacheGT.M,YottaDBRedisNode.js

    WorkerProcess

    User authentication

    Demographics

    Pharmacy

    ewd-qoper8queue

    Express

    Node.js

    socket.io

    CacheGT.M,YottaDBRedis

    Node.jsWorkerProcess

    CacheGT.M,YottaDBRedis

    Node.jsWorkerProcess

    CacheGT.M,YottaDBRedis

    Node.jsWorkerProcess

    Client Orchestration

    HTTPSWebSocketConnections

  • Copyright 2017 M/Gateway Developments Ltd

    What can QEWD be Used For?

    Federation platform Providing a REST or WebSocket-based

    middle tier to multiple REST-based back-end servers

    of any type, not just QEWD servers One request in

    Routed to all back-end systems and responses combined

    Performing a "dance" between federated servers

  • Copyright 2017 M/Gateway Developments Ltd

    Federated Access to OpenEHRBrowser QEWD

    GT.M orRedis

    ewd-qoper8queue

    qewd-rippleModule

    Express

    OpenEHRServer

    AQL overHTTP(S)

    Worker

    Puls

    eTile

    UI

    OpenEHRServer

  • Copyright 2017 M/Gateway Developments Ltd

    What can QEWD be Used For? Interface to particular databases known as

    Global Storage databases GT.M / YottaDB Cache / IRIS Redis (ewd-globals-redis module)

    All of which are abstracted as: Persistent JavaScript Objects Fine-grained Document Database

    Accessible at any level down to individual name/value pair anywhere in a JSON object

  • Copyright 2017 M/Gateway Developments Ltd

    QEWD.js Resilient / Audit mode

    When enabled, permanent record kept in the database of:

    Queued requests Processing status Response(s)

    If QEWD restarted, database is examined for queued, unprocessed requests

    Automatically re-queued Can specify retention period of database records

  • Copyright 2017 M/Gateway Developments Ltd

    QEWD.js

    Supports all browser-side JavaScript frameworks

    Some cool 3rd-party tooling and support available for React and Vue.js

  • Copyright 2017 M/Gateway Developments Ltd

    Have Your Node.js CakeAnd Eat It Too

    http://qewdjs.com

Recommended

View more >