crosstalk overview (austin javascript/node.js meetup) - 2012-12-07

Crosstalk

The Internet is broken

The Internet is brokenfor non-programmers

Composability

ComposabilityWhat skills do you need?

Internet Scale

Internet ScaleDistributed Systems

Fallacies of Distributed Computing

● the network is reliable● latency isn't a problem● bandwidth isn't a problem● the network is secure● topology won't change● the administrator will know what to do● transport cost isn't a problem● the network is homogeneous● the system is atomic/monolithic● the system is finished● business logic can and should be centralized

Internet ScaleDistributed Systems

Internet ScaleDistributed Systems

Never Done

How should we design our composable distributed software system?

non-programmers

Crosstalk

The way you should be doing things should be easy and the way you shouldn't do things should

be difficult.

-- Ryan Dahl, original Node.js presentation, JSConf 2009

Modular - Composable - Resilient

Crosstalk

Modular - Composable - Resilientin JavaScript

Crosstalk

Terminology

Worker Prime SwarmTerminology

Worker Prime SwarmTerminology

Worker Prime SwarmTerminology

Worker Prime SwarmTerminology

Architecture

Crosstalk Swarm

Crosstalk Swarm

Prime

Prime

Prime

Prime Prime

Prime Prime

Prime

Prime

Prime

Prime

Prime

Prime

Prime

Every Prime lies on the "surface area"

Every Prime lies on the "surface area"

Gossip

Gossip

Gossip

Gossip

Gossip

Prime Lifecycle

Prime LifecycleLet it crash

Φ accrual failure detection

Prime LifecycleLet it Make it crash

Worker Lifecycle

Worker LifecycleLet it crash

Worker LifecycleLet it Make it crash

Getting things done

Getting things donei.e: building workers

index.jspackage.json

Hello distributed world!

Hello distributed world!

"magic" global variable

Hello distributed world!

"magic" global variable message name

Hello distributed world!

"magic" global variable message namefunction to execute when

a message arrives

Hello distributed world!

"magic" global variable message namefunction to execute when

a message arrives

name of message to emit

Alice worker #1

Alice worker #2

Alice and a sandwich

Alice worker #1

Alice worker #2

Alice and a sandwich

it's just JavaScript

Alice worker #1

Alice worker #2

Alice and a sandwich

it's just JavaScript

emit "sandwich" messages

Alice worker #1

Alice worker #2

Alice and a sandwich

it's just JavaScript

listen for "sandwich" messages

emit "sandwich" messages

Alice's worker

Bob's worker

Alice and Bob

Alice's worker

Bob's worker

Alice and Bob

public scope

Alice's worker

Bob's worker

Alice and Bob

public scope

addressed message

Alice's worker

Bob's worker

Alice and Bob

public scope

addressed message

address is omittedwhen declaring a message listener

Alice's worker

Alice and Bob using request-reply (1 of 2)

Alice's worker

Alice and Bob using request-reply (1 of 2)

addressed message

Bob's worker

Alice and Bob using request-reply ( 2 of 2 )

Bob's worker

Alice and Bob using request-reply ( 2 of 2 )

address is omittedwhen declaring a message listener

Eve's worker

Eve enters... ( 1 of 2 )

Eve's worker

Eve enters... ( 1 of 2 )

anyone can send Alice a message

Alice's improved worker

Eve enters... ( 2 of 2 )

Alice's improved worker

Eve enters... ( 2 of 2 )

only messages from @bob (Bob) are

processed

Alice's configuration file

Eve-n better

Alice's refactored worker

Alice's configuration file

Eve-n better

Alice's refactored worker

import Crosstalk functionality using CommonJS module

pattern

Alice's configuration file

Eve-n better

Alice's refactored worker

factor out dependencies

import Crosstalk functionality using CommonJS module

pattern

Carl's worker

Generic Bob ( 1 of 2 )

Carl's worker

Generic Bob ( 1 of 2 )

send data with your messages

Previous Bob worker

Generic Bob ( 2 of 2 )

Generalized Bob worker

Previous Bob worker

Generic Bob ( 2 of 2 )

Generalized Bob worker

access data in messages

Previous Bob worker

Generic Bob ( 2 of 2 )

Generalized Bob worker

access data in messages

Previous Bob worker

Generic Bob ( 2 of 2 )

Generalized Bob worker

access data in messages

C-C-C-C-C-Callback!

New Carl worker

Bob worker using callback

C-C-C-C-C-Callback!

New Carl worker

Bob worker using callback

callback will reply to sender

C-C-C-C-C-Callback!

New Carl worker

Bob worker using callback

callback will reply to senderif sender provides it

C-C-C-C-C-Callback!

New Carl worker

Bob worker using callback

callback will reply to senderif sender provides it

by convention, the first parameter is always an error

C-C-C-C-C-Callback!

New Carl worker

Bob worker using callback

callback will reply to senderif sender provides it

by convention, the first parameter is always an error

second param is the response

C-C-C-C-C-Callback!

New Carl worker

Bob worker using callback

callback will reply to senderif sender provides it

by convention, the first parameter is always an error

second param is the response

no error

C-C-C-C-C-Callback!

New Carl worker

Bob worker using callback

callback will reply to senderif sender provides it

by convention, the first parameter is always an error

second param is the response

no errorsandwich response

Secure Callback

More secure Carl worker

Secure Callback

More secure Carl worker

scope for the callback response

Secure Callback

More secure Carl worker

scope for the callback response

only @bob can respond to this request

How many sandwiches?

Alice's generic web sandwich worker

Node.js style HTTP request handler

function

Node.js style HTTP request handler

function

built-in Crosstalk API for web services

Node.js style HTTP request handler

function

built-in Crosstalk API for web services

subdomain (required)

Node.js style HTTP request handler

function

built-in Crosstalk API for web services

subdomain (required)

request listener (required)

Node.js style HTTP request handler

function

built-in Crosstalk API for web services

subdomain (required)

request listener (required)

How many secure sandwiches?

Alice's https web sandwich worker

How many secure sandwiches?

Alice's https web sandwich worker

httpsOnly flag (optional)

Persistent sandwich

Persistent sandwich

another Crosstalk API

Persistent sandwich

another Crosstalk API

refactored dependencies

Recoverable sandwich

There's more...

Crosstalk runs on Crosstalk

Modular - Composable - Resilientin JavaScript

Crosstalk

https://github.com/crosstalk

Crosstalk

https://github.com/crosstalk

npm install -g crosstalk-idenpm install -g crosstalk-cli

Crosstalk