vortex cloud beyond cloud messaging

Angelo Corsaro, PhDChief Technology Officer

angelo.corsaro@prismtech.com

Beyond Cloud Messaging…Cloud

Infrastructure as as Service (IaaS): Elastic compute fabric that you can “consume” as opposed to own and manage, e.g., Azure, Amazon EC2, Google Compute Engine, Linode, etc.

Platform as a Service (PaaS): Enhances IaaS with a series of managed middleware and storage services ready to be “consumed” by your application, e.g., Azure Service Bus, Amazon Dynamo, Vortex Cloud

Software as a Service (IaaS): A full application as a service available for you to be consumed, e.g., SalesForces

Cloud ComputingThe main idea behind cloud computing incrementally reduce the number of layers that you are required to own and manage in order to build applications

Cloud Messaging replaces local infrastructure and integration technologies with an Ubiquitous Internet Service

Cloud Messaging allows companies to elastically and transparently deal with changes of workload and scale

Cloud MessagingCloud Messaging provides a ubiquitous and universally accessible “Internet Service” for distributing messages from producers to consumers

Virtually all the Cloud Messaging implementation available on the market focus on:

Reliable Messaging

Device-to-Cloud communication pattern, i.e., no cloud no communication

Low update rates

Additionally, a large set of Cloud Messaging implementations impose:

HTTP-based communication, e.g. Amazon SQS

Limit on message payload

Limit on deployment across “regions”

Limit on the Deployment/Licensing model, e.g. no OEM options

“Traditional” Cloud Messaging

“Traditional” Cloud messaging was designed to address the requirements of IT systems. As such its main focus is on highly reliable messaging and device-to-cloud interactions

The surge of the Internet of Things has posed a series of new requirements for Cloud Messaging, such as:

- Real-Time Data Delivery, i.e. freshness of data is more important than its reliability

- High Data Rates

- Last Value Reliability

- Support for Cloud and Fog Computing architectures

- Support for embedded and real-time platforms

- Network Efficiency

Beyond Traditional Cloud Messaging?

Vortex Cloud

4VORTEX Cloud provides a universally accessible “Internet Service” for sharing data between DDS-enabled applications

VORTEX Cloud

Elastic and Fault-Tolerant

Public/Private Clouds Deployments

Reliable and Best-Effort Data Sharing

Last value Reliability

Unicast and Multicast Communication

Support for enterprise, embedded, and mobile platforms

Source Filtering

Customisable Load-Balancing

VORTEX Cloud

Efficient Binary Protocol (DDSI)

Multiple Transports:- UDP/IP- TCP/IP- WebSockets

C/C++, Java, Scala, JavaScript, CoffeScript, and C# API

Connectivity to MQTT, AMQP, etc., via VORTEX Gateway

VORTEX Cloud

VORTEX supports both the Cloud and the Fog Computing Paradigm

VORTEX natively supports:

- Device-to-Device Communication

- Device-to-Cloud Communication

Cloud, Fog and Edge Computing

Cloud Computing

Fog Computing

Device-to-Cloud Communication

Device-to-Device Communication

Fog-to-Cloud Communication

Cloud-to-Cloud Communication

Collect | Store | Analyse | Share

Fog Computing

Performance

Device-2-Device

• Peer-to-Peer Intra-core latency as low as 8 µs

• Peer-to-Peer latency as low as 30 µs

• Point-to-Point throughput well over 2.5M msg/sec

Device-2-Cloud

• Routing latency as low as 4 µs

• Linear scale out

• 44K* msgs/sec with a single router, 4x times more the average Tweets per second in the world (~6000 tweets/sec)!

*2048 bytes message payload

Deployment Models

Cloud ComputingDevice-to-Cloud Communication

Peer-to-Peer (Brokerless)

Device communicate peer-to-peer within a fog-domain and through Cloud across fog-domains

Some device concurrently communicate with peers and the cloud

Cloud and Fog ComputingDevice communicate peer-to-peer within a fog-domain

A Vortex-Fog controls which data is exchanged with the could

Fog Computing Fog ComputingFog Computing

Fog Computing

Fog domain are transparently federated by Vortex-Fog instances

A Vortex-Fog controls which data is exchanged with the could

Fog Computing

Vortex Platform

Specialised device implementations optimally addressing requirements of OT and IT platforms

VORTEX can readily deal with data ingestion seamlessly integrating with other protocols, e.g. MQTT, CoAP, etc.

VORTEX leverages the DDS standard for interoperability and uniquely extends it with support for Internet Scale systems, mobility and Web 2.0 applications

The VORTEX Platform

PaaS/MaaS

Vortex Cloud Abstraction

Vortex provides a Distributed Data Space abstraction where applications can autonomously and asynchronously read and write data

Its built-in dynamic discovery isolates applications from network topology and connectivity details

DDS’ Data Space is completely decentralised

High Level Abstraction

DDS Global Data Space

Data Writer

Data Reader

Data Writer

TopicAQoS

TopicBQoS

TopicCQoS

TopicDQoS

DDS supports the definition of Common Information Models.

These data models allow to naturally represent physical and virtual entities characterising the application domain

DDS types are extensible and evolvable, thus allowing incremental updates and upgrades

Data Centricity

A Topic defines a domain-wide information’s classA Topic is defined by means of a (name, type, qos) tuple, where

• name: identifies the topic within the domain

• type: is the programming language type associated with the topic. Types are extensible and evolvable

• qos: is a collection of policies that express the non-functional properties of this topic, e.g. reliability, persistence, etc.

TopicTypeName

struct TemperatureSensor { @key long sid; float temp; float hum;}

4Vortex “knows” about application data types and uses this information provide type-safety and content-based routing

Content Awarenessstruct TemperatureSensor { @key long sid; float temp; float hum;}

sid temp hum101 25.3 0.6507 33.2 0.7913 27,5 0.55

1307 26.2 0.67

“temp > 25 OR hum >= 0.6”

sid temp hum101 25.3 0.6507 33.2 0.7

1307 26.2 0.67

TempSensor

For data to flow from a DataWriter (DW) to one or many DataReader (DR) a few conditions have to apply:

The DR and DW domain participants have to be in the same domain

The partition expression of the DR’s Subscriber and the DW’s Publisher should match (in terms of regular expression match)

The QoS Policies offered by the DW should exceed or match those requested by the DR

Quality of ServiceDomain

Participant

DURABILITY

OWENERSHIP

DEADLINE

LATENCY BUDGET

LIVELINESS

RELIABILITY

DEST. ORDER

Publisher

DataWriter

PARTITION

DataReader

Subscriber

DomainParticipant

offered QoS

Topicwrites reads

Domain Idjoins joins

produces-in consumes-from

RxO QoS Policies

requested QoS

Building App with Vortex

Example1: Strong Typing and Extensibility

The first step required to build a Vortex application is to define the kind of information associated with a Topic

This can be done using either IDL or Google Protocol Buffer

Topic Definition

Temperature Sensor

message TempSensor { option (.omg.dds.type) = {name: "dds.TempSensor"}; required string sid = 1 [(.omg.dds.member).key = true]; required float temp = 2; }

struct TempSensor { string sid; float temp;}; #pragma keylist TempSensor sid

IDL Google Protocol Buffer

Data Producerobject TempSensorPub { val builder = TempSensor.newBuilder() def main(args: Array[String]): Unit = { if (args.length > 1) { val sid = args(0) val period = args(1).toInt val topic = Topic[TempSensor]("TempSensor") val dw = DataWriter[TempSensor](topic) while (true) { val s = readTempSensor(sid) dw.write(s) println(show (s)) Thread.sleep(period) } } else { println("USAGE:\n\tTempSensorPub <sid> <period>") } }}

def readTempSensor(sid: String): TempSensor = { val minTemp = -20 val tdelta = 100; val minHum = 0.1F val hdelta = 9 val t = (tdelta*Math.random()).toInt + minTemp val h = (hdelta*Math.random()).toInt + minHum builder.setSid(sid) builder.setTemp(t) builder.setHum(h) builder.build() }

Data Consumerobject TempSensorSub { def main(args: Array[String]): Unit = { val topic = Topic[TempSensor]("TempSensor") val dr = DataReader[TempSensor](topic) dr.listen { case DataAvailable(_) => dr.take() .filter(s => s.getData != null) .map(_.getData) .foreach(ts => println(show(ts))) } Thread.currentThread().join() }}

Suppose that we want now to use a more advanced sensor that provides an estimate of the humidity in addition to the temperature

At the same time, while extending the type of our Temperature Sensor we want older application to continue to work seamlessly with the new sensor

Dealing with Type Evolution

The humidity attribute is declared optional to allow older consumer to match this type

At the same time, new consumer will be able to detect that old temperature sensor are not providing humidity value

Extended Temperature Sensor Type

message TempSensor { option (.omg.dds.type) = {name: "dds.TempSensor"}; required string sid = 1 [(.omg.dds.member).key = true]; required float temp = 2; optional float hum = 3; }

Example2: Real-Time Web Applications

Vortex Cloud provides a very effective way of sharing data between HTML5/JavaScript Applications

Additionally, Vortex Cloud can be used to seamlessly share data between Web and embedded applications

Real-Web Apps with Vortex Cloud

4Let’s see the steps required to build a Web Chat that may look like this

But before let’s play with it a bit

Web Chat

demo.prismtech.com

When using Vortex Cloud with web application i topic types can be declared JavaScript/CoffeScript

Topic Declaration

# The post type used by the chat applicationclass Post constructor: (@name, @msg) ->

The Chat CoffeeScript# Create useful alias for coffez and jQueryroot = thisz_ = coffez$ = jQuery

server = “ws://demo-eu.prismtech.com:9999"

# The post type used by the chat applicationclass Post constructor: (@name, @msg) ->

# Create the runtimeruntime = new dds.runtime.Runtime()

# Define the Post topic used to send and receive chat postspostTopic = new dds.Topic(0, "Post")

# Define the QoS for the DataReader/WriterdrQos = new dds.DataReaderQos(dds.Reliability.Reliable)dwQos = new dds.DataWriterQos(dds.Reliability.Reliable)

The Chat CoffeeScriptpostReader = z_.NonepostWriter = z_.None

avatar = "avatar" + Math.floor((Math.random() * 10000) + 1);

# Add post to the chat and format it to show it is from mecreateMyPost = (post) -> ...

# Add post to the chat and format it to show it is from otherscreateOtherPost = (post) -> ...

# Add post to the chat and format it to show it is from othersprocessPost = () -> msg = $("#ChatMessage").val() post = new Post(avatar, msg) # Publish the post (notice that postWriter is an Option Monad) # Take a look at (http://en.wikibooks.org/wiki/Haskell/Understanding_monads/Maybe) # or (http://www.scala-lang.org/api/2.11.0/index.html#scala.Option) postWriter.map((dw) -> dw.write(post)) $("#ChatMessageList").append(createMyPost(post)) $("#ChatMessage").val("")

The Chat CoffeeScript# Deal with click and keys events…

$("#ChatMessage").keyup( (e) -> if(e.keyCode is 13) then processPost())

$("#SendMsgButton").click( (evt) -> console.log("Send Button has been clicked") processPost())

$("#SelectAvatarButton").click( (evt) -> s = $("#AvatarName").val() if (s isnt "") avatar = s)

The Chat CoffeeScript# Handle the runtime onconnect eventruntime.onconnect = () -> # Create DataReader and DataWriter for our posts dr = new dds.DataReader(runtime, postTopic, drQos) dw = new dds.DataWriter(runtime, postTopic, dwQos)

# Register a listener with the data reader to post messages # in our chat dr.addListener( (post) -> if (post.name isnt avatar) $("#ChatMessageList").append(createOtherPost(post))) postReader = z_.Some(dr) postWriter = z_.Some(dw)

connectRuntime = () -> $("#AvatarName").val(avatar) runtime.connect(server, "uid:pwd")

$(document).ready(() -> connectRuntime())

Putting it all Together

Vortex Cloud goes beyond traditional Cloud Messaging solutions by providing support for:

- Real-Time Data Delivery, i.e. freshness of data is more important than its reliability

- High Data Rates

- Last Value Reliability

- Support for Cloud and Fog Computing architectures

- Support for embedded and real-time platforms

- High Network Efficiency

Vortex Cloud is the ideal PaaS infrastructure for Industrial and Consumer IoT applications

Summing Up

vortex cloud beyond cloud messaging

cloud communicationcloud

cloud communicationdevice

cloud communication

cloud interactionsthe

service iaas

consumerscopyright prismtech

service available

accessible internet

Software

google cloud messaging report

automotive manufacturer selects cloud-based messaging...

4talk im - a cloud messaging service

scaling data distribution service (dds) to internet scale...

1111 symsoft cloud messaging white paper

ibm message hub: cloud-native messaging

vortex-topology messaging: data vortex for high ... ·...

ibm interconnect 2016 - hybrid cloud messaging

messaging patterns in the cloud

cloud messaging service: technical overview

messaging for the cloud and microservices

cloud messaging with opensplice dds

messaging: harnessing the cloud

cloud messaging in android

cloud messaging and collaboration services

firebase cloud messaging for ios

google cloud messaging for android apps

rabbitmq messaging in the cloud

tcs cloud messaging for healthcare

a journey into google cloud messaging