omg dds tutorial - part i

Angelo Corsaro, Ph.D.Product Strategy & Marketing Manager

OMG RTESS and DDS SIG [email protected]

Delivering Performance, Openness, and Freedom

OpenSplice DDS

OMG DDS Tutorial - Part I

mailto:[email protected]


Age

nda

Motivating Forces

The Standard

Architectural Principles

Programming Model

Concluding Remarks

Complex Information Management

Scalability, Persistence, Security

Distributed Systems

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.

© 2009, PrismTech. All Rights Reserved

The Need for Data DistributionGeographical ScaleTime Scale

Near Real-Time Fault-Tolerant Information

Processing

Throughput, Availability

Real-Time Information Processing

DeterminismParallelism

Systemic Signal

Processing

Data Processing

Parallel Systems

Data Distribution

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Data Exchange / Distribution‣ Network Centric Architectures are emerging as a key trend for next generation military and civil

system of systems

‣ Efficient, scalable and QoS-enabled data dissemination is an enabling technology for Network Centric Systems

Adapted from “The Future of AWACS”,

by LtCol Joe Chapa

Joint Forces

Global Info Grid

Joint Forces

Global Info Grid

The Right Information => To the Right People => At the Right Time

Age

nda

Motivating Forces

The Standard


Programming Model

Concluding Remarks

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Addressing Data Distribution Challenges

The OMG DDS Standard‣ Introduced in 2004 to address the Data

Distribution challenges faced by a wide class of Defense and Aerospace Applications

‣ Key requirement for the standard were its ability to deliver very high performance while seamlessly scaling from embedded to ultra-large-scale deployments

‣ Today recommended by key administration worldwide and widely adopted across several different application domains, such as, Automated Trading, Simulations, SCADA, Telemetry, etc.

DDS is standard designed to address the data-distribution challenges across a wide class of Defense and Aerospace Applications

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


The OMG Data Distribution Service (DDS)

DDS v1.2 API Standard‣ Language Independent, OS and HW architecture

independent

‣ DCPS. Standard API for Data-Centric, Topic-Based, Real-Time Publish/Subscribe

‣ DLRL. Standard API for creating Object Views out of collection of Topics

DDSI/RTPS v2.1 Wire Protocol Standard‣ Standard wire protocol allowing interoperability

between different implementations of the DDS standard

Object/Relational Mapping

Ownership DurabilityContent

Subscription

Minimum Profile

Data Centric Publish/Subscribe (DCPS)

Data Local Reconstruction Layer (DLRL)

DDS Interoperability Wire Protocol

Application

UDP/IP

Real-Time Publish/Subscribe Protocol

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


OMG DDS RecommendationsMandated/Recommended by key Administrations

‣US Navy: Open Architecture

‣DISR/DISA: Net-centric Systems

‣EuroControl: Air Traffic Control Center Operational Interoperability

‣QinetiQ: Recommending DDS for VSI

A Great Time for DDS!

! The infrastructure evolution cycle is churning…– New -> Emerging -> Standard -> Commodity

– Middleware is emerging as OS declines

! …DDS is maturing…– OMG focus

– Wire spec

– Tools

– Enterprise integration

– Multiple products fielded

– Deployed applications!

! …and adoption is on the rise– Navy

– DISR

– FCS/SoSCOE

– Many significant applications

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


OMG DDS AdoptionsAdopted by Most Challenging International Programs

‣US FCS/SOSCOE

‣European Next Generation Flight Data Processor

‣US ZUMWALT [DDG(1000)]

‣VTID adopting DDS for the architectural demonstrator

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


OMG DDS ApplicabilityAdapted from NSWC-DD Open Architecture Documentation

Non Real-Time Soft Real-Time Hard Real-Time Extreme Real-Time

Com

mun

icat

ion

Tech

nolo

gies

Sta

ndar

ds

The DDS is the only technology that spans across the board.

Web SVCs

Java RTSJ

JMS

CORBA RT-CORBA

OMG Data Distribution Service (DDS)

MPI

It guarantees exceptional real-time behavior, while providing unparalleled level of throughput !

Message Passing Interface

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Some DDS Use CasesDefense

‣ Combat Management Systems

‣ Flycatcher system

‣ Unmanned Vehicles (Air + Land)

‣ Tactical Links

‣ Radar Processing

‣ Submarine Systems

‣ Future Combat Systems

‣ Simulation

SCADA/Utilities

‣ Industrial Automation

‣ Power Grids

‣ Rocket Launch Systems

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Some DDS Use Cases

Transportation

‣ Air Traffic Management/Control

‣ Metropolitan Traffic Management

‣ Underground Metropolitan Transportation

Financial Services

‣ Automated Trading Firms

‣ Compliance Systems

‣ Market Data Platforms

Age

nda

Motivating Forces

The Standard


Programming Model

Concluding Remarks

D e l i v e r i n g P e r f o r m a n c e , O p e n n e s s , a n d F r e e d o m

OpenSplice DDS

Loosely Coupled Technology

Publisher

Subscriber

Publisher

Publisher

Subscriber

Subscriber

Subscriber

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Counting the “W’s” of Coupling

Client

Request

Reply

Server The “4Ws” of Client/Server‣ Who+Where: Space Coupling

‣ What: Structural Coupling

‣ When: Time CouplingClient/Server CORBA, COM+, Java RMI, .Net Remoting, Web SVCs

DDS

A Single “W” for DDS‣ What: Structural Coupling

DDS features dynamic discovery and full time decoupling!

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


DDS vs. Client-ServerClient/Server‣ Tight Coupling (4W)

‣ Complex Deployment

‣ Inherently One-to-One

‣ Fragile to FaultServer

Client

Server

Client

Server

Client

Server

Client

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Counting the “W’s” of CouplingClient/Server‣ Tight Coupling (4W)

‣ Complex Deployment

‣ Inherently One-to-One

‣ Fragile to Fault

DDS‣ Loosely Coupled (1W)

‣ Plug & Play

‣ Inherently Many-to-Many

‣ Fault Resilient

Server

Client

Server

Client

Server

Client

Server

Client

Publisher

Subscriber

Publisher

Publisher

Subscriber

Subscriber

Subscriber


OpenSplice DDS

High Performance/Availability Pub/Sub

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


High Performance Pub/Sub

‣ Fully distributed, Peer-to-Peer Communication

‣No Single Point of Failure

‣No Single Point of Bottleneck

‣Multicast-enabled

‣High performance and highly scalable

‣High availability

‣hot-swap

‣hot-hot architecture

The right data, at the right place, at the right time

-- All the Time.

Publisher

SubscriberPublisher

Publisher

Subscriber

Subscriber

Brokers


OpenSplice DDS

Data-CentricPub/Sub



Subscription

Minimum Profile




Application

UDP/IP


Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Data-Centric Pub/Sub

‣ Distributed Relational Data Model

‣ Local Queries

‣ Continuous Queries / Content Based Subscriptions

‣Windows

‣ Object/Relational Mapping

‣ Support for a subset of SQL-92

Perfect Blend of Data-Centric and Real-Time Publish/Subscribe Technologies

‣ Data-Centric Features are built-in and don’t rely on an external DBMS

‣ Providing thus performance, scalability, and availability

A

B

CD

E

F

m

J

K

Publisher

SubscriberPublisher

Publisher

Subscriber

Subscriber

DBMS

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


“Visualizing” Data-Centric Pub/Sub

A

B

CD

E

F

m

J

K

Publisher

D

E

J

K

A F

CD

A

B D

E

J

K

Subscriber

Subscriber

Subscriber

CD

A

B

Publisher

J

K

A F

Publisher

D

E

A

B

CD

E

F

m

J

K

Publisher

SubscriberPublisher

Publisher

Subscriber

Subscriber

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Organizing Information

‣ All DDS communication is happens within a Domain

‣ Domain can divided into Partitions

‣ Topics are published and subscribed across on or more Partitions

DomainPartition

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Topics and Data-Centric Pub/Sub‣ Topics. Unit of information exchanged between

Publisher and Subscribers.

‣ Data Types. Type associated to a Topic must be a structured type expressed in IDL

‣ Topic Instances. Key values in a datatype uniquely identify a Topic Instance (like rows in table)

‣ Content Awareness. SQL Expressions can be used to do content-aware subscriptions, queries, joins, and correlate topic instances

struct TempSensor { long tID; float temp; float humidity; };#pragma keylist TempSensor tID

tID temp humidity

1

2

3

21 62

27 78

25.5 72.3

TempSensor

Instances

tID

temp

humidity

1

21

62

tID

temp

humidity

1

23

65

tID

temp

humidity

1

22

63

Timet1 t2 t3

Samples

Topic Type

Topic

SELECT * FROM TempSensor t WHERE t.temp > 25

tID temp humidity

2

3

27 78

25.5 72.3

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Distributed Relational Information Modeling

‣ Topic Keys can be used to identify instances as well as relationships

‣ Relationships can be navigated by relying on a subset of SQL 92

‣ One-to-many relationships can be captured using foreign keys

‣Many-to-many relationships need to be modeled using a topics

‣ Keys can be represented by an arbitrary number of Topic fields

tID: inttemp: floathumidity: floatrID: int

TempSensorrID: intwidth: floatheight: floatlength: floatfID: int

Room

fID: intlevel: intopen: int

Floor

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Data Centric Publish/Subscribe

Fully Distributed Global Data Space

struct TempSensor { int tID; float temp; float humidity; };#pragma keylist TempSensor tID

tID temp humidity

1 18 60

tID temp humidity

1

2

3

18 60

22 75

21 71

tID temp humidity

3 21 71

TempSensor

TempSensor

tID temp humidity

2 22 75

TempSensor

Publishers Subscribers

TempSensor

tID temp humidity

2

3

22 75

21 71

TempSensor

SELECT * FROM TempSensor t WHERE s.temp > 20

tID temp humidity

1 18 60

TempSensor

s.tID == 1

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Data Centric Publish/Subscribe

Fully Distributed Global Data Space

tID temp humidity

1

2

3

21 62

22 75

21 71

tID temp humidity

3 21 71

TempSensor

TempSensor

tID temp humidity

2 22 75

TempSensor

Publishers Subscribers

TempSensor

TempSensor

SELECT * FROM TempSensor t WHERE s.temp > 20

tID temp humidity

1 21 62

TempSensor

s.tID == 1

tID temp humidity

1 21 62

tID temp humidity

1

2

3

21 62

22 75

21 71


Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Processing Content & Structure

P

S1

S2

S3

{iID = 1}

{temp > 21 | humidity > 75}

{ temp <16 }

DDS provides some of the most useful features found in Complex Event Processing platforms!

Complex Event Processing...

tID temp humidity

3 21.5 72.3

tID temp humidity

2 20 78tID temp humidity

1 21 62

‣ Local Queries

‣ Continuous Queries (Content-based Subscriptions)

‣ Topics Joins & Projections

‣ Events windows (via History QoS)

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Processing Content & Structure

P

S1

S2

S3

{iID = 1}

{temp > 21 | humidity > 75}

{ temp <16 }

Complex Event Processing...

tID temp humidity

1 21 62

tID temp humidity

2

3

20 78

21.5 72.3

DDS provides some of the most useful features found in Complex Event Processing platforms!

‣ Local Queries

‣ Continuous Queries (Content-based Subscriptions)

‣ Topics Joins & Projections

‣ Events windows (via History QoS)


OpenSplice DDS

Object-OrientedPub/Sub



Subscription

Minimum Profile




Application

UDP/IP


Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.



‣ Automatically bridges the Object/Relational Impedance Mismatch

‣ Arbitrary object reconstructions

‣ Automatic Relationships Management

‣ Inheritance

‣ Local Operations

‣ Local/Distributed State

getTemp()getHumidity()

temp: floathumidity: float

TempSensor

set_climate();

width: floatheight: floatlength: floatclimate: int

Room

set_status()set_climate()get_level()

level: intopen: intclimate: int

Floor

setTemp()setHumidity()setFanLevel()start()stop()pause():


Conditioner * 1

*

1

*1


TTempSernsor

rID: intwidth: floatheight: floatlength: floatfID: int

TRoom


TFloor

cID: inttemp: floathumidity: floatfan_level: intstatus: intrID: int

TConditioner

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Object/Relational MappingUnleashing the power of Objects...

Relational

Object Oriented

call_sign: stringroute: listorigin: stringdest: stringaircraft: stringradar_id: string

FlightTopic

x: floaty: floatz: floatradar_id: string

RadarTrackTopicimage: stringrefresh_rate: intvisible: booluid: GUID

FlightViewTopiccall_sign: stringuid: GUIDhandle: Handle

FlightViewAssociation

OO ➡ Relational

‣ Middleware can automatically manage the generation and association between the Object-Oriented Model and the Relational Model

Relational ➡ OO

‣ The Relational Model can be mapped to an Object Oriented model‣ The mapping is under control of the architect

operationOne()operationTwo()


RadarTrack

operationX()operationY()

call_sign: stringroute: listorigin: stringdest: stringaircraft: string

Flight

*1

display()

image: stringrefresh_rate: intvisible: booluid: GUID

FlightView

**

handle: HandleCanvas

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Multiple Object/Relational Mappings

DCPScall_sign: stringroute: listorigin: stringdest: stringaircraft: stringradar_id: string

FlightTopic


RadarTrackTopicimage: stringrefresh_rate: intvisible: booluid: GUID

FlightViewTopiccall_sign: stringuid: GUIDhandle: Handle

FlightViewAssociation

DL

RL

display()

image: stringrefresh_rate: intvisible: booluid: GUIDx: floaty: floatz: floatcall_sign: string

FastFlightView

DLRL

Multiple Mappings‣ Different Object/Relatioal Mappings (ORM) can be used for different subsystems‣ The state is shared across all the local reconstruction (ORMs)



RadarTrack



Flight

*1

display()


FlightView

**

handle: HandleCanvas

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


An Example


FlightViewTopic


FlightTopic



RadarTrack


RadarTrackTopic

display()


FastFlightView

Primary RadarHigh-End 3D Visualization

Flight Data Processor

RadarTrack



Flight

RadarTrackList


RadarTrackTopic


RadarTrackTopic


RadarTrackTopicx: floaty: floatz: floatradar_id: string


RadarTrackTopic

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


An Example


FlightViewTopic


FlightTopic



RadarTrack

display()


FastFlightView



RadarTrack



Flight

RadarTrackList


RadarTrackTopic


RadarTrackTopic


RadarTrackTopic


RadarTrackTopic


RadarTrackTopic




RadarTrackTopic

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


An Example


FlightViewTopic


FlightTopic



RadarTrack


RadarTrackTopic


RadarTrackTopic

display()


FastFlightView




RadarTrackTopic



RadarTrack

Secondary Radar


RadarTrackTopic

RadarTrack



Flight

RadarTrackList

RadarTrack


RadarTrackTopic

Automatically Manage Relationships

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Custom Filter Example


FlightViewTopic


FlightTopic



RadarTrack


RadarTrackTopic

display()


FastFlightView





RadarTrack

Secondary Radar


RadarTrackTopic

RadarTrack



Flight

RadarTrackList

RadarTrack

SQL FilterCustom Filter

SQL FilterCustom Filter

Filter based on Region-of-interest

Automatically Manage Relationships


RadarTrackTopic


RadarTrackTopic


RadarTrackTopic


RadarTrackTopic




RadarTrackTopic


OpenSplice DDS

...Putting it all Together

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


DCPS Application

‣ The application works directly at the DCPS level

‣When using OO Programming Languages the Object/Relational Impedance Mismatch has to be manually dealt


Subscription

Minimum Profile


DD

S v

1.2


TTempSensor


TRoom


TFloor


TConditioner

Application

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


DLRL Application

‣ The DLRL Layer is used to provide a Language Integrated access to DDS data

‣ The Designer has great freedom in deciding how Objects have to map to Topics

‣ Different Object Reconstruction can be created for different applications Ownership Durability

Content Subscription

Minimum Profile


DD

S v

1.2



TempSensor

set_climate();set_temp()set_humidity()


Room



Floor


temp: floathumidity: floatfan_level: intstatus: int

Conditioner *1

*

1

*

1


TTempSensor


TRoom


TFloor


TConditioner


Data Local Reconstruction Layer (DLRL) DD

S v

1.2

Application

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Generic DDS Application

‣ In the most general case, different portion of the application might rely on DLRL or DCPS depending on their specific needs

‣ DCPS access might be required for accessing and tuning some specific QoS




Subscription

Minimum Profile


DD

S v

1.2



TempSensor

set_climate();set_temp()set_humidity()


Room



Floor


temp: floathumidity: floatfan_level: intstatus: int

Conditioner *1

*

1

*

1


TTempSensor


TRoom


TFloor


TConditioner

Application

Application


OpenSplice DDS

QoS-EnabledPub/Sub

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Communicating with Topics, Partitions and Domains

1 21 62

2 20 61

3 25 70 25

1 22 62 1 23 63

2 19 60

3 25 71 3 25 74 3 26 77

Samples

Topic

Instances1 21 62

2 20 61

3 25 70 25

1 22 62 1 23 63

2 19 60

3 25 71 3 25 74 3 26 77

Samples

Topic

Instances1 21 62

2 20 61

3 25 70 25

1 22 62 1 23 63

2 19 60

3 25 71 3 25 74 3 26 77

Samples

Topic

InstancesDataReaderDataReaderDataReaderDataReaderDataReaderDataReaderDataWriterDataWriterDataWriterDataWriter


Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.



1 21 62

2 20 61

3 25 70 25

1 22 62 1 23 63

2 19 60

3 25 71 3 25 74 3 26 77

Samples

Topic

Instances1 21 62

2 20 61

3 25 70 25

1 22 62 1 23 63

2 19 60

3 25 71 3 25 74 3 26 77

Samples

Topic

Instances1 21 62

2 20 61

3 25 70 25

1 22 62 1 23 63

2 19 60

3 25 71 3 25 74 3 26 77

Samples

Topic


Partition


PublisherSubscriber

Arrows show structural relationships, not data-flows

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.



1 21 62

2 20 61

3 25 70 25

1 22 62 1 23 63

2 19 60

3 25 71 3 25 74 3 26 77

Samples

Topic

Instances1 21 62

2 20 61

3 25 70 25

1 22 62 1 23 63

2 19 60

3 25 71 3 25 74 3 26 77

Samples

Topic

Instances1 21 62

2 20 61

3 25 70 25

1 22 62 1 23 63

2 19 60

3 25 71 3 25 74 3 26 77

Samples

Topic


PublisherSubscriberPartition

Domain

Domain Participant


Arrows show structural relationships, not data-flows

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


QoS Model

‣ QoS-Policies are used to control relevant properties of OpenSplice DDS entities, such as:‣ Temporal Properties

‣ Priority

‣ Durability

‣ Availability

‣ ...

‣ Some QoS-Policies are matched based on a Request vs. Offered Model thus QoS-enforcement

‣ Publications and Subscriptions match only if the declared vs. requested QoS are compatible ‣ e.g., it is not possible to match a publisher which delivers data unreliably with a subscriber which requires reliability

Publisher

DataWriter

Topic

Type

QoS

Name

writes

QoS

DataWriter

Topic

Typewrites

Subscriber

DataReaderreads

DataReaderreads

...

QoS

Name

QoS

QoS QoS

QoS matching

......

QoS QoS

Type Matching

DomainParticipant DomainParticipant

QoS QoS

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Sample QoS Policies

‣ Rich set of QoS allow to configure several different aspects of data availability, delivery and timeliness

‣ QoS can be used to control and optimize network as well as computing resource

QoS Policy Applicability RxO Modifiable

DURABILITY

DURABILITY SERVICE

LIFESPAN

HISTORY

PRESENTATION

RELIABILITY

PARTITION

DESTINATION ORDER

OWNERSHIP

OWNERSHIP STRENGTH

DEADLINE

LATENCY BUDGET

TRANSPORT PRIORITY

TIME BASED FILTER

RESOURCE LIMITS

USER_DATA

TOPIC_DATA

GROUP_DATA

T, DR, DW Y N Data AvailabilityT, DW N N

Data Availability

T, DW - Y

Data Availability

T, DR, DW N N

Data Availability

P, S Y N Data DeliveryT, DR, DW Y N

Data Delivery

P, S N Y

Data Delivery

T, DR, DW Y N

Data Delivery

T, DR, DW Y N

Data Delivery

DW - Y

Data Delivery

T, DR, DW Y Y Data TimelinessT, DR, DW Y Y

Data Timeliness

T, DW - Y

Data Timeliness

DR - Y Resources

T, DR, DW N N

Resources

DP, DR, DW N Y ConfigurationT N Y

Configuration

P, S N Y

Configuration

Publisher

DataWriter

Topic

Type

QoS

Name

writes

QoS

DataWriter

Topic

Typewrites

Subscriber

DataReaderreads

DataReaderreads

...

QoS

Name

QoS

QoS QoS

QoS matching

......

QoS QoS

Type Matching

DomainParticipant DomainParticipant

QoS QoS

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Mapping QoS

Data Latency

LatencyBudget TransportPriority

Throughput

DeadlineTimeBasedFilter

Control over Latency/Throughput tradeoff Control over data latency Control over data priority

Data Availability

OwnershipOwnershipStrength

Durability

History

Lifespan

Control over data queueing Control over data persistency Control over data sources hot-swap

Data Delivery

Reliability

DestinationOrder

Presentation

Control over data distribution reliability Control over data ordering Control over presentation

OpenSplice DDS provides programmatic QoS-driven support for configuring the most important properties of data distribution!

Which properties does QoS controls?


Data Timeliness

Data Latency

LatencyBudget TransportPriority

Throughput

DeadlineTimeBasedFilter


DEADLINE

LATENCY

BUDGET

TRANSPORT

PRIORITY

T, DR, DW Y Y

Data

Timeliness

T, DR, DW Y Y

T, DW - Y

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


DeadlineThe DEADLINE QoS policy allows to define the maximum inter-arrival time between data samples


DEADLINE T, DR, DW Y Y

Publisher

DataWriter

Topic

Type

QoS

Name

writes

QoS

QoS

DataWriter

Topic

Typewrites

Subscriber

DataReaderreads

QoS

DataReaderreads

...

QoS

Name

QoS

QoS QoS

QoS matching

QoS matching

......

You can’t be later than...

‣ DataWriter indicates that the application commits to write a new value at least once every deadline period

‣ DataReaders are notified by the DDS when the DEADLINE QoS contract is violated

Publisher Subscriber

DeadlineDeadlineDeadlineDeadlineDeadline

Deadline Violation

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Latency BudgetThe LATENCY_BUDGET QoS policy specifies the maximum acceptable delay from the time the data is written until the data is inserted in the receiver's application-cache


LATENCY

BUDGET

T, DR, DW Y Y

‣ The default value of the duration is zero indicating that the delay should be minimized

‣ This policy is a hint to the DDS, not something that must be monitored or enforced.

I need to get there in at most...

Publisher

DataWriter

Topic

Type

QoS

Name

writes

QoS

QoS

DataWriter

Topic

Typewrites

Subscriber

DataReaderreads

QoS

DataReaderreads

...

QoS

Name

QoS

QoS QoS

QoS matching

QoS matching

......

T1

T2

T3

Latency Budget = Latency = TBuff +T1+T2+T3TBuff

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Transport Priority

The TRANSPORT_PRIORITY QoS policy is a hint to the infrastructure as to how to set the priority of the underlying transport used to send the data.


TRANSPORT

PRIORITY

T, DW - Y

Publisher

DataWriter

Topic

Type

QoS

Name

writes

QoS

QoS

DataWriter

Topic

Typewrites

Subscriber

DataReaderreads

QoS

DataReaderreads

...

QoS

Name

QoS

QoS QoS

QoS matching

QoS matching

......

VIP Data, stay clear!


Data Availability

Data Availability

OwnershipOwnershipStrength

Durability

History

Lifespan


DURABILITY

DURABILITY

SERVICE

LIFESPAN

HISTORY

T, DR, DW Y N

Data

Availability

T, DW N N

T, DW - Y

T, DR, DW N N

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Durability

‣ Volatile. No need to keep data instances for late joining data readers

‣ Transient Local. Data instance availability for late joining data reader is tied to the data writer availability

‣ Transient. Data instance availability outlives the data writer

‣ Persistent. Data instance availability outlives system restarts

Publisher

DataWriter

Topic

Type

QoS

Name

writes

QoS

QoS

DataWriter

Topic

Typewrites

Subscriber

DataReaderreads

QoS

DataReaderreads

...

QoS

Name

QoS

QoS QoS

QoS matching

QoS matching

......

The DURABILITY QoS controls the data availability w.r.t. late joiners, specifically the DDS provides the following variants:

The DURABILITY_SERVICE QoS provide control over configuration of the service that implements the transient and persistent durability features


DURABILITY

DURABILITY

SERVICE

T, DR, DW Y N

T, DW N N

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Lifespan

‣ The LIFESPAN QoS policy allows to control what happens to stale data

‣ It specifies the validity interval for data written by the DataWriter

‣ The default validity interval is infinite


LIFESPAN T, DW - Y

Publisher

DataWriter

Topic

Type

QoS

Name

writes

QoS

QoS

DataWriter

Topic

Typewrites

Subscriber

DataReaderreads

QoS

DataReaderreads

...

QoS

Name

QoS

QoS QoS

QoS matching

QoS matching

......

For how long will this data be available?

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


HistoryThe HISTORY QoS policy controls whether the DDS should deliver only the most recent value, attempt to deliver all intermediate values, or do something in between. The policy can be configured to provide the following semantics:


HISTORY T, DR, DW N N

Publisher

DataWriter

Topic

Type

QoS

Name

writes

QoS

QoS

DataWriter

Topic

Typewrites

Subscriber

DataReaderreads

QoS

DataReaderreads

...

QoS

Name

QoS

QoS QoS

QoS matching

QoS matching

......

‣ Keep Last. The DDS will only attempt to keep the most recent “depth” samples of each instance of data identified by its key

‣ Keep All. The DDS will attempt to keep all the samples of each instance of data identified by its key.

‣ On the DataWriter Samples are kept until delivered to all known subscribers

‣ On the DataReader side samples are kept until the application “takes” them

How many data samples should I keep?


Data DeliveryData Delivery

Reliability

DestinationOrder

Presentation


PRESENTATION

RELIABILITY

PARTITION

DESTINATION

ORDER

OWNERSHIP

OWNERSHIP

STRENGTH

P, S Y N

Data Delivery

T, DR, DW Y N

P, S N Y

T, DR, DW Y N

T, DR, DW Y N

DW - Y

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Reliability

The RELIABILITY QoS indicate the level of guarantee offered by the DDS in delivering data to subscribers. Possible variants are:


RELIABILITY T, DR, DW Y N

Publisher

DataWriter

Topic

Type

QoS

Name

writes

QoS

QoS

DataWriter

Topic

Typewrites

Subscriber

DataReaderreads

QoS

DataReaderreads

...

QoS

Name

QoS

QoS QoS

QoS matching

QoS matching

......

‣ Reliable. In steady-state the middleware guarantees that all samples in the DataWriter history will eventually be delivered to all the DataReader

‣ Best Effort. Indicates that it is acceptable to not retry propagation of any samples

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Ownership The OWNERSHIP QoS specifies whether it is allowed for multiple DataWriters to write the same instance of the data and if so, how these modifications should be arbitrated. Possible choices are:

Publisher

DataWriter

Topic

Type

QoS

Name

writes

QoS

QoS

DataWriter

Topic

Typewrites

Subscriber

DataReaderreads

QoS

DataReaderreads

...

QoS

Name

QoS

QoS QoS

QoS matching

QoS matching

......


OWNERSHIP T, DR, DW Y N

‣ Shared. Multiple writers are allowed to update the same instance and all the updates are made available to the reader

‣ Exclusive. Indicates that each instance can only be owned by one DataWriter, but the owner of an instance can change dynamically -- due to liveliness changes

‣ The selection of the owner is controlled by the setting of the OWNERSHIP_STRENGTH QoS policy

Who owns the data?

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Ownership StrengthThe OWNERSHIP_STRENGTH Specifies the value of the “strength” used to arbitrate among DataWriters that attempt to modify the same data instance

Publisher

DataWriter

Topic

Type

QoS

Name

writes

QoS

QoS

DataWriter

Topic

Typewrites

Subscriber

DataReaderreads

QoS

DataReaderreads

...

QoS

Name

QoS

QoS QoS

QoS matching

QoS matching

......QoS Policy Applicability RxO Modifiable

OWNERSHIP

STRENGTH

DW - Y

‣ Data instance are identified by the couple (Topic, Key)

‣ The policy applies only if the OWNERSHIP is EXCLUSIVE

How strong are you?

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Ownership @ Work‣ A Topic can have Shared or Exclusive Ownership

‣ Exclusively owned Topics can be modified by a single writer

‣ Writer strength is used to coordinate replicated writers

R1

R2

R3

symbol: "MSFT"name: "Microsoft Corp."exchange: "NASD"quote: 33.73

StockQuote

symbol: "GOOG"name: "Google Inc."exchange: "NASD"quote: 663.97

StockQuote

W1

W2’’

STRENGTH=3

STRENGTH=1


StockQuote


StockQuotesymbol: "AAPL"name: "Apple Inc."exchange: "NASD"quote: 165.37

StockQuote

W1’

STRENGTH=2


StockQuote



StockQuote


StockQuote


StockQuote

symbol: "AAPL"name: "Apple Inc."exchange: "NASD"quote: 165.37


StockQuote


StockQuote

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Managing Hot-Swap‣ A Topic can have Shared or Exclusive Ownership

‣ Exclusively owned Topics can be modified by a single writer

‣ Writer strength is used to coordinate replicated writers


StockQuote


StockQuote


StockQuote


StockQuote


StockQuote


StockQuote


StockQuote


StockQuote


StockQuote

R1

R2

R3W2’’

STRENGTH=1

Age

nda

Motivating Forces

The Standard


Programming Model

Concluding Remarks


OpenSplice DDS

Shared Global Data Space

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Shared Global Data SpaceCoordination Model

‣ DDS applications are asynchronous and communicate by reading/writing from/to a Global Data Space

‣ DDS applications communicate by simply addressing items in the Global Data Space and without any direct knowledge of the parties involved in the production/consumption of data

Consistency Model

‣ The Shared Global Data Space implemented by DDS, can be configured to supports at most the “Eventual Consistency Model”

‣ Under an Eventual Consistency Model we are guaranteed that eventually all application in the system will have a consistent view of the “world”s

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Eventual Consistency & R/W Caches

Under an Eventual Consistency Model, DDS guarantees that all matched Reader Caches will eventually be identical of the respective Writer Cache

Topic

DataReader

1 1

2 1

3 1

DataReader Cache Topic

1 2

2 3

3 1

DataWriter Cache

DataWriter

1 2

2 2 2 3

Topic

DataReader

1 1

2 1

3 1

DataReader Cache

Topic

DataReader

1 1

2 1

3 1

DataReader Cache

DDS

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


QoS Impacting the Consistency Model

The DDS Consistency Model is a property that can be associated to Topics or further refined by Reader/Writers. The property is controlled by the following QoS Policies:

‣ DURABILITY ‣ VOLATILE | TRANSIENT_LOCAL | TRANSIENT | PERSISTENT

‣ LIFESPAN

‣ RELIABILITY‣ RELIABLE | BEST_EFFORT

‣ DESTINATION ORDER‣ SOURCE_TIMESTAMP | DESTINATION_TIMESTAMP


DURABILITY

LIFESPAN

RELIABILITY

DESTINATION ORDER

T, DR, DW Y N

T, DW - Y

T, DR, DW Y N

T, DR, DW Y N

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


QoS Impacting the Consistency Model

DURABILITY RELIABILITY DESTINATION_ORDER LIFESPAN

Eventual Consistency (No Crash / Recovery)

Eventual Consistency (Reader Crash / Recovery)

Eventual Consistency(Crash/Recovery)

Eventual Consistency(Crash/Recovery)

Weak Consistency

Weak ConsistencyWeak Consistency

VOLATILE RELIABLE SOURCE_TIMESTAMP INF.

TRANSIENT_LOCAL RELIABLE SOURCE_TIMESTAMP INF.

TRANSIENT RELIABLE SOURCE_TIMESTAMP INF.

PERSISTENT RELIABLE SOURCE_TIMESTAMP INF.

ANY ANY DESTINATION_TIMESTAMP ANY

ANY BEST_EFFORT ANY ANY

ANY ANY ANY N

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Eventual Consistency @ WorK

B A

B

CD

E

F

m

J

K

A

P1

P2

S1

S2

B

A

B

A

P = {A, B}

S = {A, D}

S= {A, B, J}

P = {D, C, J}

B

S4

S = {A}

A

Eventual Consistency (Reader Crash / Recovery)Eventual Consistency(Crash/Recovery)




{A}

{B}

Weak Consistency ANY ANY ANY N {J}

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Eventual Consistency @ Work

A

B

CD

E

F

m

J

KP2

S1

S2

S3

A

BA

B

S= {A, B, D, J}

J

B

D

J

S = {A, D}

S= {A, B, J}

P = {D, C, J}

S4

S = {A}

A

Eventual Consistency (Reader Crash / Recovery)Eventual Consistency(Crash/Recovery)




{A}

{B}

Weak Consistency ANY ANY ANY N {J}

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Design Guidelines

‣ For all (non-periodic) Topics for which an eventually consistent model is required use the following QoS settings:

‣ For information produced periodically, with a period P, where P is small enough to be acceptable as a consistency convergence delay, the following QoS settings will provide an approximation of the eventual consistency:

Eventual Consistency (Crash / Recovery)



Eventual Consistency (Crash / Recovery)

VOLATILE BEST_EFFORT SOURCE_TIMESTAMP INF.



OpenSplice DDS

Getting the DataRead Semanticsstruct Counter {

int cID; int count;};#pragma keylist Counter cID

Topic Used in next section.

DataReader Cache

Topic

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Reading Samples

‣ Read iterates over the available sample instances

‣ Samples are not removed from the local cache as result of a read

‣ Read samples can be read again, by accessing the cache with the proper options (more later)

DataReader

1 1

2 1

3 1

2 2

Samples Read Samples not Read

1 2 1 3 1 4

2 3

3 2 3 3 3 4 3 5

DataReader Cache

Topic

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Reading Samples

‣ Read iterates over the available sample instances

‣ Samples are not removed from the local cache as result of a read

‣ Read samples can be read again, by accessing the cache with the proper options (more later)

Samples Read Samples not Read

DataReader

1 1

2 1

3 1

2 2

1 2 1 3 1 4

2 3

3 2 3 3 3 4 3 5

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Sample, Instance and View States

‣ DataReaders are provided with data samples ane state information allowing to detect relevant transitions in the life-cycle of data as well as data writers

‣ Sample State (READ | NOT_READ): Determines wether a sample has already been read by this DataWriter or not.

‣ Instance State (ALIVE, NOT_ALIVE, DISPOSED). Determines wether (1) writer exist for the specific instance, or (2) no matched writers are currently available, or (3) the instance has been disposed

‣ View State (NEW, NOT_NEW). Determines wether this is the first sample of a new (or re-born) instance

Dat

aRea

der

History Depth = 2

DataReader Cache

Samples

1 1

2 2

3 1

1 2

2 3

SampleInfo

Topic

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Anatomy of a Read

DDS::ReturnCode_t read( TSeq& samples, DDS::SampleInfoSeq& info, long max_samples, DDS::SampleStateMask samples_state, DDS::ViewStateMask views_state, DDS::InstanceStateMask instances_state)

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Reading Data

TSeq samples;SampleInfoSeq info;

reader->read(samples, info, DDS::LENGTH_UNLIMITED, DDS::NOT_READ_SAMPLE_STATE, DDS::ANY_VIEW_STATE, DDS::ANY_INSTANCE_STATE);

// Do something

reader->return_loan(samples, info);

TSeq samples(N);SampleInfoSeq info(N);

reader->read(samples, info, N, DDS::NOT_READ_SAMPLE_STATE, DDS::ANY_VIEW_STATE, DDS::ANY_INSTANCE_STATE);

// Do something

Example 1 Example 2

Example 3

reader->read(samples, info, N, DDS::NOT_READ_SAMPLE_STATE, DDS::NEW_VIEW_STATE, DDS::ALIVE_INSTANCE_STATE);


OpenSplice DDS

Getting the DataTake Semanticsstruct Counter {

int cID; int count;};#pragma keylist Counter cID


DataReader Cache

Topic

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Taking Samples

‣ Take iterates over the available sample instances

‣ Taken Samples are removed from the local cache as result of a take

‣DataReader

1 1

2 1

3 1

2 2

Samples not Taken

1 2 1 3 1 4

2 3

3 2 3 3 3 4 3 5

DataReader Cache

Topic

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Taking Samples



Samples not Taken

DataReader 2 2

1 2 1 3 1 4

2 3

3 2 3 3 3 4 3 5

DataReader Cache

Topic

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Taking Samples



Samples not Taken

DataReader

1 3 1 4

2 3

3 3 3 4 3 5

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Anatomy of a take

DDS::ReturnCode_t take( TSeq& samples, DDS::SampleInfoSeq& info, long max_samples, DDS::SampleStateMask samples_state, DDS::ViewStateMask views_state, DDS::InstanceStateMask instances_state)


OpenSplice DDS

Writing Datastruct Counter { int cID; int count;};#pragma keylist Counter cID


Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Writing Samples

‣ Samples are written in the local cache

‣ Writer control the creation of instances

‣ The DDS ensures that the local caches for the matched DataReader will be eventually consistent with that of the Data Writer DataWriter Cache

Topic

1 1

Last Sample Written

DataWriter

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Writing Samples



‣ The DDS ensures that the local caches for the matched DataReader will be eventually consistent with that of the Data Writer DataWriter Cache

Topic

1 1

Last Sample Written

DataWriter1 2

DataWriter Cache

Topic

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Writing Samples



‣ The DDS ensures that the local caches for the matched DataReader will be eventually consistent with that of the Data Writer

1 2

Last Sample Written

DataWriter1 3

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Anatomy of a write

DDS::ReturnCode_t write(const T& sample, InstanceHandle_t handle);

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Writing Data

writer->write(sample, DDS::HANDLE_NIL);

InstanceHandle_t handle = writer->register_instance(sample);// ...writer->write(sample, handle);


OpenSplice DDS

Managing Data History

struct Counter { int cID; int count;};#pragma keylist Counter cID


Topic

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


How many samples?

‣ The History QoS Controls the number of samples-per-instance that will be stored by the middleware on behalf of a Reader

‣ Keep Last K. The History QoS can be set so to always have the latest K samples

‣ Keep All. The History QoS can be set so keep all samples produced by the writer and not yet taken, until resource limits are not reached

DataReader

DataReader

1 1

2 1

3 1

History Depth = 1 (DDS Default)

History Depth = 5

Topic

1 1

2 1

3 1

1 2 1 3 1 4 1 5

2 2 2 3 2 4 2 5

3 2 3 3 3 4 3 5

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


History in Action

Note: The Reliability QoS controls wether data is sent reliably, or best-effort, from the DataWriter to matched DataReaders

Topic

DataReader

1 1

2 1

3 1


DataReader Cache

Topic

1 2

2 3

3 1


DataWriter Cache

DataWriter1 2

2 2 2 3

Network

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


History in Action

Topic

DataReader

1 2

2 2

3 1


DataReader Cache

Topic

1 2

2 3

3 1


DataWriter Cache

DataWriter2 3

Network


Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


History in Action

Topic

DataReader

1 2

2 3

3 1


DataReader Cache

Topic

1 2

2 3

3 1


DataWriter Cache

DataWriter

Network


Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


History in Action

Topic

DataReader

1 1

2 1

3 1

History Depth = 2

DataReader Cache

Topic

1 2

2 3

3 1


DataWriter Cache

DataWriter1 2

2 2 2 3

Network


Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


History in Action

Topic

DataReader

1 1

2 1

3 1

History Depth = 2

DataReader Cache

Topic

1 2

2 3

3 1


DataWriter Cache

DataWriter

1 2

2 2

2 3

Network


Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


History in Action

Topic

DataReader

1 1

2 2

3 1

History Depth = 2

DataReader Cache

Topic

1 2

2 3

3 1


DataWriter Cache

DataWriter

1 2

2 3

Network



OpenSplice DDS

Application / DDS Coordination

struct Counter { int cID; int count;};#pragma keylist Counter cID


Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Application / DDS Coordination

DDS provides three main mechanism for exchanging information with the application

‣ Polling. The application polls from time to time for new data or status changes. The interval might depend on the kind of applications as well as data

‣WaitSets. The application registers a WaitSet with DDS and waits (i.e. is suspended) until one of the specified events has happened.

‣ Listeners. The application registers a listener with a specific DDS entity to be notified when relevant events occur, such as state changes or


OpenSplice DDS

Putting it All TogetherWriting a DDS Application

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Steps for Writing a DDS ApplicationWriting a DDS application can be decomposed in the following few simple steps:

‣ Step#1: Define Topics

‣ Step #2: Identify QoS representing key non-functional invariants for your system‣ Transport Priority

‣ Deadline

‣ Durability

‣ Step #3: Define Topics / Partition Mapping

‣ Step #4: Identify Topic Readers/Writers

‣ Step #5: Define QoS requirements for Readers/Writers‣ History

‣ Latency Budget

‣ Auto-Dispose

‣ Transport Priority

‣ Deadline

‣ Step #6: Code-it in your favorite programming language

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Making the Hello DDS World

Step #1: Topic definition

‣ We are going to define a simple key-less topic that will carry the name to greet.

module swatch {

struct hello { string<256> name; };#pragma keylist hello

};

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.



Step #2: Topic QoS

‣ Reliability QoS: RELIABLE

‣ Durability QoS: TRANSIENT

module swatch {

struct hello { string<256> name; };#pragma keylist hello

};

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.



Step #3: Topics/Partitions Mapping

‣ swatch::hello will be mapped into the default-partition (thus no action to take)

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.



Step #4: Readers/Writers

‣ A Generic DataReader that will read the topic swatch:: hello ‣We’ll be able to run as many of this as we want

‣ A Generic DataWriter that will read the topic swatch:: hello‣We’ll be able to run as many of this as we want

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.



‣ Step #5: Define QoS requirements for Readers/Writers

‣ Writer:‣ Inherit TopicQoS, and

‣ No-Auto Dispose

‣ History QoS: Keep Last N

‣ Reader‣ Inherit TopicQoS, and

‣ History QoS: Keep Last N

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Hello World in a Conceptual Picture

Network

Dat

aRea

der

History Depth = 2

DataReader Cache

Samples

1 1

2 2

3 1

1 2

2 3

SampleInfo

1

2

3

1

2

swatch::hello

History Depth = 2

DataWriter Cache

Samples

1 1

2 2

3 1

1 2

2 3

SampleInfo

1

2

3

1

2

swatch::hello

Dat

aWri

ter

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Visualizing the Strucutre...

PublisherSubscriber Default Partition

Domain

Domain Participant

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Step #6: Coding

How many lines of code is going to take this example?

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


SIMple Dds == SIMD!

‣ Today we’ll write our DDS application using SIMD

‣ SIMD is a C++ library that takes advantage of C++ Template Meta-Programming to:‣ Vastly Improve Productivity

‣ Simplify Usage

‣ Automate Resource Management (All DDS Entities are Garbage Collected via Ref-Counting)

‣ Zero Overhead

SIMD

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Hello-pub.cpp (Default QoS)

int main(int argc, char* argv[]) { if (!parse_args(argc, argv)) return 1;

// -- init the SIMD runtime simd::Runtime::init(); // -- create the DDS Topic simd::Topic<swatch::hello> helloTopic("helloTopic"); // -- create the DDS DataWriter simd::DataWriter<swatch::hello> writer(helloTopic, dwqos);

swatch::hello sample; std::stringstream ss; for (int i = 0; i < N; ++i) { ss << i; std::string tmp = ss.str() + "." + message; ss.str(""); sample.name = DDS::string_dup(tmp.c_str()); std::cout << "<<= " << sample.name << std::endl; writer.write(sample); usleep(period*1000); } std::cout << "[done]" << std::endl; return 0; }

Only 3-lines of DDS-Specific Code Business Logic

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Hello-pub.cppint main(int argc, char* argv[]) { if (!parse_args(argc, argv)) return 1; // -- init the SIMD runtime simd::Runtime::init(); simd::TopicQos tqos; tqos.set_reliable(); tqos.set_transient();

// -- create the DDS Topic simd::Topic<swatch::hello> helloTopic("helloTopic", tqos); simd::DataWriterQos dwqos(tqos); dwqos.set_keep_last(history_depth); dwqos.set_auto_dispose(false); // -- create the DDS DataWriter simd::DataWriter<swatch::hello> writer(helloTopic, dwqos);

swatch::hello sample; std::stringstream ss; for (int i = 0; i < N; ++i) { ss << i; std::string tmp = ss.str() + "." + message; ss.str(""); sample.name = DDS::string_dup(tmp.c_str()); std::cout << "<<= " << sample.name << std::endl; writer.write(sample); usleep(period*1000); } std::cout << "[done]" << std::endl; return 0; }

Business Logic

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Hello-sub.cpp (Default QoS)

int main(int argc, char* argv[]) { if (!parse_args(argc, argv)) return 1; // -- init the SIMD runtime simd::Runtime::init(); // -- create the DDS Topic simd::Topic<swatch::hello> helloTopic("helloTopic");

// -- create the DDS DataReader simd::DataReader<swatch::hello> reader(helloTopic);

swatch::helloSeq samples; DDS::SampleInfoSeq infos; while (true) { reader.read(samples, infos); for (int i = 0; i < samples.length(); ++i) { std::cout << "=>> " << samples[i].name << std::endl; } if (samples.length() > 0) std::cout << "--" << std::endl; reader.return_loan(samples, infos); usleep(period*1000); } return 0;}Only 3-lines of DDS-Specific Code

Business Logic

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Hello-sub.cppint main(int argc, char* argv[]) { if (!parse_args(argc, argv)) return 1; // -- init the SIMD runtime simd::Runtime::init(); simd::TopicQos tqos; tqos.set_reliable(); tqos.set_transient();

// -- create the DDS Topic simd::Topic<swatch::hello> helloTopic("helloTopic", tqos); simd::DataReaderQos drqos(tqos); drqos.set_keep_last(history_depth);

// -- create the DDS DataReader simd::DataReader<swatch::hello> reader(helloTopic, drqos);

swatch::helloSeq samples; DDS::SampleInfoSeq infos; while (true) { reader.read(samples, infos); for (int i = 0; i < samples.length(); ++i) { std::cout << "=>> " << samples[i].name << std::endl; } if (samples.length() > 0) std::cout << "--" << std::endl; reader.return_loan(samples, infos); usleep(period*1000); } return 0;}

Business Logic


OpenSplice DDS

Demo!

Age

nda

Motivating Forces

The Standard


Programming Model

Concluding Remarks

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Concluding RemarksState of the Art

‣ The OMG DDS Standard uniquely addresses some of the key requirements for next generation Mission Critical Systems

‣ The OMG DDS is one of the few middleware standard defining both API as well as Wire-Protocol, thus ensuring application portability and interworking

‣ The DDS Standard has been recommended/mandated by key administration throughout the world as well as adopted by the most challenging Defense programs

Looking Ahead

‣ The DDS Standard is swiftly moving forward to expand its applicability, simplify its use, and facilitate integration



Subscription

Minimum Profile




Application

UDP/IP


Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.

Online Resources

http://www.opensplice.com/

emailto:[email protected]

http://www.youtube.com/OpenSpliceTube

http://opensplice.blogspot.com http://bit.ly/1Sreg

http://www.dds-forum.org

http://portals.omg.org/dds


http://twitter.com/acorsaro/







http://opensplice.blogspot.com

http://opensplice.blogspot.com

http://bit.ly/1Sreg

http://bit.ly/1Sreg

http://www.prismtech.com/opensplice-dds

http://www.prismtech.com/opensplice-dds





Bac

kup

Data Access Pattern

Coordination Patterns

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Topic Queues

Context

‣ One commonly used technique for implementing distributed real-time embedded systems is to model applications as FSA, or DFSA (Distributed Finite State Automata)

Problem

‣ One or more DDS applications are implemented as a (D)FSA whose transitions depends on the totally ordered history of updates for a specific topic

‣ How can we ensure that each application sees exactly the same set of updates in exactly the same order?

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Topic Queues

Assumptions

‣ Single writer exists per Topic Instance

Solution

‣ Represent the state transition events by means of DDS Topics

‣ Topic Instances can be used to identify specific FSA

‣ Ensure that application always use the Take Semantics for getting data.

‣ Ensure that these topics have the following QoS Settings‣ DURABILITY: TRANSIENT | PERSISTENT

‣HISTORY: KEEP_ALL

‣RELIABILITY: RELIABLE

‣DESTINATION_ORDER: SOURCE_TIMESTAMP

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Topic Queues

DDS

1 1

2 1

3 1

Samples not Taken

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Topic Queues

DDS

1 1

2 1

3 1

1 2

3 2

Samples not Taken

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Topic Queues

DDS

1 1

2 1

3 1

2 2

1 2 1 3 1 4

2 3

3 2 3 3 3 4 3 5

Samples not Taken

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Topic Queues

DDS

2 2

1 2 1 3 1 4

2 3

3 2 3 3 3 4 3 5

Samples not Taken

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Topic K-Queues

Context

‣ One commonly used technique for implementing distributed real-time embedded systems is to model applications as FSA, or DFSA (Distributed Finite State Automata)

Problem

‣ One or more DDS applications are implemented as a (D)FSA whose transitions depends on the totally ordered history of updates for a specific topic

‣ How can we ensure that each application sees exactly the same set of updates in exactly the same order?

‣ How can we ensure that misbehaving applications consume an unbounded amount of memory?

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Topic K-QueuesAssumptions

‣ Single writer exists per Topic Instance

Solution

‣ Represent the state transition events by means of DDS Topics

‣ Topic Instances can be used to identify specific FSA

‣ Ensure that application always use the Take Semantics for getting data


‣ RELIABILITY: RELIABLE

‣ HISTORY: KEEP_LAST with DEPTH set to K

‣ DESTINATION_ORDER: SOURCE_TIMESTAMP

‣ If a FSA looses a state transition “resets the state” by some other means (e.g. DURABILITY SERVICE)

Note

‣ K can be dimensioned by considering the maximum burst of activity that should be tolerated along with the average time between state transitions

‣ e.g., if I want to tolerate 12 sec of overload and state transition occur every 4 sec then K=3

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Topic K-Queues

DDS

1 1

2 1

3 1

Samples not Taken

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Topic K-Queues

DDS

1 1

2 1

3 1

2 2

1 2 1 3

3 2

Samples not Taken

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Topic K-Queues

DDS

1 2

2 1

3 1

2 2

1 3 1 4

2 3

3 2 3 3

Samples not Taken

- One FSA has missed a sample... but does not know it yet.

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Topic K-Queues

DDS

2 2

1 3 1 4

2 3

3 2 3 3

Samples not Taken

- One FSA has missed a sample... but does not know it yet.- It detects this and gets the sample from the DURABILITY SERVICE

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Topic Caches

Problem

‣ Distributed applications often have to deal with “Hard State”, meaning state that is conceptually shared among various elements.

‣ This “Hard State” often needs to be accesses very efficiently, likewise changes in state should also consistently diffused in a timely manner

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Topic Caches

Solution

‣ Represent the “Hard State” by means of DDS Topics

‣ Application should favor the use the Read Semantics for getting data.


‣HISTORY: KEEP_LAST

‣RELIABILITY: RELIABLE

‣DESTINATION_ORDER: SOURCE_TIMESTAMP

Note

‣ Notice that “Hard State” will be eventually consistent for all reader regardless of the number of writers

‣ This technique can be exploited for writing self-stabilizing applications

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Topic Caches

DDS

1 1

2 1

3 1Dat

aRea

der

1 1

2 1

3 1Dat

aRea

der

1 1

2 1

3 1Dat

aRea

der

1 2

2 2 2 3

1 2

2 3

Dat

aWri

ter

2 2

Dat

aWri

ter

3 1

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Topic Caches

DDS

1 2

2 2

3 1Dat

aRea

der

1 2

2 2

3 1Dat

aRea

der

1 2

2 2

3 1Dat

aRea

der

2 3

1 2

2 3

Dat

aWri

ter

2 2

Dat

aWri

ter

3 1

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Topic Caches

DDS

1 2

2 3

3 1Dat

aRea

der

1 2

2 3

3 1Dat

aRea

der

1 2

2 3

3 1Dat

aRea

der

1 2

2 3

Dat

aWri

ter

2 2

Dat

aWri

ter

3 1

- Notice that one Data Writer has an inconsistent state when compared to the system state.- This is not hard to cope with, and could be simply solved by matching a reader with the writer (or using a Coordination Pattern)

Bac

kup

Data Access Pattern

Coordination Patterns

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Sequencer

Problem

‣ Often occurs in a distributed system that applications need to coordinate and take turn in performing certain actions.

‣ However DDS does not provide built-in coordination / distributed synchronization mechanisms. How can this be overcome?

Solution

‣ Define a Sequencer in your system that coordinates access to resources.

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Sequencer

Detailed Solution

‣ Define in your system the following Topics:

struct TAccessRequest { long resource_guid; long request_guid; time_t timeout;};#pragma keylist AccessRequestTopic resource_guid

struct TAccessGrant { long resource_guid; long request_guid; time_t timeout;};#pragma keylist AccessRequestTopic resrouce_guid

struct TReleaseAccessGrant { long resource_guid; long request_guid;};#pragma keylist AccessRequestTopic resrouce_guid

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


SequencerDetailed Solution

‣Make TAccessRequest and TReleaseAccessGrant Topic-Queues (with DESTINATION_ORDER set to RECEPTION_TIMESTAMP)

‣ Make TAccessGrant a Topic-Cache

‣ Use the following protocol to request/grant/release access

Sequencer (per instance to keep it simpler)

while (true) { take next sample from TAccessRequest write TAccessGrant wait on TReleaseAccessGrant}

Application

bool granted = false;write TAccessRequestwhile (!granted) { wait on TAccessGrant if TAccessGrant == myTAccessGrant granted = true;}// Do Critical Sectionwrite TReleaseAccessGrant

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Sequencer @ WorkTAccessRequest TAccessGrant TReleaseAccessGrant

SequencerReader

Writer

App-5

TAccessGrantTAccessRequest

App-10

TAccessRequest TAccessGrant

DDS

TReleaseAccessGrant TReleaseAccessGrant

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.



SequencerReader

Writer

App-5

2 5


App-10

1 10


DDS


Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Sequencer @ Work

1 10

2 5

TAccessRequest TAccessGrant TReleaseAccessGrant

SequencerReader

Writer

App-5

2 5


App-10

1 10


DDS


Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.



SequencerReader

Writer

App-5

2 5


App-10

1 10


DDS


1 10

2 5

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.



SequencerReader

Writer

App-5

2 5


App-10

1 10


DDS


1 10

2 5

1 10

2 5

1 10

2 5

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.



SequencerReader

Writer

App-5

2 5


App-10

1 10


DDS


1 10

2 5

1 10

2 5

1 10

2 51 10

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.



SequencerReader

Writer

App-5

2 5


App-10

1 10


DDS


1 10

2 5

1 10

2 5

1 10

2 51 10

1 10

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.



SequencerReader

Writer

App-5

2 5


App-10

1 10


DDS


1 -1

2 5

1 10

2 5

1 10

2 51 5

3 5

3 10

1 10

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.



SequencerReader

Writer

App-5

2 5


App-10

1 10


DDS


1 -1

2 5

1 -1

2 5

1 -1

2 51 5

3 5

3 10

3 5

1 5

3 10

1 10

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.



SequencerReader

Writer

App-5

2 5


App-10

1 10


DDS


1 5

2 5

1 -1

2 5

1 -1

2 51 5

3 5

3 10

3 53 10

1 10

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.



SequencerReader

Writer

App-5

2 5


App-10

1 10


DDS


1 5

2 5

1 -1

2 5

1 5

2 51 5

3 5

3 10

3 53 10

3 53 5

1 10

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Barriers

Problem

‣ Often occurs in a distributed system that applications need to reach a common step in computation before proceeding.

‣ This is required for distributed application creating software pipelines, or even to ensure proper state evolution of the distributed system.

‣ However DDS does not provide built-in coordination / distributed synchronization mechanisms. How can this be overcome?

Solution

‣ Use Barriers in your system to coordinate application progress.

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Barriers

Detailed Solution

‣ Define in your system the following Topics:

struct TBarrier { long computation_guid; long task_guid; long status;};#pragma keylist Barrier computation_guid task_guid

struct TBarrierCondition { long computation_guid; long status; long cardinality;};#pragma keylist Barrier computation_guid

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


BarriersDetailed Solution

‣ Make TBarrier and TBarrierCondition a Topic-Cache

‣ Use the following protocol to request/grant/release access

Application

// Do Computation

// Notify otherswrite Barrier

wait for BarrierCondition.cardinality Barrier instances to have the proper Barrier.status

// Barrier has been passed... // Take next computational step

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Barriers @ Work

TBarrierCondition

Coordinator

Reader

Writer

App-10 DDS

1 0xFF 3

1 0xFF 3

TBarrierCondition

1 10 0xF1

1 7 0xF1

1 5 0xF1

App-5

1 0xFF 3

TBarrierCondition

1 10 0xF1

1 7 0xF1

1 5 0xF1

App-7

1 0xFF 3

TBarrierCondition

1 10 0xF1

1 7 0xF1

1 5 0xF1

TBarrier

TBarrier

1 7 0xF1

TBarrier

1 10 0xF1

TBarrier

1 5 0xF1

TBarrier TBarrier

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Barriers @ Work

TBarrierCondition

Coordinator

Reader

Writer

App-10 DDS

1 0xFF 3

1 0xFF 3

TBarrierCondition

1 10 0xF1

1 7 0xF1

1 5 0xF1

App-5

1 0xFF 3

TBarrierCondition

1 10 0xF1

1 7 0xF1

1 5 0xF1

App-7

1 0xFF 3

TBarrierCondition

1 10 0xF1

1 7 0xFF

1 5 0xF1

TBarrier

TBarrier

1 7 0xFF

TBarrier

1 10 0xF1

TBarrier

1 5 0xF1

TBarrier TBarrier

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Barriers @ Work

TBarrierCondition

Coordinator

Reader

Writer

App-10 DDS

1 0xFF 3

1 0xFF 3

TBarrierCondition

1 10 0xF1

1 7 0xFF

1 5 0xF1

App-5

1 0xFF 3

TBarrierCondition

1 10 0xF1

1 7 0xFF

1 5 0xF1

App-7

1 0xFF 3

TBarrierCondition

1 10 0xF1

1 7 0xFF

1 5 0xF1

TBarrier

TBarrier

1 7 0xFF

TBarrier

1 10 0xF1

TBarrier

1 5 0xF1

TBarrier TBarrier

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Barriers @ Work

TBarrierCondition

Coordinator

Reader

Writer

App-10 DDS

1 0xFF 3

1 0xFF 3

TBarrierCondition

1 10 0xF1

1 7 0xFF

1 5 0xF1

App-5

1 0xFF 3

TBarrierCondition

1 10 0xF1

1 7 0xFF

1 5 0xFF

App-7

1 0xFF 3

TBarrierCondition

1 10 0xF1

1 7 0xFF

1 5 0xF1

TBarrier

TBarrier

1 7 0xFF

TBarrier

1 10 0xF1

TBarrier

1 5 0xFF

TBarrier TBarrier

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Barriers @ Work

TBarrierCondition

Coordinator

Reader

Writer

App-10 DDS

1 0xFF 3

1 0xFF 3

TBarrierCondition

1 10 0xF1

1 7 0xFF

1 5 0xFF

App-5

1 0xFF 3

TBarrierCondition

1 10 0xF1

1 7 0xFF

1 5 0xFF

App-7

1 0xFF 3

TBarrierCondition

1 10 0xF1

1 7 0xFF

1 5 0xFF

TBarrier

TBarrier

1 7 0xFF

TBarrier

1 10 0xF1

TBarrier

1 5 0xFF

TBarrier TBarrier

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Barriers @ Work

TBarrierCondition

Coordinator

Reader

Writer

App-10 DDS

1 0xFF 3

1 0xFF 3

TBarrierCondition

1 10 0xFF

1 7 0xFF

1 5 0xFF

App-5

1 0xFF 3

TBarrierCondition

1 10 0xF1

1 7 0xFF

1 5 0xFF

App-7

1 0xFF 3

TBarrierCondition

1 10 0xF1

1 7 0xFF

1 5 0xFF

TBarrier

TBarrier

1 7 0xFF

TBarrier

1 10 0xFF

TBarrier

1 5 0xFF

TBarrier TBarrier

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Barriers @ Work

TBarrierCondition

Coordinator

Reader

Writer

App-10 DDS

1 0xFF 3

1 0xFF 3

TBarrierCondition

1 10 0xFF

1 7 0xFF

1 5 0xFF

App-5

1 0xFF 3

TBarrierCondition

1 10 0xFF

1 7 0xFF

1 5 0xFF

App-7

1 0xFF 3

TBarrierCondition

1 10 0xFF

1 7 0xFF

1 5 0xFF

TBarrier

TBarrier

1 7 0xFF

TBarrier

1 10 0xFF

TBarrier

1 5 0xFF

TBarrier TBarrier

Pro

prie

tary

Info

rmat

ion

- D

istri

butio

n w

ithou

t Exp

ress

ed W

ritte

n P

erm

issi

on is

Pro

hibi

ted.


Leader ElectionProblem

‣ Several distributed algorithms require some form of leader

‣ Problems requiring a leader, can be addressed using the Sequencer Pattern

‣ However, what if the sequencer crashes?

Solution

‣ Use the DDS OWNERSHIP_STRENGH as a mechanism to do leader election via DDS

Observation

‣ The basic Leader Election functionality provided by DDS can be used to easily replicate service implementation such as those of the Sequencer

‣ If the leader is stateful, its state should be stored within DDS

omg dds tutorial - part i

Technology

datadistribution challenges

dds tutorial

dds applicabilityadapted

dds adoptions

dds standardudpip

dds sig

dds recommendations

written permission