cadance: a criticality-aware deployment and configuration engine

1

CaDAnCE: A Criticality-aware Deployment And Configuration

Engine

Vanderbilt University Nashville, Tennessee

Institute for Software Integrated Systems

Gan Deng, Douglas Schmidt & Aniruddha Gokhale

www.dre.vanderbilt.eduPresented at ISORC 2008Orlando, FL, USA, May 5-7, 2008

2

New Demands on Open Distributed Real-time & Embedded (DRE) Systems

Open Distributed real-time & embedded (DRE) systems• Network-centric & larger-scale

“systems of systems”• Dynamic context• Stringent simultaneous QoS demands

• e.g., scalability, dependability, throughput

Open Distributed real-time & embedded (DRE) systems• Network-centric & larger-scale

“systems of systems”• Dynamic context• Stringent simultaneous QoS demands

• e.g., scalability, dependability, throughput

Middleware Bus

Container

…

SecurityReplication TransactionPersistence

…

Container

… …

3

NASA MMS Mission System Case StudyOperational String C

(Low Criticality)

Operational String A(High Criticality)

Operational String B(Medium Criticality)

FilteringTrace

DetectionState

EstimateGizmo

ControlTask

Comm

Data Analysis

Monitor

ScienceAgent

Science Agent

FilteringData

FusionMessaging

Gizmo

FilteringGizmo

• Operational String A A mission-critical task that collects important field data when a satellite moves to particular locations

–High criticality• Operational String B Domain-centric data analysis based on

predefined analysis models

–Medium criticality• Operational String C Collect auxiliary field data occasionally, such as

Sun zenith, satellite view zenith (only when requested by B)– Low criticality

A

B

CThreeOperational

StringExample

4

Execution Manager

DomainApplication

Manager

startLaunch

App AppApp AppstartLaunch

Node ManagerNode

ApplicationManager

NodeApplication

spawn

install

Node ManagerNode

ApplicationManager

NodeApplication

startLaunch

spawn

• Operational string deployment via Synchronous Method Invocation (SMI)

–ExecutionManager (EM) iteratively makes synchronous invocation on each NodeManager (NM) to install components

• Each NM propagates component port references back to the EM–DAM dispatches component port object references to NMs for connection

establishment, i.e., fulfill dependency requirement between components

Operational String Deployment: OMG D&C

Facet

Event Sink

Facet

Event Sink

5

• Context–A deployment order is the order in which operational strings are deployed,

which is determined by the dependencies between operational strings

• Problem–Deployment order inversion happens when a higher criticality operational

string has a dependency to a lower criticality operational string–A deployment order inversion can propagate across multiple operational

strings due to such dependencies

Challenge 1: Avoid Deployment Order Inversion

Operational String C(Low Criticality)



FilteringTrace

DetectionState

EstimateGizmo

ControlTask

Comm

Data Analysis

Monitor

ScienceAgent

Science Agent

FilteringData

FusionMessaging

Gizmo

FilteringGizmo

6

• Context–The dynamic nature of open DRE systems requires on-demand

deployment of a number of operational strings in one request–Each operational strings has certain utility value to the entire DRE

system

• Problem–While the D&C framework improves the predictability of high-criticality

operational strings by minimizing their D&C latencies, it should also preserve the D&C predictability of other operational strings

Challenge 2: Preserve Predictability of Other Operational Strings

Operational String C(Low Criticality)



FilteringTrace

DetectionState

EstimateGizmo

ControlTask

Comm

Data Analysis

Monitor

ScienceAgent

Science Agent

FilteringData

FusionMessaging

Gizmo

FilteringGizmo

7

Our Solution The CaDAnCE Approach• Develop an D&C framework called

Criticality-Aware Deployment And Configuration Engine (CaDAnCE)

–Based on our prior work on DAnCE

• Step 1 Convert a set of operational strings (based on XML-based deployment descriptors) into a set of directed graphs

• Step 2 Recompose the graphs based on operational strings criticalities & their dependency relationships by promoting components from one to another

• Step 3 Convert recomposed graphs back into a new set of in-memory operational strings

2

1

3

Benefit: Minimize D&C latency of mission-critical operational strings

8

Medium PriorityHigh Criticality Low Criticality

CaDAnCE(After Processing High Criticality String)

1st Iteration

Medium CriticalityHigh Criticality Low Criticality

CaDAnCE(After Processing Medium Criticality String)

2nd Iteration

Medium CriticalityHigh Criticality Low Criticality

• Sort & iterate through all the operational strings, from the highest criticality to the lowest

–Process all the external dependencies of each operational string sequentially

–For each operational string, identify the components causing deployment order inversion by tracing criticality-inverted dependency trace

–Remove all external dependencies causing deployment order inversion by promoting components from lower criticality ones to higher criticality ones

Overview of the Recomposition Algorithm in CaDAnCE

Component Promoting

Component Promoting

Removing all criticality-inverted dependencies avoids deployment order inversion

11

Execution Manager

DomainApplication

Manager

startLaunch

Node ManagerNode

ApplicationManager

Node ManagerNode

ApplicationManager

NodeApplication

NodeApplication

App AppApp AppstartLaunch

startLaunch

spawn spawn

install install

• Asynchronous deployment & serializability

–Apply Asynchronous Method Invocation (AMI) to take advantage of parallel processing among many nodes

–AMI, however, does not provide a mechanism to coordinate different nodes, which is required by D&C for connection establishment

–To ensure serializability of D&C process, we combine AMI with Asynchronous Method Handling (AMH)

Predictability Parallel Deployment via AMI/AMH

?

12

Empirical Evaluation of CaDAnCE – HW/SW Testbed• Run experiments on a prototype

implementation of NASA MMS Mission System – 15 components in a sample

operational strings– Experimented with up to 64

operational strings & 960 components in total

• Experiments performed in the ISIS Lab– Dual 2.8 GHz Xeon CPUs,

1GB of ram, 40GB HDD, & gigabit ethernet cards

– Real-time Fedora Core 4 Linux kernel version 2.6.20-rt8-UNI

– Used up to 6 nodes

Comm Ground

Gizmo 1 Filter 1 Analysis 1


Science Agent



13

Effects on Operational String Recomposition• Hypothesis

– CaDAnCE should not change the functional correctness while producing correct dependencies between operational strings

• Experimental design

– The experiments consist of 3 operational strings with criticality level of high, medium, and low, respectively

– Before the experiment, 2 arbitrary criticality-inverted external dependencies are populated

– Measure the total # of components & # of dependencies (both internal & external) of each operational string before & after applying the CaDAnCE algorithm

Comm Ground



Science Agent


Gizmo 4 Filter 4 Analysis 4 High

Comm Ground



Science Agent



Medium

Comm Ground



Science Agent



Low

15CaDAnCE minimizes D&C latency of mission-critical operational strings

End-to-End Depl oyment Latency of Operati onal Stri ngs

0

1000

2000

3000

4000

5000

Late

ncy

(mil

isec

onds

)

Low Pri ori tyMedi um Pri ori tyHi gh Pri ori tyLow Pri ori ty-AMIMed Pri ori ty-AMIHi gh Pri ori ty-AMITotal LatencyTotal Latency-AMILow Pri ori ty 650826 2010396

Medi um Pri ori ty 1303584 1605292Hi gh Pri ori ty 1908699 887649Low Pri ori ty-AMI 216942 670132Med Pri ori ty-AMI 434528 401323Hi gh Pri ori ty-AMI 636233 211345Total Latency 3863109 4503337

Wi thout CaDAnCE Wi th CaDAnCE

Without AMI/AMH

With AMI/AMH

Results & Analysis for the Experiment

• CaDAnCE reduced the D&C latency of high criticality operational strings significantly (Both w/ AMI/AMH & w/o AMI/AMH)

• Without AMI/AMH, the total end-to-end latency of all operational strings increased due to component distribution effect

• With AMI/AMH, the total end-to-end latency of all operational strings is almost the same as without CaDAnCE

16

D&C Latency vs. Criticality• Hypothesis

– CaDAnCE can avoid deployment order inversion when higher criticality operational strings have dependencies on lower criticality operational strings

• Experimental design

– Two experiments with different operational string configurations

– The first experiment has 3 operational strings with low operational string growth effect

• Each string has 15 components evenly distributed across 5 nodes

– The second experiment has 2 strings with high operational string growth effect (worst case scenario)

• Same configuration as above

Comm Ground



Science Agent



Comm



Science Agent



Ground

Comm Ground



Science Agent



Comm



Science Agent



Ground

Low Growth Effect

High Growth Effect

17

Results & Analysis for the Second Experiment

CaDAnCE preserves predictability of lower criticality strings in case of merge

• The latency of deploying the high criticality operational string is nearly the same as deploying it without applying the CaDAnCE algorithm

• However, the D&C latency of low criticality operational string increases & becomes the same as that of high criticality string – Due to the operational string merge effect

• In worse case scenario, CaDAnCE performs the same as baseline, i.e., without using CaDAnCE

Operational String Merge (Worst Case Scenario)

0

100

200

300

400

500

600

700

Lat

ency

(M

ilise

cond

s)

Low Priority

High Priority

Low Priority 216942 629922 220942

High Priority 636233 629922 625662

WithoutCaDAnCE

CaDAnCE w/oOptimization

CaDAnCE w/Optimization

18

Concluding Remarks• Component middleware has already

received widespread acceptance in enterprise business domains

– e.g.,J2EE, Web Services

• Developers of DRE systems have encountered limitations with the available component middleware platforms

• This research applies methodologies & techniques to improve both the human productivity & computing performance of D&C of component-based DRE systems

• Our techniques are most relevant with below two component models:– OMG CORBA Component Model

– OSOA Service Component Architecture (SCA)

Open-source software can be downloaded from www.dre.vanderbilt.edu/CIAO & www.dre.vanderbilt.edu/CoSMIC

DissertationResearch

Dimenstions

OptimizedD&C

Profile

D&C Profile

ComponentServer

Container

CCMComponent

POA

Internal

Interfaces

Ev

en

tS

ink

sF

ac

ets

Re

ce

pta

cle

sE

ve

nt

So

urc

esCo

mp

on

en

t C

on

text

Object Request Broker

Notification Service Lib

Event Service LibRT-Event

Service Lib.......

RT-EventService

Event ServiceNotification

Service

Service Usage Metadata



Middleware Service Configurator

Priority High

Operational String Deployment

Via CaDAnCE

Priority Low

……

Priority High Priority Low

……

2 3

Deployment Target Node

Node ManagerNode

ApplicationManager

Execution Manager

System Deployer

1. Deploy an component assembly

NodeApplication

Container

4. configure component server resources

DomainApplication

Manager

7. Install component

& homes

6. Createcontainers

StandardConfigurator

8. Configure attributes

3. Create component server

5. Load and initialize middleware services

Repository Manager

ITS D&CProfileD&C

Profile

2. Container-based publish/subscribe service deployment and configuration

1. A standards-based D&C framework for real-time component middleware

3. Predictable D&C middleware via operational string recompostion

1

19

Thank You

cadance: a criticality-aware deployment and configuration engine

Documents