Test Data Management System

(TDMS)

Richard Haire, Ravi Sundaar, Ashi Chaturvedula,

Aretus and TRAX Int’l

May 17, 2012

Overview

• Problem Statement: Disjointed Data Management

• Data Acquisition on the Ranges

• A Comprehensive Approach to Data Management

– Highly Integrated Virtual Environment (HIVE): Vision & Benefits

– Design Considerations

• Test Data Management System (TDMS)

– Attributes, Current State, Next Steps

• Summary

• TDMS Current State Demonstration (Optional)

2

Disjointed Data Management

3

• Limited Situational Awareness

– Parochial view of testing; a focus on the individual event test

– Limited visibility of testing status, testing results, test data management (e.g.

islands of excellence)

• Lack of Operational Efficiency

– Scheduling test resources and services

– Setting up test and coordination among various stakeholders

– Assessing the impact of IA policies (DCN, ICAN, SIPR)

– Executing the Concept of Operations (CONOPS)

• Inadequate test data management processes – Directing data and content management/architecture strategy

– Securing test resources and ensuring IT alignment

RTSS

TrackEye

DWDM

Data Acquisition

= Current Initiatives

Report

Customers: Test Officer, Analyst, etc.

Data Center • Security

• Monitoring & management

• Applications, content & DBs

• Storage & backup

• Temp control

• Power conditioning

• Fire detection & suppression

Power,

HVAC,

Upgrade

Storage

Upgrade

TDMS

ADSS

SharePoint

Transport

Plan & Data Management

Analyze

Data Acquisition

Android-based devices and iPhones

for data processing in near real-time.

FSTE

Pedestal

Network

Wireless

Mesh

RRRP

LMR

1

2

3

4

5

5

Enterprise Service Bus (ESB)

Platform

ADSS

(PMES)

Application

Service

Platform

RTSS

Application

Service

Platform

TDMS

Application

Service

Platform

UFR/TDAP

Application

Service

Platform

TQD/WDB

Application

Service

Platform

ACAS

Application

Service

Applications & Analysis

Test Officers

Dashboard

Investments Financials

<TBD> Situational

Awareness

Joint Testing

Reporting

Range

Operations

Asset

Management

Strategic

Planning Application

ABC Application

XYZ

HIVE Certification

Web Services

Infrastructure

(WSDL)

HIVE is a platform that

allows global access to

content through user

developed applications.

Highly Integrated Virtual Environment Envisioned End-State

HIVE Architecture Benefits

6

Provides consistent, accurate, and reliable query results.

Meets ACID (atomicity, consistency, isolation, durability) requirements of transactions.

Unified interface and access-controlled via existing Active Directory & functional roles.

Significantly improves operational efficiency.

Automates data aggregation across support elements.

Dramatic improvements collaboration among Test Officers and support elements.

Establishes foundations for:

Collaboration across T&E installations.

Horizontal and vertical scaling to support multiple T&E installations.

Expansion of architecture (“hybrid” architecture) – precise nature of SQL-based structured

content, and NoSQL-based unstructured content.

Providing “What if” scenario analysis that leverages structured and unstructured data

sources across and within Test Centers.

Provides minimal disruption to current Test Center operations.

Comprehensive Data Management Design Considerations

7

• Coexistence

― Avoid major disruptions to existing test environment (allow JAVA & Microsoft)

― No adverse impact on established applications and processes at each Test Center

• Flexibility

― Ensure ACID of User Queries

― Provide dynamic metadata & metadata curation

― Enable horizontal & vertical scaling of underlying infrastructure

• Presuppositions

― Test data resides at source (avoid transfers of large volumes of data)

― Require each Test Centers’ current applications to “plug into” Enterprise Service

Bus to enable a Service Oriented Architecture

― “Push” Application/Code to source of data for Computation & Analytics

TDMS Attributes

8

YPG INITIATIVE WITHIN THE HIVE FRAMEWORK

• Provides Test Center-centric lifecycle management

of test data. (“cradle to archive to final purge”)

• Enables metadata lifecycle management.

(“standardize as you proceed without losing integrity”)

• Allows for a provider and consumer of information in a Service

Oriented Architecture (SOA) framework.

• Provides seamless authentication and authorization.

• Addresses specific needs of organization stakeholders to include

test officers, data analysts/technicians, and decision makers for

limited situational awareness.

Platform

TDMS

Application

Service

YPG INTEGRATED TEST DATA REPOSITORIES

File Systems SQL Server

CORE TDMS FUNCTIONS

Data Acquisition

Search/Data Integration

Taxonomy & Metadata

Normalization & Aggregation

USER & SECURITY MANAGEMENT

User Management

Authorization/Authentication

BUSINESS SERVICES

Basic Situational Awareness

Enterprise Search & Retrieval

Collaboration/Publishing/Distribution

Reporting &Metrics

Basic Business Analytics

Test Asset Management

Indexing

Archiving

Personalization

Security & Audit

ORG Accepts

New Test Program

Test Officer develops

Test Plan,

Schedules

Results

Compilation

Support Elements

collaborate &

Perform Test Data Acquisition

Test Data

Manipulation

and Analysis

Data and Results

Distribution

TDMS: Current State

(PMES)

Note: Expanded services and growth in several commodity areas underway.

(automotive, electronics, optics, air delivery, etc.)

Data Management – Next Steps…

10

What is needed?

• Allow the user community to execute specific queries that require

consistent and accurate results, for example,

– Enable user to retrieve “MRAP braking data” to get a big picture of what

has been done to date, and sample data to confirm testing approach.

– Enable user to locate and download all “MRAP mass property data” for

date range 1-Dec-07 to present.

• Allow the user community to conduct computation and predictive

analysis, and “What if” analysis, for example,

– Before an unfunded project gets approved, the decision makers need to

understand the impact on test capabilities should that request not be

funded.

• Expansion of the current architecture to leverage newer technologies

(Hadoop, NoSQL), and further design elaboration to accommodate

other Test Centers or MRTFBs.

– Expand people, organizations, tests, assets, & investments domains to

accommodate the larger T&E community.

NoSQL DB

YPG INTEGRATED TEST DATA REPOSITORIES

File Systems SQL Server

CORE TDMS FUNCTIONS

Data Acquisition

Basic Search & Retrieval

Taxonomy & Metadata

Normalization & Aggregation

USER & SECURITY MANAGEMENT

User Management

Authorization/Authentication

BUSINESS SERVICES

Situational Awareness

Enterprise Search & Retrieval

Collaboration/Publishing/Distribution

Reporting &Metrics

Business Analytics

Test Asset Management

Indexing

Archiving

Personalization

Security & Audit

Next Steps…

Users (JMETC, Evaluators, HQ), Applications & Analysis

Test Officers

Dashboard

Investments Financials

Near Real-Time

Test Validation Situational

Awareness

Joint Testing

Reporting

Range

Operations

Asset

Management

Strategic

Planning

Application

ABC Application

XYZ

Application

XYZ

Enterprise Service Bus (ESB)

Hadoop Framework (MAP-REDUCE) NoSQL Database For Metadata

NoSQL DB

Summary

• The HIVE platform can support the needs of T&E community;

HQs, Test Centers, evaluators, and PMs. This is possible

because:

– Underlying architecture and framework is flexible and leverages the strengths of

“Structured Content Management,” and “Unstructured Content Management” with

newer technologies.

Newer technologies include Hadoop/Map-Reduce/NoSQL used by Facebook, Twitter, Google, &

Amazon

– Platform enables individual Test Centers to Develop “Apps” to support their

stakeholders needs

• Implementation with minimal disruption to existing operations at

installations.

12

Q&A

13

TDMS Current State Demonstration (duration ≈10 minutes)

14

FOCUS AREAS

• Define concept of test program / test events / test items (system

under test (SUT)) / test data (csv, .xls, .txt, .ppt, .doc, .pdf, graphs,

images, video, etc.).

• Definition of metadata (under admin screens) – demonstrate the

flexibility of adding, disabling, creating new metadata types.

• Demonstrate the integration with another application via Web

Services (ATEC’s ATEC Decision Support System (ADSS)).

• TDMS scheduling framework is used to manage back-end jobs

(data import jobs across multiple sources on the Data Control

Network (DCN), database maintenance jobs, etc.).

Alternate Architecture

Investigated

15

GOOGLE APPLIANCE with Existing Test Data File Shares

• Test Data is typically stored in file shares in various formats

(unstructured content).

• Google appliance may be used to crawl content to build an index,

while a search interface is used to search against this index to

retrieve content.

• Uncertainty in meeting ACID (Atomicity, Consistency, Isolation,

and Durability) transaction requirements.

• Low reliability and consistency of query results due to lack of

“semantic” consistency of metadata (National Cancer Institute’s

CaDSRProject).

16

Enterprise Service Bus (ESB)

Hadoop Framework (MAP-REDUCE)

PlatformAD

SS

(PMES)

Application

Service

Platform

RTSS

Application

Service

PlatformTD

MS

Application

Service

PlatformUF

R/TDAP

Application

Service

PlatformTQ

D/WDB

Application

Service

PlatformAC

AS

Application

Service

Applications & Analysis

Test Officers

Dashboard

Investments Financials

Near Real-Time

Test Validation

Situational

Awareness

Joint Testing

Reporting

Range

Operations

Asset

Management

Strategic

Planning Application

ABC Application

XYZ

HIVE Certification

NoSQL Database For Metadata

Domain Data, Data

Location, Summary,

Identifying Data,

Metadata Extraction

Web Services

Infrastructure

(WSDL)

Time Series

Analysis,

Predictive

Analytics

HIVE is a platform that

allows global access to

content through user

developed applications.

Highly Integrated Virtual Environment Envisioned End-State