M&S Based System Development and Testing in a Joint Net-Centric

Environment

Bernard P. Zeigler, Ph.D.,Co-Director,

Arizona Center for Integrative Modeling and Simulationwww.acims.arizona.edu

andJoint Interoperability Test Command

Fort Huachuca, AZ 85613-7051

There is an acute need for a new Net-centric testing paradigm at the enterprise level where joint and coalition operations are conducted (Editorial, ITEA Jnl, Sept, 2005)

Net-Centric T&E places an increased emphasis on standards conformance self-testing over the Global Information Grid Service Oriented Architecture (GIG-SOA ) and emphasizes services rather than systems

Simulation-based acquisition requires the use of M&S in all phases of the system development life-cycle

ConceptRefinement

TechnologyDevelopment

SystemDevelopment

& Demonstration

Production & Deployment

Operations & Support

Modelingand

Simulation

especially starting early in system development when defects can be caught and corrected with low cost

Systems of

Systems

Simulation-based development implies the need for simulation-based testing

-- an infrastructure to test a system that is first formulated as an abstract model is offered by distributed simulation

-- both the System under Test (SUT) and the test device are coupled by exchanging data packets on a network

Network

Test Device

send/receive messages

System Under Test (SUT)

send/receivemessages

Raises the question: How is the Test Device developed in an authoritative manner?

Specified as abstract

model, e.g. UML

Connecting middle ware, e.g. HLA

Problem: Testing Polar Opposite Requirements

To deal with the increasing complexity and advanced decision capabilities of C4ISR systems

=>testing methodology has to become more rigorous, in-depth and thorough

To keep up with the rapid change and short development life cycles expected from the system builders

=> tests have to be ready to conduct in time scales compatible with the agile development strategies of new systems.

Solution: employ formal M&S

• to increase capabilities for simulation-based testing and •as a basis to increase the automation of testing processes.

Testing of interface standards is a focus area for automated simulation-based testing.

Link-16 is required in all Joint and multi-national operations.

The Joint Interoperability Test Command (JITC) has developed an automated test generation methodology as its core technology for testing conformance of systems to Link-16 This methodology is fundamentally enabled by the DEVS formalized modeling and simulation approach

AWACS

TheaterWarning

ABL

DSP/SBIRS

F-15

JLENS

THAAD

PATRIOT

MEADS

ATACMSAVENGER

TEL

AEGIS (CEP)

SIS(MSCS)SIS(MSCS) Link-16specification

The JITC employed M&S-based testing for the initial major milestone evaluation of the Integrated Architecture Behavior Model (IABM) developed by the Joint Single Integrated Air Picture (SIAP) System Engineering Organization (JSSEO) in 2005.

Recent Successful Application

The test exercise produced significant results that uncovered flaws in the model design and added acknowledged value to the model development.

The ACTGen Development Team, NGIT & ACIMS was selected as the winner in the Cross-Function category for the 2004/2005 M&S Awards presented by the National Training Systems Association (NTSA).

Types of Distributed Simulation Testing

Testing Description Example

standards conformance

test whether system conforms to standardsupports interoperability

Tactical Data LinkStandards, Link-16, VMF, USMTF,

interoperability test whether systems can interoperate (at the syntactic, higher, levels)

Joint Translator Forwarder (JXF) – air to/from land exchange of tactical data

mission/capabilities test whether system of systems have capabilities required for mission

Joint Close Air Support (JCAS)

Levels of Distributed Simulation Testing

Level/Variation Description Example

One-on-OneStimulus/Response- managed time- real time

Test Driver sends message stimuli to which SUT responds

IABM SIAP

One-on-Many Federation-------------------------- managed time- real time

Test Driver sends message stimuli and controls supporting federates including sensors

IABM Peer to Peer Architecture

Multiplatform(Many-to-Manyinteraction)-------------------------- managed time- real time

Active mode (test driver controls scenario interaction)

Passive Mode (test manager opportunistically observes interaction

JIT 06-03 live C2 systems from USAF, USMC, Navy in remote locations

Joint Combined Hardware-in-the-Loop (HWIL) Event (JCHE) Phase 3 CNI pilot event “Runs for Record” (~Aug 06) (CNI = Navy Common Network Interface)

JXF tests

Test Driver controls the scenario

Multiplatform Distributed Simulation – controlled testing

Platform(System,

Component)

Platform(System,

Component)

Platform(System,

Component)

Test Driver

Distributed Observers look for opportunities to test

Multiplatform Distributed Simulation - uncontrolled testing

Platform(System,

Component)

Platform(System,

Component)

Platform(System,

Component)

Observer Observer Observer

Test Coordinator

Joint Test Enterprise Requirements

• Must respond to requirements for increasing complexity, evolving from standards conformance, to interoperability, to joint operations capabilities testing

• Top down test development methodology – aim for maximum reuse of test assets by supporting configuration of assets to meet objectives of each new exercise

• Requires support of appropriate concepts and levels of abstraction – e.g., Objectives, capabilities, functionalities, rule sets, test model federates

• Requires development of software at levels of abstraction with appropriate flexibility to interface with configurable middleware and hardware to meet conditions of tests, e.G., High speed/accuracy implies real-time operating system replaces default windows OS

• Develop new assets when no configuration of existing assets can satisfy objectives; then develop with reusability and fit within methodology in mind

Joint Test Enterprise Methodology

• Determine objectives up front and drive test configuration process

• Determine capabilities to be tested• Develop scenarios/mission threads to involve SUT use of

identified capabilities• Identify lower level functions supporting capabilities• Decompose functions into relevant sub sets of rules• Develop testable sequences of rules from relevant rule

sets• Apply automated development of test sequences as test

model federates

Semi-automated test suite design based on Experimental Frames

Standard ProvidesBehaviorRequirements at one or morelevels of SystemSpecification

Reference Master Model

Formalization by mapping into DEVS representationsCorresponding levels ofSystem Specification

Real-timeimplement-ation and execution

Simulation based testing in net-centric environment

Integrated Development and Testing Process

Summary

• Interoperability is mandated throughout the system development process to achieve mission effectiveness

• The emerging challenges of testing in a joint environment require a new paradigm

• A test methodology at the enterprise level that scales to net-centric services on the GIG is needed

• Automation of the generation of test environments is needed – illustrated for tactical data link standards conformance testing