test & evaluation/science & technology program · test automation: this domain addresses...

Distribution Statement A. Approved for public release: distribution unlimited.

Test & Evaluation/Science & Technology Program

Testing Domains Supporting SoS T&E

C4I & Software Intensive Systems Test (C4T)Test Technology Area (TTA)

13 December 2018Mr. Kenneth SanchezPhone: 805-989-0785Email: [email protected]

2Distribution Statement A. Approved for public release: distribution unlimited.

MissionDevelops technologies to test C4I and Software Intensive Systems that operate incomplex military environments. With emphasis on testing warfighter systemsmore thoroughly through earlier T&E in the acquisition lifecycle; improvedinteroperability of the DoD ranges and labs; and improved efficiency of DoD T&Eresources. C4T is divided into the following three domains:

1. Test Automation: This domain addresses automation technologies to assessC4I and Software Intensive Systems within the Joint Information EnvironmentNet-Centric Operational Environment (Theater to Tactical Edge). Test automationtechnologies will allow more thorough and optimized testing of such warfightersystems.

2. Distributed Testing: This domain addresses technologies to facilitate testquality environments utilizing a distributed infrastructure consisting of ServicesMajor Range and Test Facility Bases.

3. Modeling & Simulation: This domain addresses technologies to increase theuse of modeling and simulation for test and evaluation across a distributedinfrastructure consisting of live and constructive warfighter systems/environments; and validation of this infrastructure to test quality standards.

C4I & Software Intensive Systems Test (C4T)


C4TOverview

Innovate T&E: Joint, Early, Often, and Agile

Complex Warfare Environments

Battle increasingly sophisticated adversaries in increasingly complex environments*

Innovative approaches to how we fight, posture our force, and leverage our asymmetric strengths and technological advantages*

Distributed Testing (DT)• Reduce Distributed T&E Effects• Improve Security Posture of Systems• Advance Mission Context Data Collection• Analysis & Visualization

Modeling & Simulation (M&S)

Test Automation (TA)

• Advance Big Data Collection, Analysis & Visualization

• Improve Test Execution thru Automation/Cloud Computing

• Advance Testing for Next Generation of Handhelds

• Improve Automated Control of Targets

• Required Fidelity in Live and Simulated Environments

• V&V Techniques• Aggregation Techniques• Run-time Performance for RT

Applications• Systems, Communications,

Environmental Representations

C4T Overview

BIG DATA & CLOUD COMPUTING


C4T Definitions

Command, Control, Communications, Computers, and Intelligence Systems (C4I): Support net-centricity service-based architecture pattern for information sharing via building joint architectures and roadmaps for integrating joint airborne networking capabilities with the evolving ground, maritime, and space networks.

Software Intensive Systems (SIS): Increasingly complex systems characterized by thousands of platforms, sensors, decision nodes, weapons, and warfighters connected through heterogeneous wired and wireless networks.


C4T TTA

Test Automation Domain


C4T: Test Automation Test Use Case 1: Improve Automated Data Collection, Visualization,

Analysis & Reporting

TACTICALTSPI

STRUCTUREDDATA

C2 Operations/Analysts

Improve the test communities ability to analyze and make decisions from large volumes of structured and unstructured test data.

SEARCH:Parallel Search Engine

Technology

TARGET LOCATION

Test Conductor

Advanced Semantic Algorithms

UNSTRUCTUREDDATA

Use Smart Algorithms &Database Design Technologies

to Structure Data

VIDEO CHAT

TEXT VOICE

SCREENS


Software Intensive Systems (SIS) Typically Operate Within a System-of-System (SoS) to Assess Effectiveness

C4T: Software Intensive Systems TestingTest Use Case 2: Automated Testing in a Cloud Environment

Automated Testing in a Cloud Environment Executing 1000’s of Test Case Scenarios in Parallel

Automated Tools For Testing System

Effectiveness &

PerformanceVirtualized HW

Aspects of System(s) Under Test

Test Harness Technologies • Support automated/agile testing (individual system)• Minimize labor-intensive test procedures • Support of regression, developmental, operational,

and interoperability testing

Virtual Test Harness Virtual Test Harness

Virt

ual T

est H

arne

ss

Virtual Test Harness

System Under Test

New virtualization and cloud-based test technologies are needed to automatically create, execute, and control a large number of test cases operating in parallel


C4T: Test Automation Test Use Case 3: NextGen Handhelds and Widgets


C4T: Test AutomationTest Use Case 4: Automate Control of Tightly Maneuvering

Attacking Targets

A2AD

C2 Big Data

Agile Communications

Target Control Stations

ThreatAir Swarm

ThreatLand Swarm

ThreatSea Swarm

SystemsUnder Test

Coordinate and choreograph multiple targets with fewer operators


C4T TTA

Distributed Testing Domain


C4T: Distributed TestingTest Use Case 5: Remove Test Infrastructure Biases

Improve distributed test infrastructure through advanced synchronization algorithms to remove test biases

TACTICAL DATA

C2 DATA

INTEL DATAJo

int I

nfor

mat

ion

Envi

ronm

ent

Joint Information Environm

ent

Joint Mission Environment

Distributed Test InfrastructureT&E Instrumentation

Live Warfighter Systems

Virtual and Constructive Systems/Environments

Network Error:Latency & Packet Drop

Terrain Error:Tank in the Air

Target Positional Error: synchronization, terrain, and network

Test Control & Analysis


C4T: Distributed TestingTest Use Case 6: Cross Domain Solutions and Multi-Level Security

Multi-Level Security


Distributed Test Infrastructure

Improve testers ability to handle multi-level security (data at rest) and cross domain solutions (multiple data classifications in motion)

T&E Instrumentation



Joint Mission EnvironmentCDS

CD

SC

DS

CD

S




C4T: Distributed TestingTest Use Case 7: Assess Big Data Warfighter Systems

Improve ability to create and assess big data warfighter systems under emulated wartime modes of operation



Test Control

Distributed Common Ground System (DCGS) -C4I cross Service System employing Data-to-Decisions• Data generators and

simulators for Velocity, Variety, and Volume

• Platform analytics & fusion assessments

• Big Data environment replication

• Effectiveness and Performance of Individual system within SoS

• Automate regression testing


T&E Instrumentation




C4T: Distributed TestingTest Use Case 8: Agile Communications & Contested/Dense

Communications Environments

Improve distributed test environments ability to assess agile communication networks and Contested/Dense Communication Environments at wartime modes of operation

Test Control

Evolving/Agile/Complex Warfighter Communications Networks• Anti-Access/ Area Denial• Joint Aerial Layer Network• Moving Target Defenses

Contested/Dense Communication Networks• Stimulation/Emulation to

Replicate: Denied/degraded communications, Urban communications


T&E Instrumentation





Command & Control

Dense Urban EnvironmentCell Tower


C4T TTA

Modeling & Simulation Domain


C4T: Modeling & SimulationTest Use Case 9: Battlespace Environments Aggregation &

Determining Required Fidelity

Low Resolution

50 K

ilom

eter

s

Develop analytical aggregation techniques that will allow scaling of the test environment, while maintaining SUT required fidelity and update rates.

Representing a large numbers of target objects with tactical relevancy (200-300 high-resolution entities in a real-time test execution)

Forces on Test Range

High Resolution

Airborne Sensor Under Test

Objects in Mechanized Infantry Battalion

- Armored Personnel Carrier (50)

- Wheeled Personnel Vehicle (10)

- Ground Missiles (30)- Mortar (10)- Personnel (500)- Approx. Tactical

Objects (600)

75 Kilometers

High Resolution

Low Resolution


Simulated Test Environment of Battlefield Robustness

Sensor Platform/Aircraft Movement SimulationUpdate Rate (200 msec)

Terrain Generator100 Meters Per Grid Tile Rocket Sensor Guidance Software

System Under Test (SUT)

Rocket Fly Out Simulation

Update Rate (100 msec)Non-Continuous

Ground Target/VehicleMovement Simulation

Update Rate (1 sec)Non-Continuous

Sensor Update Rate (50 msec); Guidance Update Rate (100 msec); Accuracy ≈ 6m (CEP)

Mature technologies that increase the scope of mission performance testing, that could occur across the Department of Defense (DoD) distributed T&E infrastructure .

C4T: Modeling & SimulationTest Use Case 10: M&S for T&E of Systems in Complex

Environments (Representations and V&V)


C4T: Modeling and SimulationTest Use Case 11: M&S for T&E of Systems in Complex

Environments (Run-Time Performance)

Distributed M&S Infrastructure for T&E

T&E Instrumentation

Warfighter Systems

High Fidelity Simulated Environments & Systems

Live Environments & Systems

Virtual and Constructive M&S

Improve Simulation Run-

time PerformanceM&S Technologies for Torpedo Testing


Closing

• C4T TTA investing in technologies to support SoST&E– Test Automation– Distributed Testing– Modeling and Simulation

test & evaluation/science & technology program · test automation: this domain addresses...

Documents