overview - international test and evaluation association...• the dcgs‐a pmo can publish a...

Test.Forge.mil

Overview

© 2011 All Rights Reserved

Improving Software Development

Reduce Cost

More Testing

Faster Delivery

Agile Development

????

Virtualization

Cloud


Current Process

Program Office

Development Contractor(s)

TestContractor(s)

IV&VAgency(s)

Ops Team


Test.Forge.mil Goals

Provide access to the all resources necessary…

Desktops & servers

…while leveraging automation and virtualization to expand resources.

Operating Systems & updates

Applications & code

Test cases

… so they can support &perform rapid, repeatable, agile, on‐demand testing of software systems…

Virtual services

Functional Performance Regression DT OT

Virtual machinesVirtual users

...to all parties involved…

Program Office Developer Tester Community Member


O/S & Application Provisioning

Machine VirtualizationCloud

Virtual Services

Application Lifecycle

ManagementSource Code

Build Test Tools

What it is

Test.Forge.mil

* some products for illustrative purpose; not all listed are implemented at this time

Hosting

Messaging

Backb

one

Data Center

PKI AuthenticationForge.mil SSO Database

Community.Forge.milIntegration

Remote Access Solutions

UI & Other Requirements

CONS3RT PaaS BaaS TaaS


Local Cloud Private Cloud Public Cloud

Test as a Service

Tools

How it Works

Asset Library

O/S Test AssetsAppsVirtual ServicesConfigs

Scenarios/AppliancesVMs Servers Devices

Scenario #7

Web Collaboration Scenario

Source Code

Web Collaboration Scenario

Platform as a Service

Provisioning

Virtualization

Build as a Service


Sample Project

Sample Project: ObjectiveBuild a web dashboard for senior leadership that displays DoD activities alongside current events and weather for situational awareness purposes.

DCAPES

Map

Weather

News

Troops

Location: Kabul, Afghanistan

84,375

21,222

18,153

34

111

Use of program/organization names are for illustration purposes and do not represent any official involvement with Test.Forge


Test.Forge Asset Library

Test.Forge Dashboard Project

O/S Test AssetsAppsVirtual ServicesConfigs

Scenarios/Appliances

• The Master Dashboard PMO can define the standard configurations for the systems to be used in development & test.• The DCGS‐A PMO can publish a standard DCGS system to be used in development & test.• The TBMCS PMO can publish a virtual service model with standard inputs/outputs for development & test.• JITC can provide standard interoperability tests tat are run using existing test tools.• The developer can build & save virtual service models of some of the industry news feeds to expedite efforts.• The developer can run continuous integration testing with every daily build.• The testers can begin creating & validating test cases in parallel with development.• The developers & testers can quickly evaluate a “what‐if” scenario by replacing one part of the system (e.g. IBM vs. JBOSS).

Forge.mil

Master Dashboard Project

External CloudTest.Forge Services

CONS3RT

Infrastructure(NTP, DNS, DCHP)

Test as a Service(LISA, Selenium, W3AF, etc.)

Virtual Service Engine(s)

DCGS‐A Project TBMCS Project

Test.Forge Cloud

External CloudMaestro

FAP

VSManager

GUI

TestManager

Messaging

Dashboard v0.2

Program Office Developer Tester Community Member

Use of program/organization names are for illustration purposes and do not represent any official involvement with Test.Forge

DCGS v10.1

Dashboard v0.3


What it does

Test.Forge.mil enables the ability to …– build virtual machines

– install multiple software applications onto these virtual machines

– pull, build & deploy source code onto these virtual machines

– mange virtual service models

– combine virtual machines & virtual service models into complex “systems of systems” scenarios

– deploy (turn on) these scenarios

– select the destination cloud for the deployment

– run automated tests against a deployment

– save and reuse all the components, scenarios and deployments

– execute these tasks • on demand

• with any combination on Windows and Linux components


How to use it

Test.Forge.mil can be used for…– Platform as a Service

• build workstations for developers to write code• build workstations for testers to write test• perform manual tests• “what if” analysis

– Test as a Service• execute regression testing on new builds• perform smoke tests• performance testing• “what if” analysis

– Continuous Delivery• continuous integration (big & little)• compliance testing• Build as a Service

– Collaboration• share knowledge, techniques & technology across communities • connect non‐developers & non‐testers


Benefits

Goals & Benefits:– On‐demand environments to support all levels of testing

– Shorten timelines, reduce cost for testing

– Push quality down to source of problems

– Test early, test often (every week, every day, every hour)

– Pull in all parties involved in development (not just developers)

– Act as a “Test Range” for IT Systems

– Incorporate best of breed COTS/GOTS test tools

– Transform “cloud” from buzz word to useful tool

– Reduce risk

– Work with common tool sets for testing, automation


Contact Information

John CaseboltPresident, CTOJackpine [email protected]: (978) 263‐6025Cell: (978) 394‐2195

Peter WalshProgram Manager, NPLACEWVHTC Foundation, [email protected]@hanscom.af.milPhone: (781) 377‐7211DSN: 478‐7211Cell: (617) 816‐6001

ESI3 Lab15 Eglin StreetBuilding 1607, Shelter 10Hanscom AFB, MA 01731(781) 377‐7184DSN 478‐7184https://www.esi3l.ceif.hpc.mil

Backups


Future PhasesPilot Phase

Test.Forge Feature Roadmap*

Scenarios, Assets & Clouds

• build & deploy virtual machines

• build & deploy virtual services

• deploy into other cloud(s)

• connect your cloud

• deploy on hardware

• config & deploy devices

• nested scenarios

• automated O/S updates

• dynamic resource management

Testing

• automated test workflow

• execute iTKO LISA tests

• execute soapUI tests

• execute Selenium tests

• execute W3AF scans

• execute Retina scans

• execute Fortify code scans

• execute HP LoadRunners tests

• consolidated results reporting

Other

• integration in Forge.mil

• single sign‐on

• users & groups

• CAC authentication

• asset visibility rules

• NIPR accreditation

• project level resources

• billing model metrics

• license management

*Feature set subject to change during the development process


How We are Building it

local (IDE) build

Jenkins & MavenNew Code

Good build?

Deploy in INT

Debug & test

Pass tests?

Deploy in CI

Deploy in QA

Deploy in Prod

Smoke/Regression

Smoke/Regression

New tests

Pass tests?

New stuff?

Jenkins & Maven

Develop

erIntegration

Continuo

usIntegration

QA

Prod

uction

Pass tests?

New stuff?

Jenkins & Maven

Tag

Tag

Delete artifact

svncommit

overview - international test and evaluation association...• the dcgs‐a pmo can publish a...

Documents