cybersecurity test and evaluation
TRANSCRIPT
Approved for public release; distribution is
unlimited. Understanding
Cybersecurity Test
and Evaluation Achievable and Defensible
Architectures
October 2015, ITEA Francis Scott Key Chapter
Mr. Robert L. Laughman for COL Scott D. Brooks, Director, Survivability Evaluation Directorate, Army Evaluation Center
U.S. Army Evaluation Center
Agenda
• Trends
• Technology
• Cyber Security
• Testing & Evaluation
• New Approaches
• Challenges
3
2020
2015
4
Reported Breaches by Year
5
Operational Cybersecurity Testing
Blue Team Assessment Tools
• Nmap – network mapping, traffic generation
• Q-Tip, Retina, Nessus – signature based
vulnerability scanners, malware signatures
updated daily
• SCAP Compliance Checker – Automated
scanner of systems based on DISA Secure
Technical Implementation Guidelines
(STIGs)
• Burp Proxy – web application proxy (man-in-
the-middle for assessing web application
vulnerabilities)
• Wireshark, Tcpdump – traffic analysis, can
capture wired and wireless packets.
• John, Cain – password crackers
• THC-Hydra – password guessers
Red Team Assessment Tools
• Nmap – covert network mapping, firewall
evasion, traffic generation
• Metasploit – exploitation and post-
exploitation toolset (exploits vulnerabilities
and delivers a payload)
• Meterpreter – Windows Metasploit Payload
used for keyboard logging, enabling camera,
microphone, data theft, maintaining access,
and covert communications.
• Burp Proxy, Zed Attack Proxy –web
application attacks
• BEEF— web browser exploitation toolset
• MimiKatz – memory forensics
• Cobalt Strike – advanced exploitation
toolset with graphic interface
• John, Cain – password crackers
• THC-Hydra – password guessers
6
Cybersecurity Testing • Shift Left
• Formally add cybersecurity DT to the TEMP
• ATEC: Leverage existing test capabilities
rather than build new
• Build T&E plans starting with Risk
Management Framework (RMF) products
• Risk Management Framework (RMF)
replaces DIACAP with intent to
manage risk over the system’s lifecycle
• Score Card
7
A B C IATT OTRR
OT Cooperative
Vulnerability
and Penetration
Assessment
OT
Adversarial
Assessment
FRP
Phase 5
Test
Event
Phase 6
Test
Event
DT
Adversarial
Assessment
Phase 4
Test
Event
Phase 3
Test
Event
DT
Cooperative
Vulnerability
and Penetration
Assessment
Events derived from DASD(DT&E) DoD Cybersecurity
Test and Evaluation Guidebook 1 July 2015 ver 1, and
DOT&E Cybersecurity Operational Test and
Evaluation Guidance Memo (01 August 2014)
Phase 2 Phase 1
Understand
Cybersecurity
Requirements
Characterize
Cyber Attack
Surface
Developmental Test
Integrated DT/OT Operational Test
Analysis phase
Test phase
RMF
Shift Left Cybersecurity T&E Earlier Than IOT&E
https://acc.dau.mil/adl/en-
US/722865/file/80161/Cyberse
curity%20TE%20Guidebook%
20July%201%202015%20v1_0
Cybersecurity T&E approach (IAW AR 25-2, DoDI 8510.01, and DASD(DT&E) & DOT&E guidance*) mitigates software and security risks of fielding unproven platform equipment. Applicable data will be leveraged whenever available.
Cybersecurity Test and Evaluation Approach
Major Software Updates
New Software
Existing Evaluation New Integration
New Hardware
Su
bsyste
m E
xam
ple
s
- Computing Systems - Improved Displays - New Processor Units - Maneuver Control Enhancements
- Cross Domain Solution Adjustments - Enhanced Training - Improved Vehicle Management - Improved Communications Manager
- CREW Device - Tactical Communication Devices - Battle Command Systems - Power Distribution Systems
OEM Cybersecurity
Testing
Software Drop
Post-OEM Testing
DT Cybersecurity
Testing
Software Drop
Post-DT
OT Cybersecurity
Testing
Software Drop
Post-OT
Software Lifecycle
Maintenance
Continuous Cooperative Vulnerability and Penetration Assessments (CVPA): • system focused • Risk-Based – most likely to be
exploited • Actionable Information
Network End to End • Data Exchange Relationships • Security Approach
8
Security Engineering
Challenges •Incorporation of security engineering as
a discipline of systems engineering –Engineering methodology, processes, and
practices
–System security engineering workforce
•Quantification of security risks –Vulnerability detection, and validated mitigation
•Articulation of security requirements –Threat-driven, evolving over time
–Risk-based affordable trade off analysis;
Measurable, testable system specifications
•Protection of technical data –Consequences of unclassified controlled
technical information losses
NDIA Summit DoD Program Protection May 19-22 2014
http://www.ndia.org/Divisions/Divisions/SystemsEngineering/Pages/Past_Projects.aspx
9
Common Themes: • Security Engineering
as Discipline
• Earlier & Often in the
Development
Process
• Architecture
• In Contracts: Part of
Section L and M in
RFPs
• Cyber Testing
Challenges for T&E
• OSD policies on cybersecurity T&E still draft
• DoDI 5000.02 states need for cybersecurity in DT
• AR 73-1 Draft in Process
• Modeling & Simulation
• Operational Requirements
• Addressing DOTMLPF
• Training and CND activity at Echelon
• Metrics – Work underway with MIT-LL
• Measurable, Testable, Repeatable
• Configuration
• Operational Mission Risk
10
Measures Account Management - Accounts are established only after screening users for membership,
need-to-know, and functional tasks, and disestablished promptly when retired. Default
credentials are designed into software to be changed on first use.
Least Privilege – Use Role based account privileges assuring only access to systems/applications
user has need to use.
Identification and Authentication - Organizational users are uniquely identified and authenticated
when accessing the system, including accounts. Two level authentication or higher.
Content of Audit Records - Audit records contain sufficient information to establish the nature,
time, location, source and outcome of malicious events, as well as the identity of any
individuals associated with such events.
Audit Review, Analysis and Reporting - Audit records are reviewed and analyzed promptly for
indications of inappropriate activity, and any findings are reported to the defenders.
Continuous Monitoring The system is continuously monitored for vulnerabilities, to include regular
assessments by test teams.
Configuration Settings .The system is installed in accordance with an established baseline
configuration following the principle of least functionality, and any deviations from this
baseline are recorded.
Backup, Recovery and Restoration System data is backed up and preserved, and a recovery and
restoration plan for the system is
Device Identification and Authentication
The information system uniquely identifies and authenticates devices before a
connection.
Authenticator Management The cryptographic strength, maximum lifetime and
storage methods for system authenticators(e.g., password, tokens) are
compliant with organizational policy.
Default Authenticators System authenticators (e.g., password, tokens) are changed
from
their default settings.
Physical Access Control The information system, including data ports, is physically
protected
from unauthorized access appropriate to the level of classification.
Boundary Protection The system monitors and data exchanges at the external
boundary and at key internal boundaries, including: Firewalls or guard;
IPS/IDS/HBSS
Secure Network Communications Network communications are secure and remote
sessions require a secure form of authentication.
Update Management Security-related software firmware updates (e.g. patches) are
centrally managed and applied to all instances of the system in accordance
with the relevant direction and timeliness.
Malicious Code Protection Mechanisms for preventing the deployment of malicious
code (e.g.,
viruses, malware) are installed, configured and kept up-to-date.
11
DOT&E Cybersecurity Operational Test and Evaluation Guidance Memo (01
August 2014)
Path to Achievable and
Defensible Networks
• Operational Requirements Documents
• Contract Language
• Architecture
• Design and Planning
• Inherited Controls
• Testing (ACAS, SCAP, CVPA and Adversarial)
• Changes in HW, SW or Architecture
12
Defensible
Systems