10/7/2019

1

Governance for Artificial Intelligence/

Machine Learning

Akbar Siddiqui

Technical Director

Civil Liberties, Privacy, and Transparency Office

National Security Agency

1

2

10/7/2019

2

The NSA Mission

Signals Intelligence

The National Security Agency is responsible for:

Providing our nation’s policy

makers and military commands

with foreign intelligence to gain a

decisive advantage.

U.S. CybersecurityProtecting and defending

sensitive information systems

and networks critical to national

security and infrastructure.

What is AI/ML?

Unsupervised

Learning

MACHINE

LEARNING

Reinforcement

Learning

Supervised

Learning

Game AI

SkillsAcquisition

Learning Tasks

Robot Navigation

Real-time Decisions

Estimating Life Expectancy

PopulationGrowth Prediction

Market Forecasting

Weather Forecasting

Advertising Popularity Prediction

Classification

Regression

Diagnostics

Customer Retention

Image Classification

Identity Fraud DetectionMeaningful

Compression

Clustering

Big Data Visualization

Structure Discovery

Feature Elicitation

Recommender Systems

Targeted Marketing

Customer Segmentation

Dimensional

Reduction

Raw Data

Labeled Training Data

Experimental Parameters

Models

Output

3

4

10/7/2019

3

Governance in the Process

Purpose Collect Process Evaluate Retain Disseminate

Authority

Training

Guidance

Compliance Controls

Technical Safeguards

Governance in the Process

Purpose Collect Process Evaluate Retain Disseminate

Authority

Training

Guidance

Compliance Controls

Technical Safeguards

5

6

10/7/2019

4

Governance in the Process

Purpose Collect Process Evaluate Retain DisseminatePurposeTraining

DataDevelop Model

Apply to Data

Use Outputs

Feedback

Machine Learning Process Safeguards

Purpose

• Authorities, Ethical Use

• Explainable Purpose and Methods

Training Data

• Collect or generate training data and test data

• Documentation: Datasheets for Datasets

Develop Model

• Explainability

• Testing/Validation

• Confidence Levels

• Documentation: Model Cards

Apply to Data

• Use Limitation

• User Interpretation and Validation

Use Outputs

• Accountability

• Explainability and Redress

• Human control

• Confidence level follows outputs

Feedback

• Build user into workflow

• Check for adversarial techniques

7

8

10/7/2019

5

Defined Purpose and Use

• Governance Bodies

• Check for Authorities

• Check for Ethical Use (Principles)

• Explainable Purpose and Methods

Purpose

Training Data

• Collect or generate training data and test data• Data Selection, Feature Engineering, Labeling

• Issue: Collecting and maintaining “negative” examples

• Documentation: Datasheets for Datasets• Identify and document features, purpose, limitations, and

known issues

• biases (explicit and implicit)

Training DataPurpose

9

10

10/7/2019

6

Model Development

• Explainability

• Testing and Validation

• Check for bias in weights/methodology

• ID situations where model performs poorly/unreliably or is vulnerable to

adversarial techniques

• Confidence Level

• Documentation: Model Cards

Develop Model

Training DataPurpose

Stakeholders

ModelMission Review

Peer ReviewSenior Operations

Data Officer (SODO),

Senior Operations Analytics Officer (SOAO)

Sharing Equities

(Equities Review Board)

Mission Risk Acceptance

(Mission Element Owners)

Security

Business Intelligence Metrics

(ROI)

Labeled Data

Civil Liberties, Privacy,

and Transparency (CLPT)

Compliance for Dissemination (LPOC)

Legal

Deployment Compliance (AVG)

Chief Data Officer (CDO)

11

12

10/7/2019

7

Applying Models to Data

• Use Limitation

• User Interpretation and Validation

Develop Model

Training DataPurpose Apply to Data

Using Outputs

• Accountability

• Explainability and Redress

• Human control

• Confidence level follows outputs

Develop Model

Training DataPurpose Apply to Data Use Outputs

13

14

10/7/2019

8

Feedback

• Build user corrections into workflow

• Drift

• Biased weights

• Check for adversarial techniques

Develop Model

Training DataPurpose Apply to Data Use Outputs

Feedback

Q & A

15

16

10/7/2019

9

18

Mitigating Adversarial Machine Learning

*Popular Science: Fooling The Machine, The

Byzantine science of deceiving artificial intelligence; Dave Gershgorn (March 30, 2016)

Original outcome →

Tricked outcome →

Machine learning (ML) can be a solution to scalable defensive and offensive measures for cybersecurity. These can range from semi-automated decision support to fully-automated capabilities. However, ML models can be exploited in at least four ways. Adversaries can:

(a) poison training data used to train ML algorithms to degrade prediction quality, or redirect predictions, altogether;

(b) evade by manipulating runtime data to ensure ML models misclassify malicious behavior as benign;

(c) infer records into the training data; and

(d) reconstruct the ML model for further analysis and exploitation.

When ML models of varying qualities are integrated into an ensemble, an adversary can exploit weaknesses in individual models to coordinate a malicious effect in the overall system.

17

18