Lieutenant General Jack Shanahan OUSDI Director for Defense Intelligence (Warfighter Support)

20 March 2018

Disruption in UAS: The Algorithmic Warfare Cross-Functional Team

(Project Maven)

UNCLASSIFIED

Unmanned Systems Full Motion Video Demand

Potential Increase In FMV

Thou

sand

sofFlightHou

rsperYear

450

600

650

700

750

550

500

400

800

850

900

950

350

The Maven Journey

Taskto

ReviewPEDAutomationforTactical

UAS

ProblemFramingandAnalysis

PEDAutomationWorkingGroup

AIResearchwith

AcademiaandIndustry

“TaskForce”Proposaland

COAs

DepSecDefEstablishesAlgorithmicWarfareCFT

“Thevolumeofdatabeingcollectedvastlyexceedsavailablemanpower…weneedanewapproach...”

32GovernmentProgramsevaluated;12inAI/computervision• AverageCost:$500k• AverageGPU:0• CommercialCloudCapacity:0• AverageSetofLabeledData:~10kobjects• Algorithms:Github•  IntegrationPlan:N/A

Partners:startups,bigcompanies,topuniversities,thinktanks,researchlabs

Technology AutomationValue

DataTransport(PEDCONUS) 1x

BetterUI/UX 2x

SpeechtoText 3x

SensorMgmt/Optimization 5x

ComputerVision/AI 50x

OCT2016 NOV DEC JAN2017 FEB APR

• Consolidateexistingalgorithmictechnologyefforts

• FieldAIforPEDofTUAV,mediumaltitude,andWAMIFMV

• Smallteam;movefast;fail;learn• Firstalgorithmsinoperationbyendofcalendaryear2017

Conclusions:• DeepLearningalgorithmscanperformatnear-humanlevels

• AIcanonlybedoneatscale;buildaPipeline

•  Integration&Cultureisthe“hardpart”

Partners:DODLabs,CCMDs,CSAs,DIUx,SCO,Services

Computer Vision

Pilot Program Early 2017

Ø USDI Warfighter Support directed to consolidate existing algorithmic technology efforts and advance AI/ML efforts across the Defense Intelligence Enterprise

Ø  Field AI for PED of TUAV, medium altitude, and Wide-Area Motion Imagery FMV

Ø  First algorithms in operation by end of calendar year 2017

Ø Prioritize the integration of similar technologies in other defense intelligence mission areas

TARGETSYSTEMANALYSISCAPTUREDENEMYMATERIALPLATFORMS&SENSORS

Metadata

1. Analyst sees vehicle enter area of interest & snaps a picture

2. Algorithm compares image of object to those previously observed

3. Algorithm displays lists of FMV collections with similar vehicles

1. Analyst draws a bounding box around objects of interest

2. Algorithm generates a count of all objects in the bounding box by class

- 73 Total Objects - 73x Personnel

1. Analyst asks algorithm for a count of objects in the area over time

= 2. Analyst receives graph output of object count by class over time

Reverse Image Search

Quick Count

Count Over Time

Downstream Analytics

1. Analyst selects moving object of interest on FMV and applies label

TerroristFinancier001 TerroristFinancier001MapTrack

2. Geolocations of object automatically recorded on analyst mapping tool (ARES, GOOGLE EARTH, ARC)

1. Multiple emergency vehicles appear on FMV

2. Analyst labels each of the vehicles & tags the event “Emergency”

FireTruck

Ambulance

3. Event tagged metadata teaches the algorithm to recognize events

Event Tagging

Downstream Analytics

Selective Track

Event = “Emergency”

“Breakthrough technology results from two distinct activities that generally require

different environments—invention and innovation. Invention is typically the work of

scientists and researchers in laboratories, like the transistor, developed at Bell

Laboratories in the 1940s. Innovation is an invention put to commercial use, like the

transistor radio, sold by Texas Instruments in the 1950s. Seldom do the two

activities occur successfully under the same roof. They tend to thrive in opposite

conditions….”

- Derek Thompson, “Google X and the Science of Radical Creativity”, The Atlantic

Maven Success Formula

Industry/academia mentors

Commitment from the top & top cover

Tenacious project lead

Talented/aggressive/enthusiastic action officers

Sufficient resources

Ground swell & buy-in from the field

Subject matter expertise +

+

+

+

+

+

Early & frequent user engagement +

DetectErrors

SendErrorReport

IdentifyData/Relabel

Retrain

ReintegrateBinaries

GPU Resources

Cloud Computing

Commercial Algorithms

Open Source Algorithms & Data

User Engagement

Metadata Removal Randomization Human Review

CTR Labeling (General)

GOV Labeling (Expert)

Job 1: Object Classification Job 2: Bounding Box Job 3: Refined Classification

Labeled Data

EO IR WAMI Still Build Test Accredit

DATA ACQUISITION, PROCESSING & LABELING ALGORITHM DEVELOPMENT ALGORITHM INTEGRATION

FIELDING & FINE TUNING

Labeled Data

DATA LABELING

Built to POR Reqs

Sensor Calibration

Software Mods Bench Test

Flight Test

Final Integration

Test IATT

IATO

ATO

PRO

GR

AM

PR

OTE

CTI

ON

AWCFT Pipeline

Algorithm Test, Validation

& Evaluation Model Accuracy

Initial Model

Improved Model

IOC Model

“The creation of a triangular relationship among

government, industry, and academia was, in its own way,

one of the significant innovations that helped produce the

technological revolution of the late twentieth century.”

- Walter Isaacson: The Innovators

“Our attitude matters, and not because we can stop the march of

technological progress even if we wanted to, but because our

perspective on disruption affects how well prepared for it we will be.

There is plenty of room between the utopian and dystopian visions of

the fully automated and artificially intelligent future we are heading into

at rapidly increasing speed. Each of us has a choice to make: to

embrace these new challenges, or to resist them. Will we help shape the

future and set the terms of our relationship with new technology or will

we let others force the terms on us?” - Garry Kasparov

Deep Thinking: Where Machine Intelligence Ends and Human Creativity Begins