ThisworkislicensedunderaCreativeCommonsAttribution-NonCommercial4.0InternationalLicense.

ArtificialIntelligenceandMachineLearningAppliedtoCybersecurityTheresultofanintensivethree-dayIEEEConfluence6-8October2017

Todownloadacopyofthepaperandtoprovideyourcomments/feedback,pleasevisit:https://www.ieee.org/about/industry/confluence/feedback.html

2

ExecutiveSummary

Inrecentyears,cybersecuritythreatshavechangedinthreeimportantways:

1. Theadversarialmotivationhaschanged.Earlyattackprogramswerewrittenasaresultofanindividual’scuriosity,morerecentattacksarewrittenbywell-fundedandtrainedmilitariesinsupportofcyberwarfareorbysophisticatedcriminalorganizations.

2. Thebreadthandspeedofattackadaptationhaveincreased.Whereasthefirstattacksexploitedsoftwareweaknessesfoundbyhand,werepropagatedusing“sneakernet,”andaffectedsinglecomputers,today’sattacksexploitweaknessesfoundautomatically;areautomaticallypropagatedovertheInternet,packagedevenbyunsophisticatedattackers;andaffectcomputers,tablets,smartphones,andotherdevicesacrosstheglobe.

3. Thepotentialimpactofanintrusionhasincreasedsubstantially.GloballyconnecteddevicesandpeoplemeanthatattacksaffectnotonlythedigitalworldasinthepastbutalsothephysicalworldthroughtheInternetofThings(IoT)andthesocialworldthroughubiquitoussocialmediaplatforms.

Ourentirecommunityneedstorespondanddevelopthetechnology,anddatastructures,andthelegal,ethical,legislative,andcorporategovernancemechanismsneededtosecureanenvironmentthatisincreasinglyundersiege.

Thegrowingsizeoftheattacksurfacepresentsbothathreatandanopportunity[1].Thethreatisthattherapidlyincreasingadoptionofconnecteddevicesequippedwithconventionalsecuritymeasureswillrenderhumansecuritypersonnelincapableofdefendingtheentiresystem.Thesheernumberofdevicesacrosstheglobemakesevenasmallpercentageoffailuresandcompromisesasignificantevent,beyondtheabilityofhumanoperatorstocopewith.Considerthatforapopulationof1billion(109)devices,a1percentvulnerabilityrepresents10milliondevices.Theopportunity,nearlyanecessity,isforsecurityartificialintelligence(AI)/machinelearning(ML)toactasaforcemultiplierbyaugmentingthecybersecurityworkforce’sabilitytodefendatscaleandspeed.

TheagilitycreatedbyAI/MLaugmentationofacybersecuritysystem(henceforth,“securityAI/ML”or“securityAI/MLsystem”)istwosided.Alongwitharapidresponsetobothdetectionandremediationcomesthepotentialforanequallyrapidcorruptionofsystems.Computersdowhattheydoreallyquickly,whichcanincludedoingthewrongthing.Itisessentialtokeepinmindthat,withtheincreasinguseofAI/ML,badactorsandentitieshaveAI/MLattheirdisposalaswell.

AI/MLsystemsarealreadyabletoidentifyanddevelopzero-dayexploits,apartoftheU.S.DefenseAdvancedResearchProjectsAgency(DARPA)2016CyberGrandChallenge.Whilethetechnologywasintendedtohelphumansmorerapidlyidentifyandfixvulnerablesystems,itisequallyeffectiveforadversarialuseinfindingandexploitingsystems.MalwareisalreadyusingAI/MLtodetectwhenitisbeingmonitoredwithina“securitysandbox,”andtoalteritsbehaviortoescapedetection.SuchastrategyissimilartoVolkswagen’seffort[2]toprogramaroundsandboxtestingofdieselemissions.Inbothcases,adeptcoderscreatedsystemscapableofbehavingininnocuouswayswheninasecuritysandboxbutinadifferent,malevolentwaywhenemployedinoperationalsystems.

Inthistrendpaper,we’lladdresssixdifferentdimensionsoftheintersectionofAI/MLwithcybersecurity.Theyare:legalandpolicyissues;humanfactors;data;hardware;softwareandalgorithms;andoperationalization.Theserecommendationsareintendedforindustry(I),academia(A),government(G),andstandardizationbodies(S).Inadditiontospecificrecommendationswithineachofthesesixdimensions,wemakethefollowingfivecross-cuttingrecommendations,indexedbydimensions(1–5)andtowhomtheyaretargeted(I,A,G,and/orS):

• Thefutureneedsofcybersecuritywillrequireaninterplayofadvancesintechnology(hardware,software,anddata),legalandhumanfactors,andmathematicallyverifiedtrust(1,2,3,4,and5)(I,A,andG).

• Itwillrequireconcertedbusinesseffortstocreateproductsacceptabletothemarket,certifiedbyestablishedregulatoryauthorities(1,2,4,and5)(IandG).

• IfhumansaretotrustAI/ML,AI/ML-fueledcybersecuritymustbebasedonstandardizedandauditedoperations(1and5)(IandS).

3

• Regulatorswillneedtoprotectresearchandoperationsandestablishinternationallyrecognizedcooperativeorganizations(1and2)(SandG).

• Data,models,andfaultwarehouseswillbeessentialfortrackingprogressanddocumentingthreats,defenses,andsolutions(3,4,and5)(S,I,andA).

Ourrecommendationscanbeappliedatdifferenttimehorizons.Operationalizationtakestheleastamountoftimeandcouldbeaccomplishedinnomorethantwoyears.Thisissimilarlytruefordataandsoftware.Addressinglegalandpolicyissuestakesmoretime,atleastseveralyears.Hardware,e.g.,newprocessorarchitectures,typicallytakesmorethanfiveyearstomaterialize.

TheSixDimensionsofIntersectionofAI/MLandCybersecurity

Inthefollowingsections,wedescribeindetailthesixmostimportantdimensionsrelatedtotheintersectionofAI/MLwithcybersecurity.Weidentifythreats,challenges,andopportunitiesandmakerecommendationsforeachdimension.

1. LegalandPolicyIssues:BuildingTrustthroughAccountabilityAI/MLaugmentationofcybersecuritysystemsmayseemahighlytechnicaltopicbestlefttoasmallgroupofexpertcomputerscientists.However,themostformidablechallengesforthefutureofAI/MLarelikelytobesocialinnature.WhileAI/MLpromisestoimprovesecuritybyautomatingsomeaspectsofdefense,cautionisneededforthecreation,deployment,anduseofthesesystems.Unlessdevelopedandusedverycarefully,Al/MLmayirretrievablydamagenationalsecurity,economicstability,andothersocialstructures.Assuch,itshouldnotbeviewedasapanacea,andoursocialstructures(andthehumanswhorelyonthem)mustbepreparedfortheinevitabilitythatthesystemswillfailinbothanticipatedandunanticipatedways.Safetynetsoflegalandethicalconstraintsareneeded.Buildingaworld—asocial,ethical,andlegalcontext—thatisreadyfortheincorporationofAI/MLmattersasmuchasthecreationofthetechnicalsystemsthemselves.

Despite(orperhapsbecauseof)theconsiderableenthusiasmforAIinmarketingcircles,themeaningofthetermisnowambiguousincommonparlance.ThisfactshouldactasawarningtoproceedwithcareasweenteranageofAI/ML.CreatorsandusersofAI/MLshouldnotbefinanciallyrewardedforshippingorimplementingcodeprematurelywithoutathoroughanalysisandtesting.Whileitiscommonforcompaniestoshipcodehavingknownerrorswithplanstocorrecttheseinafutureupdate,thismodelofshippingcodedoesnotworkforAI/ML.ThestakesofpossibleharmaresimplytoohighwithflawedAI/ML.In2016,theMiraibotnetheraldedthearrivalofanewcategoryofattack:distributeddenialofservice(DDoS)attackscarriedoutbybotnetsconsistingentirelyofvulnerableIoTconsumerdevices.Despitethesedeviceshavingrelativelylittlecomputingpower,MiraineverthelesssucceededinDDoStosomeofthebest-defendedwebsitesontheInternet[3].Now,imaginethescaleofdamageasophisticated,well-resourced,andsecurity-compromisedAI/MLmightcauseinthephysicalworld.

InsituationswhereadeveloperoroperatorlosescontrolofsecurityAI/MLandcausescatastrophicharm,thepublic’strustinAI/MLwillbeshaken.AstronglegalresponsewillbeneededtorebuildpublictrustinAI/ML.Uptonow,however,courtsandregulatorsinmostcountrieshavebeenslowtoassesslegalliabilityforharmarisingfromsoftwaremalfunction.ThisreticencewillneedtochangeinthecontextofAI/ML.Courtsandregulatorswillbewillingtoascribeliability—andperhapsevencriminalculpability—whencorporateassets,humans,andinfrastructurearephysicallyharmedbecauseofmalfunctionsorinadequatecareinthecreation,deployment,anduseofAI/ML.

WeshouldstartpreparingnowfortheAI/ML-causeddisastersthatwillinevitablyoccur.Hereare10thingswecandonow.

Software&Algorithms

HumanFactors

Hardware

LegalandPolicy

Ope

ratio

nalization

Data

Cybersecurity

ArtificialIntelligence/MachineLearning

4

1. Supportthecreationofenhancedregulatorystructures.Asstandardsstarttoemergefromthetechnicalcommunity,regulatorswillbegintoconstructaproactivesetofsharedminimumbaselinesforreasonableconduct—whatmightbecalled“floorsofcare”—forsecurityAI/ML.Wehaveseenthesefloorsofcarealreadyemergegenerallyincomputersecurityenforcement.WhileconsensusabouttheoptimaldesignofAI/MLsystemsmaystillbedeveloping,consensusaroundbasictypesofbuilding,implementation,andsecurityerrorslikelyexistsalready.Asetoftailoredregulatoryandenforcementmeasureswillbeneededtopreventlow-qualityorotherwiseflawedsecurityAI/MLfromdamagingthemarketforresponsiblebuildersandoperators.Forexample,intheUnitedStates,enforcementwilllikelyfallpartiallywithinthejurisdictionoftheFederalTradeCommission(FTC)underitsSection5FTCActauthority.However,inmostinstances,theFTCcurrentlydoesnothavestandalonerulemakingandfiningauthority.Assuch,thelegalevolutionofcertainregulators’authority(andbudgets)islikelynecessaryforbuildingtrustinandpolicingAI/ML.

2. Urgethecreationofadditionaltechnicalfeedbackloopsforregulators.Animportantsteptowardensuringsuitableregulatoryapproachesinvolvesconstructingformaltechnicalfeedbackloopsinsidenationallegislativeandregulatorybodies.IntheUnitedStates,forexample,CongressandregulatoryagencieswillserveasastartingpointformostAI/MLpolicy.However,Congresscurrentlylacksafundedofficeoftechnologyassessmenttooffertechnicalfeedback.Assuch,CongressshouldresurrectthistechnicalbodyastheOfficeofInformationTechnologyAssessment,withabudgetandstaffoftechnicalpractitionersknowledgeableaboutAI[4].

3. Urgestrongerlegalprotectionforsecurityresearch.AsingleuncaughtvulnerabilityinsecurityAI/MLmayresultinsignificantharm.Similarly,trainingAI/MLsystemswilldependontheavailabilityofhigh-qualitysecurityresearch.Whilerogueattackersrequireprosecution,legalsystemsshouldalsobecarefultofacilitate,ratherthanburden,thisessentialsecurityresearch.Forexample,intheUnitedStates,securityresearchersbothinsideandoutsidetheacademiarequireatleasttwocorrectivelegalbuffersassoonaspossible:1)congressionalcodificationofthesecurityresearchexemptiongrantedbytheLibrarianofCongresstoSection1201oftheDigitalMillenniumCopyrightAct[5]and2)anamendmentoftheComputerFraudandAbuseActtoprovideclarityarisingfromstatutoryambiguitiesregardingcomputerintrusion,absentdefinitionsofkeystatutoryterms,andjudicialdivisionsininterpretation[4].

4. Recognizethatinternationallegalandregulatoryharmonizationwillpresentchallenges.AstherecentnegotiationsoversecuritytoolsandtherestrictionsoftheWassenaarArrangement[6]havedemonstrated,coordinationandharmonizationofregimesacrossbordersandpolicyareaspresentformidableobstacles—andmayalsorequireyearsofnegotiations.DiscussionsconcerningAI/MLwillalsotriggeraneedtoreconcilevariouslegalframeworksfrompriorerasandacrossjurisdictions.Becauseofdifferentlegalapproachestoprivacy,security,andtortrecourseforconsumersinparticular,thecreatorsandoperatorsofAI/MLmayfindthemselvesthesubjectoflitigationininternationalforumswheretheirAI/MLhasallegedlycausedharm.Thecontractualchoiceofforumprovisionsandlimitationsofliabilitywillnotbeuniversallyenforced.However,anydiscussionofstatutorylimitationsofliabilityforAI/MLisprematureatthisjuncture:itwoulderodepublictrustinthesesystemsandcreatenegativeincentivesforunsafeconductbybuildersandusers.

5. DemandthatcriminalenforcersbewaryofsecurityAI/ML.AsthedefeatdevicesemployedbyVolkswageninitsdieselcarsremindus[7],computercodecanbeleverageddirectlyforpurposesofavoidingregulatoryrequirementsandfacilitatingcriminality.ThecurrentsetoftoolsavailableforidentifyingandprosecutingcrimesfacilitatedbyAI/MLmayrequirereassessment.Regulatorsshouldconsiderofferingavenuesforbothcorporateandgovernmentalwhistle-blowerstoreportdangerousAI/MLsystemsinamannershieldedfromlegalconsequences.Withouttheseopportunitiesforwhistle-blowing,dangerousAI/MLsystemsarelikelytoresultinavoidableandseverelevelsofharm,which,inturn,willresultinabreakdownoftrustinAI/MLasawhole.

6. RecognizethattheuseofsecurityAI/MLforcriminalenforcementhasthepotentialtoviolateindividualcivillibertiesguarantees.Asdescribedinthe“HumanFactors”section,AI/MLsystemsareonlyasgoodasthehuman-curatedtrainingdataandthestrategicchoicesmadefortrainingmethodologies.Eventhehighest-qualitysystemscanproducefalsepositiveresults.Particularlywhenfulltransparencyintotheirfunctionalityisabsent,AI/MLsystemsdonotmeetthelegalstandardsofindividualizedjusticebeyondareasonabledoubtforacriminaldefendant,andthisshouldnotbeusedinlieuofthoroughinvestigationsand/ortheindependentjudgmentofafinderoffact(ajudgeorjury).CreatingtrustinAI/MLrequires,firstandforemost,preservationoftraditionallegalbaselinesoflibertyandjusticeforcitizens.

5

7. Usethelegallagfortechnicalstandardscreation.Alagalwaysexistsbetweenprogresswithinthescientificcommunityandthecatchingupoflegalandregulatorymechanisms.However,thislagsometimesrepresentsapositivefeatureratherthanabug.ForsecurityAI/ML,thelegallagcreatesawindowofopportunityforbuilders,operators,andinternationalorganizationssuchastheIEEEtoinitiateapolicyconversationwithregulatorstocraftsharedminimumbaselines,orfloors,forreasonablecareoftheAI/ML.Inotherwords,theseconversationsshouldoccurinadvanceofanycatastrophicglobalAI/MLincidentthatwilllikelytriggerreactionaryandpotentiallyaggressiveregulation.

8. Correctcurrentlyimperfectsecurityindexingandreportingstructures.Ourcurrentsystemsofassessingvulnerabilityandissuingadvisoriessufferfromdeficienciesinscalabilityandaccuracy[8].Similarly,patchinganddisclosurepracticesvaryacrossentitiesinwaysthatsometimesplaceconsumers,nationalsecurity,andbusinesspartnersatavoidablerisk.InpreparationfortheexpeditedpaceofflawsthatsecurityAI/MLwilluncoverandreport,thesedeficienciesrequireimmediateremediation.Correctionswillnecessitate,amongotherthings,standardizingsecurityadvisoryformatstothegreatestextentpossibleandenforcingfailurestoaccuratelydisclose(andpatch)flawsinareasonablytimelymanner.Thesecorrectionstoexistingstructureswillpavethewayforthetypesoftransparencydisclosures(thelimitationsofselecteddatasources;training,strategy,andendoflifeplans;andotherkeycharacteristics)thatwilldistinguishvariousAI/MLsystemsfromoneanother.

9. Supporttherobustenforcementofsecuritybydesign.PreservingpublictrustinsecurityAI/MLrequiresthat,firstandforemost,thesesystemsbeconstructedassecurelyaspossiblefromthebeginning.Securitycannotberetrofitted;addingsecurityaftercodehasshippedinavulnerablestateinevitablyintroducesnewvulnerabilitiesandundesirablyincreasescomplexity.Forthisreason,thecurrentfocusonpost-breachenforcementshouldbereplacedwithafocusonsecuritybydesignandsecurityprocesses.Unfortunately,lessonsfrompastgenerationsofsecurityproductswarnthatsuchproductsaresometimesthemselvesvulnerable,placingtheirusersatgreatersecurityriskratherthanbetterdefendingthem[9].Insuchcircumstances,regulatoryenforcementactionshouldrequireimmediatecorrection:levyingfinesand/orrequiringtheremovalofunsafeAI/MLfromthemarketplace.

10. Engageindiscussionandthechoiceofethicaldesign.ThecreationofpolicyguidelinesaroundissuesofethicaldesignpresentsanopportunityfororganizationssuchastheIEEEtoexpandtheircurrentdiscussions.EngagingthebroadertechnologyandusercommunityinissuesofethicaldesignforsecurityAI/MLwillfacilitatepublictrustandnudgeimprovementsamongbuildersandoperators.Inthecaseofanindividualbuilder,arobuststrategyforlegalriskmitigationshouldinvolveworkingwithcounseltodocumentthecorporatedecision-makingprocessaroundethicaldesignchoices.Forexample,astheWannaCrywormdemonstrates[10],thepresenceofaremote(albeitaccidentalinthisinstance)“kill-switch”andconstanthumanoversightaretwostrategiesformitigatingharmwhencodemalfunctions.Ultimately,regulatorswillanalyzewhetherthereisproofofethicalandsaferdesignchoices.Suchdocumentedchoicesaimedatreducingrisktoinnocentthirdpartiesdemonstrateadegreeofcarethatislikelytomitigatefindingsofliability.

2. HumanFactors:BuildingTechnicalandHumanTrustIn1983,StanislavPetrov,aSovietofficer,helpedavertnuclearwar.PetrovhadbeenondutyattheSerpukhov-15secretcommandcenteroutsideMoscowwhentheattackdetectionalgorithmsrunninghissystemswarnedthattheUnitedStateshadlaunchedfiveintercontinentalballisticmissilesattheU.S.S.R.[11].Insteadofreportingthealarmtohissuperiors,Petrovpaused[11].Althoughheknewthatthealgorithmshadprocessedover4,000variables,hisyearsofexperience(andhisawarenessthatthesystemhadbeendeployedinahurriedmanner)counseledcaution[11];hedidnottrustthesystem[11].Deemingthenotificationafalsealarm,hechosenottoreportit.Later,forensicanalysisshowedthatPetrov’sdistrustofthesystemwaswellfounded.Thepredictivealgorithmshad,indeed,beenconfused.Thealarmhadbeenfalselytriggeredbythesun’sreflectionsfromclouds[11],adatainputthesystem’sprogrammershadapparentlynotadequatelyanticipated.

Whilenoteverysecuritysituationisasseriousasanuclearstandoff,thisincidentservesasanimportantreminderthatthefutureofsecurityAI/MLwillrelynotonlyontechnicaltrustbutalsoonhumantrust.Eventhebest-engineeredsystemscanfail.Thekeyquestion,then,becomeswhethertheywill“failwell,”thatis,inamannerthatpreserveshumans’trustinsecurityAI/MLandminimizesharm.Indeed,itwillbethesehumantrustfactorsintheoperationalizationofAI/MLsystemsthatwilldictatetheiradoptionrates.

6

1. Trust-buildingthroughtransparencyandpreemptiveriskassessment.Inthe1980s,inhistreatynegotiationswiththeSoviets,USPresidentRonaldReaganoftenquotedtheRussianproverb“trustbutverify.”It’sstillausefultouchstoneaswediscusssecurityAI/ML.TrustinAI/MLcanbesubstantiallybuttressedthroughthebuilders’transparencyindisclosingstrategicchoices,updatingprocesses,andprovidingcontingencyplanstoassisttheirsystemsin“failingwell.”a. Alltrainingdataarenotequal.Thefirstlayerofnecessarytransparencyinvolvesthehuman

processesforselectingthedatausedtotrainsecurityAI/ML.Associalscientistsandstatisticianshaveamplydemonstrated[12],theselectionofdatasetswillpotentiallysufferfromanumberofsamplingerrorsandbiases.Everytrainingsamplewillhaveacertaindegreeofsamplingerror,andthiserrorrequiresanalysisanddisclosuretoavoidcreatingafalsesenseofconfidenceinaparticulartrainingmethodology.Differenttrainingmethodologieswillvaryinsuccessbased,inpart,ontheextentofthissamplingerror.BuildersofAI/MLsystemsshouldalsodisclosetheextentofanyaffirmativestepstheyhavetakentoavoidsamplingbiasinselection.Inotherwords,theyshouldarticulatewhytheyareconfidentthatthesampleusedfortrainingdatais,infact,accuratelyrepresentativeoftheentirepopulationofreal-worlddeploymentsituationsthattheAI/MLsystemislikelytoencounter.Forexample,oneinfamousAI/MLtrainingfailureoccurredinMarch2016whenMicrosoftintroducedTay,an“AIchatbot”onTwitter.Withinoneday,Twitterusers“taught”TaytospoutracistandNazipropaganda[13],ahighlyundesirableoutcomefromMicrosoft’sperspective.Mostimportantly,aswitheveryrigorousscientificprocess,themeasurementandselectionprocesseswithrespecttotrainingdatashouldbereplicablebyindependentthirdparties.Replicablemeasurementprocesses,alongwithwhatsocialscientistscall“interraterreliability”checks,buildconfidenceandtrust.Itisthroughthislevelofrigor,planning,andtransparencythatbuilderscanreassurebothusersandpolicymakersthattheirsystemsarewellbuiltandthus,tothegreatestextentpossible,protectedagainstmalfunctioningincatastrophicways.

b. Buildingneedstobeaccomplishedwithattackersinmind.Aswithallcode,thequestionrelatedtoanAI/MLsecuritycompromiseis“when”andnot“if.”Yet,asdailyheadlinesaboutdatabreachesremindus,boththepublicandprivatesectorstillstrugglewithevenrudimentaryquestionsofsecurity,andlegalaccountabilityhasbeenslow.Adversarieswillattempttofoolsystemsasbuiltandtrytorepurposesystemsfortheirownnefariousinterests.BuildersandoperatorsofAI/MLsystemsmustrecognizeandplanforthisunfortunateandinevitablesecurityreality,preparingtechnicalincidentresponsecapabilitiesandcorporateprocessesformitigatingharmstothirdpartiescausedbycompromisedAI/ML.

c. Riskmanagementshouldleveragehumansintheloop—asafeature,notabug.Asecondtrust-buildingdisclosureinvolvesanhonestacknowledgmentofthelimitationsofsecurityAI/MLsystemsandtheirrisks.AlthoughsecurityAI/MLpresentsapotentiallygame-changingimprovementforextendingthecapacityofcomputersandhumanstojointlydefendagainstattackers,asPetrov’sstorycautions,malfunctionscarrysignificantrisksandpotentiallydevastatingconsequences.Inparticular,themoresensitivethedeploymentcontext,themoreimportantitbecomestoretainhumanoversightasapartofthedecisionloop.SomecontextsmayevenprovetoofragilefortheuseofAI/ML.Whenappropriate,properlydesignedandimplementedAI/MLcanleveragepreexistingandnewknowledgetoassistinmoreeffectivelysecuringsystemsataspeedandefficiencybeyondhumanabilities.However,theimplementationofAI/MLsystemsshouldnotbeviewedasanexcusetoeliminatehumansorlimittheexerciseofnecessarydiscretionandjudgment.Indeed,humansshouldremaintheultimatearbitersforalldecisionsthatmayhavepotentiallycatastrophicconsequences.

AstheDARPACyberGrandChallengeorganizersexplain[14],carefulplanningbeforehandwasrequiredtoconstrainthecompetitors’securityAI/MLsystemsandpredicttheirpossiblemalfunctions.AccuratelypredictingandavoidingharmconstituteadispositivecomponentofbuildingtrustinAI/MLsystemcapabilities.Similarly,thecompetitorsintheDARPAGrandChallengedemonstratedthat,evenwhentwosystemsappearontheirfacetobeparallelintheirfunctionalityandtrainingdata,thebuildersofeachhavemadedifferentkeystrategicbehavioraldeterminations.Theyhavealsopotentiallyemployeddissimilartrainingmethodologies[14].Consequently,individualAI/MLsystemswillbehavedifferently,eveninthesamedeploymentenvironmentandrelyingonthesametrainingdata.ThesestrategicchoicesbybuildersshouldsimilarlybedisclosedtogeneratetrustinAI/ML.Disclosurewillassistthemarket(and,later,legalenforcers)inmoreaccuratelyassessing

7

suitabilityforparticulardeploymentsandtheextentofcarethatwentintotheconstruction(orselection)ofparticularAI/MLsystems.

2. Trustbuildingthroughaccountability.ItisinevitablethatsomesecurityAI/MLwillmalfunction,justasPetrov’ssystemdid.Tomaintaintrustinlightofthisexpectedmalfunctionreality,buildersandoperatorsshouldstrivetoself-audit,third-partyaudit,andbuildsystemsthatfailsafelyinwaysthatlimitharm.However,somebuilders’attemptsatself-auditandcorrectionwillproveinadequate.Inthesecircumstances,preservingtrustinAI/MLwillnecessarilyleadtoregulation,enforcement,andlegallymandateddamagesrecoveriesbyharmedthirdparties.

WeofferthefollowingfourrecommendationstoassistwiththedevelopmentofhumantrustinsecurityAI/ML.

1. Participateinstandardsdevelopment.Becausestandardsusuallychaseindustryinnovation,werecommendthattheacademiccommunity,standards-settingorganizationssuchastheIEEE,buildersofAI/MLsystems,andregulatorsconvenestandardsmeetingsonanongoingbasistoarticulatetheminimumfloorsofcarerequiredinbuildingandoperatingsecurityAI/ML.ExamplesaretheIEEECybersecurityinitiative[15]andIEEEStandardforEthicallyAlignedDesign[16].a. Inparticular,thisgroupofinterdisciplinaryexpertsshouldissuerecommendationswithrespectto

dataselectionforsystemstraining,borrowingmethodologiesfromsocialscienceandstatisticsresearchregardingsamplingbiasanderrors.IdentifyingdatablindspotsandarticulatingfloorsofcareinaninterdisciplinarymannerwillbuildtrustinAIandsecurity.

b. Further,thisbodyofinterdisciplinaryexpertsshouldcontinueconveningonaregularbasistoengagewithevolvingpracticesinAIastheyareimplementedinindividualsecurityAI/MLsystems.Inparticular,theseexpertsshouldperformpostmortemanalysesofsystemsthatmalfunctionduetoidentifiabledesignorstrategicchoicesmadebytheircreatorsandusers.

2. Assistindemystification.Academia,industry,andregulatorsshouldeachindependentlyengagewiththedauntingprocessofpubliceducationtodemystifythebenefitsandlimitationsofsecurityAI/ML.Asoneimportantexample,thepubliccurrentlylacksasetofnarrativesthatrealisticallyassessesthefunctioningofAI/ML.Currentnarrativeseithererronthesideofunrealisticallyutopianvisionsordramaticallydystopianonesleadingto,forexample,theextinctionorenslavementofhumanity.

3. Regularlyperformrobustself-audit.BuildersandoperatorsofsecurityAI/MLshouldengagewithexistingstandardsofcare,suchasthosereflectedbyInternationalOrganizationforStandardization(ISO)standards[17],andanalyzetheirorganizationsfortheexistenceofrobustandrigorousself-auditandtechnicalgovernanceprocesses.EachAI/MLbuilderandoperatorshould,inparticular,ensurethatsecuritybydesignprinciplesareinplacethroughouttheorganization.BecauseofthesevereriskspresentedbymalfunctioningAI/ML,eachorganizationshouldmakecertainthatadesignatedethicsofficerisinplacewhohasexpertiseinbothAIandsecurity.Thisethicsofficershouldworkcloselywiththechiefinformationsecurityofficer(CISO),generalcounsel,andotherC-suiteexecutivestocraftmeaningfulaccountabilityprocessesthataccuratelyassessthelimitationsofAI/ML.

4. Regularlyperformrobustexternalaudit.Rampantdatabreachesandvulnerabilitiesremindusthatallcodecontainserrors.Inadditiontoself-auditmechanisms,third-partytechnicalauditsofferakeyverificationmethodforsecurityAI/MLsafety.Robustregulatoryenforcementpresentsanothernecessarytrust-preservingauditmechanismforthefutureofAI/ML.

3. Data:NewInformationFrontiersIn2014,theInternationalDataCorporationreportedthattheamountofdatawasdoublingeveryyearandwouldreach44zetabytes(44×1021bytes)by2020[18].Thisfigureincludesdatafromindividuals,devices,technicalnetworks,socialnetworks,andvariousapplications.AssecurityAI/MLrequireslargeanddiversedatasetsforeffectivetrainingandthenetworksthattheAI/MLwillbeappliedtoproducesignificantamountsofreal-timedata,itisclearthatdatarepresentacriticaldimension.

Tobeeffective,securityAI/MLalgorithmsmustbetrainedonlarge,diversetrainingdatasets.Assuch,theeffectivenessofthealgorithmsisdirectlyproportionaltothequantityandqualityofthedata.Whilelargetrainingdatasetsareoftenavailable,onechallengeisthecompletenessofthedata.Existingdevicesandnetworkswerenotoriginallydesignedwithinstrumentationandmeasurementasanintegralfeature;therefore,thedataavailablefromthesedevicesandnetworksarenotcapturingcriticalconditions.

8

Additionally,datasetsareoftenincompletebecauseindividualsandorganizationsareinfluencedbyliabilityandreputationalconcernsandwithholddataaboutpotentiallyembarrassingcybersecurityeventsthatcouldreducecustomerandinvestorconfidence.Consumerprivacyconcerns,governmentpoliciesandregulation,andprotectionofproprietaryinformationalsocontributetoincompletedatasets.

Relevancyandintegrityareadditionalfactorsassociatedwithdata.Whilesimulateddatasetsareconvenienttogenerate,theyareoftenartificialbecausetheydonotproperlyencapsulaterealityandthehumandimensionofadversarialactions.Additionally,tobeeffective,datasetsmustbecontinuallyupdatedsotheyincludethemostrecentevolutionofthreatresults.Datathatdonotincludethemostrecentattackdatacannotbeeffectiveagainstthoseattacks.DataintegrityaffectsboththeeffectivenessofandconfidenceinAI/ML.Datacollectiontechniques,bytheirverynature,oftenincludeunintendedhumanandtechnicalbiases.Understanding,documenting,andsharingthosebiasesareimportanttoensureAI/MLeffectivenessandoperation.DataintegrityalsoaffectshumanconfidenceinAI/ML.IftheAI/MLtrainingdatasetisincomplete,includesquestionablebiases,oris,ingeneral,notfullyunderstood,thenconfidenceintheentiresystemisdiminished.Preprocessingofthedatapriortousefortrainingcanalsoalterdataintegrityandreduceconfidence.

BeyondtheactualdatausedforthetrainingandoperationalemploymentofAI/MLincybersecurityapplications,storing,sharing,andensuringtheintegrityofthedataimpacttheeffectivenessofandconfidenceintherespectivesystems.Nocentralized,standardized,andqualifieddatawarehousesforcybersecuritydatacurrentlyexistthatallowbroadsharingacrossindustry,government,andacademia.

Becausedatainthecybersecuritydomaincontinuetogrowatanincreasingrate,itisimportanttoconsideralternativealgorithmicapproachesthatabstractthreatanomaliesfromthedataleveltohigher-levelensembleindicators.CharacterizingcommonattackpatternswillallowAI/MLmodelstofocusonfeaturesthatpredictoutcomes.Additionally,rarethreatevents,whilepotentiallydevastating,areoftenunderrepresentedinaprobabilisticmodelthatencompassesallthreats.Asaresult,thereisaneedwithintheAI/MLdevelopmentcommunitytodeviseafeature-engineeringapproach.ThiswillallowAI/MLsystemstoanalyzecommonattackpatterns,thengeneraterepresentativeattackscenarios,subsequentlyanalyzethosepatternstoidentifyvariations,andultimatelyupdateandimprovethealgorithms.

Itiswellknownwithinthemilitarythat,whileoperationsareplannedwithgreatprecision,theenemygetsavoteonthefinaloutcome.SecurityAI/MLmodelsarecomplex,andasophisticatedadversarycandeterminetheboundariesofthemodelandpotentiallyexploittheseboundaries.Thefundamentalchallengeisthatdetection-drivendatapotentiallycreateafalserepresentationofanattacklandscape,andthemodelsarethenupdatedtopreventonlyattackerswhoarewillinglyorunwittinglytransparent[19].Primarydatagatheredfromprofessionalattackersshowacompletelydifferentlandscapethanthecorrespondinglandscapeinferredfromdetections[20].

Weofferthefollowingrecommendationsforthedatadimension.

1. Thesponsorshipofdatawarehouses,withsupportfromanalysts,canmaintaindataqualityandfacilitatefeatureengineering.Governmentandindustryarebothcapableofprovidingfinancialsupportandleadershipforcoordinateddatamanagement.

2. Asponsoreddatawarehousingorganizationshoulddriveamovetowardinternationaldatastoragestandardstofacilitateinformationsharingacrossorganizations.Thesestandardsshouldbesufficientlyflexibletoevolveasthreats,models,andnetworkschangeovertheten-yearhorizon.

3. Ifwetrytoharmonizerulesandstandards,therecouldbearacetomeetthelowestcommondenominator.Thismaylosedatagranularity.Weshouldlookat“norms”inadditiontostandards.Weshouldalsousetheformatofdata,ormetadata,toensuretrustandinteroperabilityamongorganizationswithdifferentsecurityAI/ML.Governmentsshouldestablishregulations,rules,andnormsonnew,frontierdatasetsforsmartcities,smartcars,andtheIoT.Careshouldbetakenregardinghowdataarehandled:Canpersonaldatabecollected?Dovendorsgetaccess?

4. Academiashouldbeinvitedtoworkonaframework.Weneedcross-disciplinaryresearchacrossAI/ML,cybersecurity,datascience,human-factorcyberengineering,thesocialsciences,andtheworkoffuturists.Researchisrequiredforcontextualandinferentialdatacollection,usingdataformattingaswellassensorstocollectdata.

5. Economicincentivesshouldbeintroducedsothatsensorsareinplacetocollectdata.Governmentsatalllevelsshoulddecidehowtocollectandmakeuseoftheirdata.Informationshouldbecollectedthat

9

respectssetpolicies.Toleveragedataforcybersecurity,whilemaintainingprivacyandsecurityconsiderations,thecybersecuritycommunityshouldinvestigatesharingdatathroughtrustedthirdparties.

6. Mechanismstomeasuretheconfidencelevelofdata’srelevanceandaccuracyshouldbeestablished.Willthemarkethelpensuredataaccuracyandrelevance,especiallywhenpeoplearepayingforthedata?

7. Datacollectionandfeatureengineeringshouldfocusoncybersecurityattributesthathaveareasonablysmallprobabilityofbeingmanipulatedbybadactors.Oversamplingtechniquescanincreasethepresenceofthreatcasesinthetrainingset.Wherepositiveexamplesarerare,thenumberoffeaturesinanymodelshouldbelimited.

4. HardwareforAI/MLandCybersecurityNathanielFick,chiefexecutiveofficerofEndgame,hasstatedthat“theattackers’advantageisgettingeverstronger.Companieshavegrowingattacksurfacesdrivenbydeviceproliferation:theIoT,mobility,automationandAI,andinfrastructureasaservice(IaaS).Meanwhile,barrierstoentrytocreatinganddeployingsophisticatedcyberweaponscontinuetofall.”Thenetworkisnolongerdefinedbytheelectronicequipmentwithinthephysicalprotectionofbuildingsandcampuses.Today,thenetworkconsistsofhumanusersconnectedbymobiledevicesanywhereintheworldandautonomousdevicesbroadcastingsensorinformationfromremotelocations.ThisisaverylargeandvariedattacksurfacetomanageanddefendagainstadversariesdeployingsophisticatedcyberattacksaidedbyAIbots.Theproblemappearsimpossiblyhardtosolve,withmanyleadingCISOsadmittingthattheynolongerviewcyberattacksasaquestionofwhethertheywillbehacked,butratherwhen.

Hardwareisanintegralpartofthissolutioninthreeways.Thefirstisbyintegratingsecurityintohardwaredevicedesigns.Thesecondisbycreatinghardwarenetworkarchitecturesthatcanintelligentlymonitorthenetwork’ssecuritystate.ThethirdisbycreatinghardwarethatallowsAI/MLsystemstosolvemorecomplexproblemsbyeliminatingexistingcomputebarriers.

BecauseIoTandmobiledevicesusuallylackthecomputationalpowerneededtorunadvancedsecuritysoftware,securitymustbeembeddedwithinthehardwareofthedevicesthemselves.Thedevicesmustbecomethefrontlineofdefense,ortheywillbeusedtoenableattacks.ThiswasshownintheOctober2016DDoSattack[21],[22]inwhichmillionsofDVRsandwebcamswereconvertedintobotnetsbytheMiraimalwareandthenusedtolaunchacontinuousandmassivestreamoftrafficthatresultedinshuttingdownNetflixandothermajorwebsites.Theabilityexiststomitigatetheseattacksormakethemmoredifficultbyimplementinghardware-basedsecurityfeaturessuchasARM’sTrustZonetechnology[23],whichsupportssecureendpointsandadevicerootoftrust.ThesefeaturesareessentialforanAI-basedsystem,ifIoTdevicescanmanagetodefeatsimpleattacksandprovideanAIalgorithmnotonlytounderstandthecurrentstateofthenetworkbutalsotofindanddefendagainstanomalies.Inthehighlycompetitive,low-costenvironmentoftheIoT,itishardtoconvincedevicemanufacturerstocommitdesigntimeandresourcestoimplementingthesefeatures.ThisisclearlyshowninthecaseofMeltdownandSpectre,wheresimplesecurityfixescouldhavepreventedlarge-scalesecurityflaws;buttherewasnoincentiveforindustrytofindandimplementthosefixes.Governmentagencies,standardsorganizations,andconsumersmustactinconcerttodemandthatsecuritybeintegraltothesedevices;end-point/edgedevicesmustalsobestrengthenedtomakethemhardertocompromisebydeployingatleastpartsofanAI/MLsystemontheedgedevicesthemselves.

Amodeltofollowcouldbethe1890establishmentofUnderwritersLaboratory(UL)todevelopstandardsforelectricalwiringbecauseofthepotentialtocreatefires.TheNationalFireProtectionAssociationwasalsofoundedatthattimetoinitiatefirecodesandpromotelawsforfiresafety.Bothoftheseeffortshelpedtocreateademandforcertifiedequipment.Consumerswantedtobeassuredthatthedevicestheyboughtwouldnotbeadangertotheirhousesandfamilies.Alltheyhadtodowaslookforthe“UL”seal.This,inadditiontoproductsafetystandardslegallyimposedbyappropriateregulatorybodies,forcedmanufacturerstoaddsafetyfeaturestotheirproductstosellthem.AsimilarUL-likesealforsecurityisneeded.Unfortunately,despitehavingbeendiscussedforyearsandsomerecenteffortsbeingmade,theideahasneverbeenimplementedatscale.Weneedtotreatcyberincidentsinamannersimilartotraditionalsafetyincidents.

EffectivelyusingAI/MLtodefendagainstcyberattacksrequirestheabilitytomonitornetworksecurityhealth,assessthreatstothenetwork,andprovidesolutionstocyberanalyststodefeattheattack.Monitoringthe

10

networkandassessingthreatsrequireinformationintheformoftelemetry.NetworksshouldhaveimbeddedhardwaremonitorsthatcanbroadcastthestatusofdifferentdevicesinthenetworktoacentraldefenseAI/MLsystemandsodetectanddefeatthreatsbeforetheydamagethenetwork.Thechallengehereisthat,tocreatesuchasystem,thecomputerandnetworkarchitecturemustbedesignedwithsecurityinmind.Itisnotenoughtosimplyplacemonitorsintohardware;thoughtmustbegiventowhatinformationisneededandhowbesttodeploythemonitorstoensureadequatecoverageofthenetworkaswellasreal-timealertingofattacksastheyoccur—and,ofcourse,thesecurityofsuchasystem.TheNationalScienceFoundationandDARPAhavebeguninvestigatingwhatthisnext-generationnetworkwouldbe,butmoreneedstobedone.Industryandacademiamustalsostepupandexplorewhatthissystemwouldlooklikeandhowitwouldfunction.Thisresearchwillhelpusenormously,notjusttodeployanAI/MLsolutionbuttodeploytherightsolution.

Finally,today’scomputerarchitecturewasdesignedtodocomplexcalculationsonrelativelysmallamountsofdata.ThisarchitectureisnotsuitedtothetypeofcomputationsperformedbymodernAI/MLsystems.AI/MLalgorithmsfindclustersofdataorassociationstoconnectobservedinformationtogetherandsoprovidecontextfortheobservations.Thiscontextallowsthemachinetounderstandtheperceivedworldandmakedecisionsabouthowtorespondtowhatthesystemisobserving.Toaccomplishthis,AI/MLalgorithmsprocessalargeamountofdataandperformrelativelysimpleoperations(e.g.,matrixmultiplications)onthosedata.Thisisafundamentallydifferentprocessingparadigmfromwhatiscommontoday.Becauseofthisdisconnect,AIrequiresalargeamountofcomputinghardwaretodothetraining,therebyprecludingthereal-timethreatassessmentandresponserequiredbycybersecurityfornewthreats.Tosolvethisproblem,computerarchitectsneedtofundamentallychangetheirapproachtocomputing.Weneedtotakeamoredata-centricapproach,focusingonhowdataflowthroughaprocessor,andalessprocessor-centricapproach,whichfocusesonhowcomputationsaredone.Academia,fundedbygovernmentagenciesandindustry,canleadthewaybyexperimentingwithnewandnoveloutside-the-boxarchitectures.Innovativeapproachesaretheonlywaytoshakeupafieldthathasn’teffectivelychangedinthelast50years.Withoutanewarchitecture,AI/MLwillbeunabletosolvelarge-scaleproblemssuchasthoseinthecybersecurityapplication.

AI/MLcanalsobeusedtodesignbetterhardware.Itisdifficulttocreatehardwarethatfunctionspredictablyandsecurelybecausethoseattributestraditionallydependontheexperience,foresight,andknowledgeofhumandesigners.AIcanbeintegratedintocurrentdesigntools,likethoseproducedbyCadenceandMentorGraphics,insuchawaytofindcommondesignmistakesorerrorsearlyinthedevelopmentcycle.Thiswouldbeasignificantaidtothehumandesigners.AI/MLisabletoexploremorepossiblefailuremodesandcanlookforcomplexfailuremechanismsburiedinadesignthatwouldotherwisebemissed.Eliminatinghardwarefaultscangoalongwaytowardmakingthenetworksecurebecausehardwarefaultsanddesignerrorsareamongthemostreliabletargetsforexploits.Basedona2015studybyMITRE,2,800cyberattackscouldbetracedbacktosevenclassesofhardwarebugs.EliminatingthesebugsusingAI/MLinthedesignprocesswillcloseseveralattackavenuesusedbyhackers.Theelectronicdesignautomationcommunitywillneedtoinvestindevelopingthesetools,andtheiruserswillhavetoprovidefaultinformationsothatanAI/MLsystemcanlearnfromthosemistakes.Thiseffortshouldbemostlyindustryfocused,withthegovernmentplayingasupportingroleinencouragingthedevelopmentofthesesystems.

Wemakethefollowingrecommendationsregardinghardware.

1. Investinginnewmemoriesandinterconnectswillmoreefficientlyprocesslargedata.Currently,anywherebetween40and96%oftime/energyisspentmovingdataaround,andbetween4and60%isspentprocessing[24],[25].

2. Solvingimportant,real-worldproblemswillrequiremanymoregraphicsprocessingunits(GPUs),centralprocessingunits(CPUs),application-specificintegratedcircuits(ASICs),andfield-programmablegatearrays(FPGAs)thanarepractical.Improvingdatamovement(see1,above)willenablenewAIalgorithms.

3. TheIoTneedssecuritystandards,developedbyastandardsbodysuchastheNationalInstituteofStandardsandTechnologyortheIEEE.Anotherorganization(akintotheUL)andregulatorsshouldenforceadoption.

4. Educatingthepublicaboutthevalueofcertificationandcreatingamarketfunctiontoforcehardwaremanufacturerstoincorporatesecurity,includinginacceleratorssuchasGPUs,FPGAs,andtensorprocessingunits(TPUs),areessential.

5. Academia,industry,andgovernmentshoulddevelopamethodologyforbuildingasecurehardware(wemightcallit“designforsecurity”).

11

6. Industryshouldestablishanaffordablemeansforsecuritytestingandcertification.Today,suchlaboratoriesaresoexpensivethatmostcompaniesdonotusethem.

7. Securitymiddlewaretomonitorasystemandissuealertsusingcurrenthardwaremonitorsshouldbedevelopedalongwithnewonestodeterminesystemsecurity.

8. Expertsshoulddevisecertificationsenablingmanufacturerstoregardsecurityasacontributortoprofitsandallowingconsumerstodifferentiateintheirpurchasingbehaviorbasedonsecurityrobustness.

9. Universitiesshouldincorporatesecurityintothehardwaredevelopmentcurriculumofsystemdesigncoursesandincludehardwareintocyberanalysts’andprogrammers’training.

5. SoftwareAndAlgorithmsforAI/MLandCybersecurityCounteringcybersecurityattacksinacompletelyautonomousway,usingsophisticatedAI/MLalgorithmsandwithouthumansupervision,isbothappealingandcontroversial.SecurityAI/MLsoftwareobservessystemusage,estimatinginrealtimewhetherthereisathreat.ToenableMLsystemstoconstructadetailedmodelofascenario,developersarechallengedtoquicklyunderstandnormalandthreateningscenariosandtheirassociatedfeaturespaceatahighlevel.Fivebasicprincipleshaveguidedthisanalysisofhowcorporations,governmentagencies,andotherinstitutionsshouldbestdeployAI/MLsoftwareandalgorithmstoaddressgrowingcybersecuritythreats.

1. Forbothtechnologicalandpolicyreasons,acompletelyautonomoussystemfordetectingandrespondingtothreatsisnotalwaysanappropriateoption.Balancingthebenefitofhumanversusmachine—giventhattheybothmakemistakes—shouldbeusedtodecidewhoorwhatmakesthedecision.

2. TheunderlyingtechnologiesofcybersecurityandAI/MLareevolvingrapidly;therefore,anadaptableAI/MLframeworkmustbedeveloped.Focusingonaspecificmethodologyoralgorithm,suchasdeeplearning,wouldbeunwisebecausedevelopmentsinafewyearsarelikelytosupersedeit.Forthesamereason,thesearchforasingle“proven”cybersecuritymodelisachimera.

3. AI/MLapproachestocybersecuritymustbeproblemspecific.Asuccessfulapproachwillfeaturemorethanonemodel,operatinginsequence,inanyconceivablecircumstance.

4. AI/MLmodelsforcybersecuritywillbeappliedintwophases.Thefirstphasewillinvolvedevelopinganunderstandingofthenormalhistoricallandscapeofnetworkdatatraffic,extractingactionableinsightsaboutthreats,andlearningtoidentifyanomaliesinnetworktraffic.Thesecondphasewillconsistofapplyinganunderstandingof“normal”toidentifyanomaloussituationsrequiringhumaninteractionandactionagainstknownthreatprofiles.

5. AI/MLforcybersecurityissimilarinnaturetotheapplicationofAI/MLforfraud:bothareadversarialandongoing.Ineithercase,perpetratorswillmodifytheirbehaviorwhentheiractionsaredetectedandthwarted,necessitatingconstantlyevolvingcountermeasures.

Becausetypicalcybersecuritydatasetsareextremelylarge,networksfordatadeliveryandtheprocessingofMLmodelsmustbecapableofefficientlyhandlingstaggeringamountsofdiversedata.Thescarcityofsuchnetworkstodayisamajorhindrancetoprogressinthefield.Achievingsuchnetworksforreal-timeanalyticsrequiresevenmorecarefulsoftwaredesignandalgorithms.

Additionally,AI/MLcanbeappliedtocybernetworksineitheraproactiveorapassive(forensic)way.Thisdistinctionmeritsexplicitinclusioninplanninganddesign.Proactivemodelsleverageinsightsgainedfromhistoricalanalysistocontinuallymonitornetworkactivityagainstknownindicatorsofattackpatterns.Asanewinputarrives,itiscomparedtoallknownpatternsofattack.Asknowledgeofthesepatternsdeepens(afunctionofboththedataandananalysisofhistoricalinformation),amoreaggressiveapproachforreactingtosuspiciousactivitycanbeemployed.

Incontrast,passivemodelscollectsufficientdatatoenabletheposthocanalysisofattacksthatwereunanticipatedinkind.Thisallowsanorganizationtouseatipfromanotherdomaintolearnabouthowanattackwascarriedoutandpossiblyalsotobeabletoattributetheattacktoaspecificoperator.Collecteddatashouldincludethosethatprovidebroadvisibilityintoenterpriseactivities,asawaytounderstandhowmalicioussoftwarecanspread,aswellasdeepvisibilityintospecificsystemactivities,asawaytounderstandhowmalicioussoftwareexecuteditsattacks.Thefirstusuallyrequirescapturingnetworkactivity,whilethesecondusuallyrequirescapturingsystemactivityoneachsystem.

12

Naturallanguageprocessing(NLP)makesitpossibletoderiveactionableinsightsfrompreviouslyinaccessibledata.AnalyzingunstructuredtextwithNLPenablestheextractionofkeyactorsfrompastcyberincidents,newsstories,analysisreports,andmanyothersimilartextsources.KnowledgeGraphtechnologyenablesthediscoveryofnonobvioussecondaryandtertiaryrelationshipsbyconnectingindividualnodesandalsoprovidesinsightsintosequencesofevents.Itispossibletodeepenourunderstandingofthecyberlandscapetoidentifyprecursorstothreatsandmorereadilydeterminedeviationsthatcouldindicatehazards.

Cybersecurityishighlydynamicbecausetheunderlyingtechnologiesareevolvingrapidly,andtheoffenseanddefensearelockedinathreat–response–threatcoevolution.Thisdynamicandconstantlyevolvinglandscaperequiresconstantvigilanceandupdatestothreatclassification,identification,andresponse.

Finally,theadversarialnatureofthecyberdomainpresentsamodelingchallengethatisalsoanopportunity.Cybercompetitions,inwhichteamsactandreacttoothers,arevaluablelaboratoriestoexploreinteractions.Thegoaloftheseexperimentsistoimitateprocessesbywhichanadversarylearnsofdefensivemeasuresandthenpreemptsevasivemeasures.Understandinganadversary’sstrategy,then,helpsrefinethemodels.

Wemakethefollowingrecommendationsregardingsoftwareandalgorithms.

1. MLshouldbeusedasatooltoenhanceandextendhumancognition.Ifmodelsreducetheburdensofroutineactivityandidentifypotentiallyriskyactivity,theprobabilityofthreatavoidanceincreases.MLshowssignificantpromiseinsupportofforensics,intrusiondetection,andattackresponse.

2. Academic,industry,andgovernmentpartnershipsshoulddevelopgame-theoreticmodelsforadeeperunderstandingofthemotivationsandbehaviorsofthreatactors.

3. Everyappropriateformofdatashouldbeaggressivelyleveraged.NLPtechniquescanbeusedtoextractartifactsfromunstructureddata,andKnowledgeGraphtechnologycanbeleveragedtoidentifynonobviousrelationshipsbetweenentitieswhilerecognizingthedatasamplingconcernssetforthinthe“HumanFactors”sectionofthistrendpaper.Thesewillidentifyprecursorstothreatincidentsandsupportautomaticdetectionofnefariousactivity.

4. Systemsshouldbearchitectedaroundtheuncertaintyofcyberdefense.Lessfocusshouldbegiventospecificthreatindicators(oftenunknowable)andmoretounderstandingwhatisdifferentoranomalous.Thisrequiresadeeperunderstandingofwhat“normal”lookslike,sounusualindicationscanbedetectedmorerapidlyandwithgreaterfidelity.

5. MLmodelsarenotstatic;theymustadaptasthreatsdevelop.Tokeeppacewithdevelopingthreats,asystemrequirestheattentionofMLscientists.MLsystemsneedaready-madedevelopmentenvironment,witheasydataaccess,tofacilitateexperimentswithfeaturesetsandfunctionalforms.Itmustbesimpletopushmodelsintoproduction.

6. Academic,industry,andgovernmentpartnershipsmustfostercooperationonmodelingadvancesforparticularcyberchallenges.Governmentandindustryorganizationsshouldfundacademicresearchandprovidesufficientguidanceonspecificproblemsrequiringcreativetechnicalapproaches.Similarly,governmentandindustryshouldencouragedatasharing,somodelscanbetrainedwiththemostcomprehensivedatapossible.

7. MLfocusesonstatisticallybasedmethodologies,butthesearenotalwaysappropriateforunderstandingthedynamicsofanadversarialsystem,asincybersecurity,wherethreatactorsmodifybehaviorwhenitbecomesineffective.

8. Modelsmustadaptquicklytodynamicthreats.Complexmodelsthattakeweekstomodify,train,andpushtoproductionwillbetoobrittletoprovideadequateprotection.Hybridtechniquesthatenablequickchangesthatprotectagainstrisingthreatscouldaugmentrobust,carefullytrainedsystems.

9. TheeffectiveimplementationofanML-basedcyberstrategyrequirescloseintegrationofdiverseexpertise.CyberandMLexpertsmustcollaboratetounderstandthenatureofthreats,soimplicituncertaintiescanbeexplicitlymodeled.Fieldleadersmustfindwaystoincreaseprofessionalcollaboration.

6. Operationalization:PuttingItAllTogetherTheworldhasfiniteresourcestodedicatetoimprovingcybersecurity,afactthatwillinevitablyleadtoissuesofresourceallocation.ImagineafuturemeetingtocreateanindustryorgovernmentroadmapforresearchandthedevelopmentofsecurityAI/ML.WebelievetheparticipantswouldagreethatproperlydevelopedanddeployedAI/MLwouldbehighlydesirabletogivethegoodguysatthemeetinganadvantageoverbadactors.

13

Buttherewouldbedisagreementoverwhichgoodguy’sbusinessmodelneedsprotectionfirst—orwhichnation’slawsshouldprovidethetemplateforcybersecuritylawandpolicy.

Thecounterpointtothegrowingsizeofthecyberphysicalattacksurfaceisthatitsgrowthrepresentsenormousopportunities.Throughhardwareimprovementsandproliferation,overthecomingdecade,organizationswillbeabletointegrateAI/MLintocyberspaceoperationsinwaystheywouldnothaveanticipatedevenfiveyearsago.AI/MLwillhelpcreateintegratedmeaningfromhundredsandthousandsofdisparatedatastreams;supportautomated,real-timepreventionplatforms;andaugmenthumans’decision-makingability.

SubstantialopportunitiesexistfordetermininghowhumanslearntotrustAI/MLsystemsandtheentitiesthatuseAI/ML.Thelogicalextensionofsuchresearchistoexaminehowhumans(oncetheyhavelearnedtotrusttheoutputsoftheAI/MLsystemstheyinteractwith)copewithviolationsofthattrust—suchasincorrectoutputs,lostdata,dataaggregationacrosssystemsthatviolateprivacyexpectations,andadversarialmanipulationoflearningstrategiestopoison“trusted”systems.Thisknowledgewillultimatelybecometherulesoftheroadforalong-termcyber-enabledsociety.Thereisacallforcollaborationamongresearchersinfieldsofpersonalandorganizationaltrustandthedesigners,developers,andtrainersofAI/MLsystems.

Asdiscussedinthesection“HumanFactors,”trustinthetechnologymayrequiresubstantialfinancialsupportandattentionbykeydecisionmakers.Asitevolves,AI/MLismorelikelytoreachconclusionsorperformactionsthathumansdonotfullyunderstandorthatdifferfromtheresultsoftypicalhumanjudgment.Handledpoorly,recommendationsoractionsbyAI/MLincreasetheprobabilitythattheAI/MLindustrywillrecreate,ratherthanlearnfrom,experiencessuchasthenuclearpowerindustry’shandlingofnuclearplantaccidents.

Securityfatigueislikelytobeachallengeuniquetoeachindustrysegment.ProbabilisticAI/MLsystemswillneedtolearnwhileavoidingmisclassificationintermsoffrequencyorseverity(intheeyesoftheuser,notthesecurityspecialist)thatcouldleadtodistrustanddisbelief—electronicversionsoftheboywhocriedwolf,inasense.Thepunishmentinthestorywasthattheboywaseaten;theoutcomeinthisdiscussioncouldbereducedbusinessgrowthduetogeneraldistrustofcomputertechnology.

Itiseasytoforgettheconsumingpublicwhileindustrysectorsvieforleadershipincybersecurityorotheraspectsofcomputing.TherewillbenewandtraditionalchallengestotheintegrationofAIandMLintocybersecurity.Repairingormitigatingvulnerabilitieswillremainachallenge.Mostuserseitherdonotknowordonothaveawaytoreportdiscoveredvulnerabilities.Inotherinstances,involvementinandadditionalautomationofrepairingmightberejectedbyorganizationsunabletoacceptmuchdeviationincompatibilityandperformance.

WhilethesolidpoliticalsupportofsmallbusinessessuggeststheywillhaveaplaceatthesecurityAI/MLtable,smallbusinessesmaybedisadvantagedbyalackofdatasetsorresourcestocollectsuchsets.ThispresentsopportunitiesforlargerorganizationstoproductizelargerAI/MLsolutionsorforneworganizationstostepintothemarketplacewithmeaningfulandusefuldatasets.

Currentusecases,suchasfrauddetectioninthebankingindustryanddiagnosisinthehealth-careindustry,serveasenablersforthefutureoperationalizationofAI/MLinthecybersecuritydomain.AlthoughnotallusecasesandcurrentAI/MLalgorithmsaredesignedtobeemployedinreal-timeenvironments,theyserveasfoundationsforreal-timedetect–defendordefend–attacksituationsincybersecurity.Forcertaindomains,theabilitytoconsciouslydisableAI/MLactionsordisregardrecommendationsisanenablerofAI/MLoperationalizationforcybersecurity.Insuchcases,itisimportanttohavetheabilitytodisableoralterspecificsystemaspectswithoutnecessarilyturningeverythingoffwhile,atthesametime,comprehendinganyrepercussions.

WhileunderstandingandtrustmaygrowonasocietalleveltoeventuallyallowAI/MLtomakeresponsedecisions,humansmustalwayshaveawaytovetothosedecisions,particularlywhenpreplannedfail-safesfail.However,inmanyothersituations,havingAI/MLrunclosedloopwillbefine(perhapsevenpreferable)—butnotalways.Clearcategorizationisrequiredtodeterminewhenahumanshouldbeintheloopversuswhennot.Forexample,safetyfavorsahumanintheloop,whilelimitationsinscalinghumans’abilitytoarbitratefavorsautomation.

DuelingsecurityAIsystemsisanarearipeforlong-termresearch,associetywilleventuallyneedtoconfrontthefullpotentialofAI.GooglerecentlyannouncedthatAlphaGoZerolearnedhowtobeatAlphaGowithout

14

humantraining.Althoughclearlyconstrainedtoawell-structured(thoughexceedinglylarge)universe,thetrendlinefromcomputersbeatinghumanstocomputersbeatingothercomputerswillsteepen,notflatten.AsAI/MLsystemsgainexpertiseinconducting,orhelpingconduct,cyberspaceoperations,therewillcomeatimewhenAI/MLwillfaceAI/ML.Learninghowtorecognizethesituation,establishinghowtooff-ramporescapethesituation,anddetermininghowandwhen(orevenif)toinvokehumanexpertiseareallfieldsofresearchthatmustbeexplored—iffornootherreasonthanknowingbadactorswillbeusingAI/MLtohelpthemachievetheirownobjectives.

Allindustrysectorstogetherhaveacommoninterestinmanagingthecybersecurityworkforceasitgrowsandchangesitsskillmix,drivenbytheever-increasingpresenceofAI/ML.Thereishistoricalprecedentforworkforceevolutionintheautomotiveindustry.Attheindustry’sbeginning,littleeffortwasrequiredtolearnhowtomaintainandoperateanautomobile.AI/MLusageanddevelopingtrustwillnotrequireextensivegroundinginthetheoryandfundamentalsofAI;drivingamoderncardoesnotrequiretheoperatortoknowtheintricaciesoftheignitionsystem.However,theAI/MLindustrymustbecomebetteratmaintainingandretainingskilledlabortodesign,build,operate,maintain,anddefendAI/MLsystems.

ThesupportingpartnersfortheoperationalizationofAI/MLincybersecurityaregovernments,industry,academia,andtheconsumingpublic.Atthecore,industrypartnershipswithacademiawillbethestrongestwaytobringtheresearch-drivenAI/MLcapabilitiestooperationaluseincybersecurity.Atthegovernmentlevel,researchfundingforacademiaandincentivesforindustrytoparticipatewithacademiaarerequired.Industry,intheformofconsortia,canfacilitateworkshopsandthecreationofstandards,ascross-companybodiesplayingaspecificroleintermsofarticulatingproblemscanhelpilluminatetheriskandsharebestpractices.StandardsorganizationsandconsortiumsliketheIEEEhavearoleinestablishingcommonbusinesspractices.Suchstandardizationmustriseabovethetendencytoseekasealofapprovalorachecklistofminimalbehaviorsthatareassessedonceandthenforgotten.

Wemakethefollowingrecommendationsonoperationalization.

1. DemonstratethecompellingcasethatAI/MLsystemsembeddedwithincyberspaceoperationsmakeoperationsbetteralongmultipledimensions,e.g.,speedtopatch,remediatingamaliciousevent(orevents),increasingup-timeinsystemsofinterest,decreasingthenumberofincidentsandthenumberoffalsepositives,increasingaction–reaction–counteractiontimecycles,anddecreasingunintendedconsequencesofcybersecurityoperationsdecisionsandactions.

2. Retainhumanandorganizationalresponsibilityfordecisionsmadebytheorganization’shumansandsystems.Disclaimingresponsibilityfororganizationalactions(orinactions)becausetheAI/MLinfluencedormadeadecisionisunwiseandwillcontributetopublicandregulatorybacklash.

3. Togainthetrustofthosehumansresponsibleforcybersecurityoperations,AI/MLsystemsandtheirmakersmustpreparetobetransparentabouttheprocessesbywhichtheirsystemsaretrainedandtested,evolve(atboththeoperatingsystem/applicationlevelsandthedataprocessing/recommendationlevels),makedecisions,receiveandprocessfeedbackforimprovement,andprovideindicatorsandwarningsofbeingunderattack(fastandovertaswellasslowandsubtledatapoisoning).

4. RigorousacademicandindustryreviewofthoughtleadershiponAI/MLtopicsincybersecurityisneededtoaddressthelackofvettingandopennessofpractitionerinfluence.Interdisciplinaryreviewmaybeappliedpriortopublication,gettingthecorrectinformationout.First,publicationsmayresultininaccuracies.Industryshouldbeaskedtofundinterdisciplinarypolicychairsatleadinguniversitiestoconnectresearchfromindustryandacademia.Weareatthedawnofpublicationsinthisfield,andaseriousshortageofinterdisciplinaryAIpolicyscholarsexists.

5. TheevolutionoftheworkforcemustbesupportedbyencouraginguniversitycurriculainAI/ML,withspecificcoverageofsecurity,suchthatfuturedesignersandoperatorsgainamutualunderstandingofthelimitationsandrisks.

Summary

AI/MLwillbecomeoneofthekeycomponentsofnext-generationsecurity,enablingelevateddegreesofcybersecurity.Atthesametime,AI/MLcanbecomeathreatusedbyattackers.Inthistrendpaper,weaddressedsixdifferentdimensionsrelatedtotheintersectionofAI/MLwithcybersecurity:legalandpolicy

15

issues;humanfactors;data;hardware;softwareandalgorithms;andoperationalization.Asnotedearlier,theserecommendationsareintendedforindustry(I),academia(A),government(G),andstandardizationbodies(S).Inadditiontospecificrecommendationswithineachofthesesixdimensions,wemakethefollowingfivecross-cuttingrecommendations,indexedbydimensions(1–5)andtowhomtheyaretargeted(I,A,G,orS):• Thefutureneedsofcybersecuritywillrequireaninterplayofadvancesintechnology(hardware,

software,data),legalandhumanfactors,andmathematicallyverifiedtrust(1,2,3,4,and5)(I,A,andG).• Itwillrequireconcertedbusinesseffortstoestablishmarket-acceptedproducts,certifiedbyestablished

regulatoryauthorities(1,2,4,and5)(IandG).• AI/ML-fueledcybersecuritymustbebasedonstandardizedandauditedoperationsifhumansaretotrust

AI/ML(1and5)(IandS).• Regulatorswillneedtoprotectresearchandoperationsandestablishinternationallyrecognized

cooperativeorganizations(1and2)(SandG).• Data,models,andfaultwarehouseswillbeessentialfortrackingprogressanddocumentingthreats,

defenses,andsolutions(3,4,and5)(S,I,andA).

Ourrecommendationscanbeappliedatdifferenttimehorizons.Operationalizationtakestheleasttimeandcouldbeaccomplishedinundertwoyears.Thisissimilarlytruefordataandsoftware.Legalandpolicyissuestakelonger,uptofiveyears.Hardware,e.g.,newprocessorarchitectures,typicallytakesmorethanfiveyearstomaterialize.ItwillbeessentialtocontinueevaluatingandadvancingcontributionsofAI/MLtocybersecuritythroughfocusedeffortsofgovernments,industry,andacademia.

Afterword:Background,Motivation,andOverview

TheIEEEhasarichanddistinguishedheritagedatingbacktotheAmericanInstituteofElectricalEngineers,foundedin1884,andtheInstituteofRadioEngineers,foundedin1912.NotableearlypresidentsoftheIEEEanditsfoundingorganizationswereengineersandpractitioners,includingAlexanderGrahamBell,CharlesProteusSteinmetz,RobertH.Marriott,WilliamR.Hewlett,andIvanGetting.Overthedecades,IEEEmembershiphasfundamentallychanged,withthoseworkinginindustryincreasinglyoutnumberedbyacademics.Andthistrendcontinues,withthenumberofIEEEMemberswhoidentifyindustryastheiremployercontinuingtodecline.Since2000,thepercentageofIEEEMembersfromindustryhasfallenfromroughly60%to39%.Ourcontenthasdiminishingrelevancetoindustrybecauseitisprogressivelymoreacademicinnature.Ourcareerdevelopmenteffortsarenotoptimallyalignedwithemergingindustryneeds.

Overthepastseveralyears,theIEEEleadershiphastakengreatstridestoengagewithindustryandmountedaconcertedefforttoprovidetechnicalprofessionalswiththetoolsandinformationtheyneedtoexcel.WehaveaggressivelyengagedwithindustrytounderstanditsneedsalongwiththoseofMemberswhoworkinindustryandsobringforthproductsandservicesofvalueandimportance.In2015,wemetwithover175industryleadersfrom45companiesinChina,Germany,Japan,andSiliconValleyintheUnitedStates.In2016,wemetwithover270leadersfrom70companiesinCanada,China,India,Israel,Japan,Singapore,SouthAfrica,SouthKorea,Taiwan,theUnitedKingdom,theUnitedStates,andUruguay.Thesediscussionsprovidedimportantinsightsintoindustryneeds.Onerecurringthemeheardfromawidevarietyofdifferentindustrieswastheimportanceoftechnologytrendpapersandroadmaps.Asaresultofthisinput,werespondedbycharteringtwotrendpapersin2016,oneon5Gandasecondonsmartcities.Thesetwotrendpapersweredeliveredinthethirdandfourthquartersof2017,respectively.Whilethecontentofthesetrendpaperswasvaluable,themorethan12-monthdeliverytimewascontrarytoindustry’sneedforrapidandrelevantinformation.Tomorequicklydelivercontemporaryandrelevanttrendpapers,weconsideredanalternativemodel.

InpartnershipwithSyntegrity,agrouphavingalong-standingrelationshipwiththeIEEE,weconceivedtheideaofbringingtogetheragroupofexpertsinatechnologyverticalandusingtheSyntegrationprocesstorapidlydevelopatechnologytrendpaper.Aftercarefulconsiderationofthetechnologylandscapeandthoseareaswiththegreatestinterestandimpact,wechosetheintersectionofAIandMLasappliedtothebroadfieldofcybersecurity.Inthiscontext,cybersecurityencompassesthefinancialservices,criticalinfrastructure

16

(e.g.,smartgridandSCADA[supervisorycontrolanddataacquisition]),anddefensesectors.Syntegritycombinedinsightsfromgeometry,neurology,andcyberneticswithadvancedmathematicalmodelsandsocialtechnologiesintheSyntegrationprocess,whichenablesgroupinteractiontoconsolidatethinkingandultimatelyformulatesolutionsindramaticallycompressedtimeframes.

On 6–8October 2017,we convened 19 experts from theAI,ML, and cybersecurity sectors in Philadelphia,Pennsylvania,UnitedStates,foratwo-and-a-half-daycollaborativesessionfocusedonthefollowingcomplexquestion:

Given the rapid evolutionofAI/ML technologies and the enormous challengesweall facewithrespecttocybersecurity,whatisneededfromAI/ML,wherecanitbebestapplied,andwhatmustbedoneoverthenexttenyears?

Duringthefirstday,thegroup,asawhole,identifiedchallengesassociatedwiththequestion,proposedmultipletopicsfordiscussionthatcouldpotentiallyaddressthequestion,andthencollectivelyprioritizedsixspecifictopicsthegroupbelievesmustbeaddressedtoanswerthequestion.Overtheremainingtwodays,thegroupconductediterativeandfocuseddiscussionsregardingeachofthesixtopicstoreachamorerefinedunderstandingofthechallengesandidentifythemostviablesolutions.Bytheendoftheconfluence,thegroupproducedadraftofthistrendpaperthatwillbesharedwiththegreatercommunitytoaddressthechallengesassociatedwiththequestion.

References

[1] B.D.Johnson.(2017,Mar.).Awideningattackplain.[Online].Available:http://threatcasting.com/wp-content/uploads/2017/03/A-Widening-Attack-Plain.pdf

[2] BBCNews,Volkswagen:Thescandalexplained.[Online].Available:http://www.bbc.com/news/business-34324772

[3] http://fortune.com/2017/10/25/reaper-botnet-mirai-iot-ddos/[4] A.M.Matwyshyn,“CYBER!,”2017BYUL.Rev.,vol.101,2018.[5] See80FR208,65956.[Online].Available:https://www.copyright.gov/fedreg/2015/80fr65944.pdf[6] http://www.wassenaar.org/[7] http://www.marketwatch.com/story/volkswagen-diesel-emissions-fixes-approved-2017-10-23[8] https://www.csoonline.com/article/3122460/techology-business/over-6000-vulnerabilities-went-

unassigned-by-mitres-cve-project-in-2015.html[9] https://www.csoonline.com/article/3146046/security/security-products-are-among-the-most-

vulnerability-riddled-software-products.html[10] https://www.wired.com/2017/05/accidental-kill-switch-slowed-fridays-massive-ransomware-attack/[11] https://www.nytimes.com/2017/09/18/world/europe/stanislav-petrov-nuclear-war-dead.html?_r=2[12] See,e.g.,http://psc.dss.ucdavis.edu/sommerb/sommerdemo/sampling/intro.htm[13] https://www.theverge.com/2016/3/24/11297050/tay-microsoft-chatbot-racist[14] https://www.darpa.mil/program/cyber-grand-challenge[15] IEEECybersecurityInitiative.Available:https://cybersecurity.ieee.org/[16] IEEEEthicallyAlignedDesign(EAD),Version2,AVisionforPrioritizingHumanWell-beingwith

AutonomousandIntelligentSystems.[17] See,e.g.,ISO29147and30111.[18] https://www.emc.com/leadership/digital-universe/2014iview/executive-summary.htm[19] https://christian-rossow.de/publications/sandprint-raid2016.pdf[20] https://www.rand.org/pubs/research_reports/RR1751.html[21] https://www.wired.com/2016/12/botnet-broke-internet-isnt-going-away/[22] J.Mirkovic,S.Dietrich,D.Dittrich,andP.Riher,InternetDenialofService:AttackandDefense

Mechanisms.London:Pearson,2004.

17

[23] https://www.arm.com/products/security-on-arm/trustzone[24] S.BorkarandA.Chien,“Thefutureofmicroprocessors,”CommunicationsACM,vol.54,no.5,pp.67–

77,May2011.[25] R.Gioiosa,D.J.Kerbyson,andA.Hoisie,"Quantifyingtheenergycostofdatamovementinscientific

applications,"inProc.EnergyEfficientSupercomputingWorkshop,2014,pp11–20.

Othermaterialusedinpreparationforthepaperincludesthefollowing:

[1] ISACA.(2017,Feb.).Stateofcybersecurity2017:Currenttrendsinworkforcedevelopment.[Online].Available:http://www.isaca.org/Knowledge-Center/Research/Documents/state-of-cybersecurity-2017-part-2_res_eng_0517.pdf

[2] ISACA.(2017,June).“Stateofcybersecurity2017:Currenttrendsinthethreatlandscape.”Available:http://www.isaca.org/Knowledge-Center/Research/Documents/state-of-cybersecurity-2017-part-2_res_eng_0517.pdf

[3] TheNewDogsofWar:TheFutureofWeaponizedArtificialIntelligence.Available:http://threatcasting.com/wp-content/uploads/2017/09/ThreatcastingWest2017.pdf

[4] https://www.rsaconference.com/writable/presentations/file_upload/spo1-t11_combatting-advanced-cybersecurity-threats-with-ai-and-machine-learning_copy1.pdf

[5] https://www.rsaconference.com/writable/presentations/file_upload/exp-t11-advances-in-cloud-scale-machine-learning-for-cyber-defense.pdf

[6] https://www.statista.com/statistics/471264/iot-number-of-connected-devices-worldwide/[7] https://www.dhs.gov/sites/default/files/publications/CSD-DHS-Cybersecurity-Roadmap.pdf[8] A.EpishkinaandSergeyZapechnikov,“ASyllabusonDataMiningandMachineLearningwith

ApplicationstoCybersecurity.”Proceedingsofthe2016ThirdInternationalConferenceonDigitalInformationProcessing,DataMining,andWirelessCommunications(DIPDMWC),Moscow,2016,pp.194-199.

[9] D.Zhu,H.Jin,Y.Yang,D.Wu,andW.Chen,“DeepFlow:Deeplearning-basedmalwaredetectionbyminingAndroidapplicationforabnormalusageofsensitivedata,”2017IEEESymposiumonComputersandCommunications(ISCC),Heraklion,2017,pp.438-443.

[10] J.B.FraleyandJ.Cannady,“ThePromiseofMachineLearninginCybersecurity.”SoutheastCon2017,Charlotte,NC,2017,pp.1-6.

[11] A.Tuor,S.Kaplan,B.Hutchinson,N.Nichols,S.Robinson,“DeepLearningforUnsupervisedInsiderThreatDetectioninStructuredCybersecurityDataStreams.”ProceedingsofAIforCyberSecurityWorkshopatAAAI2017.

[12] K.AlrawashdehandC.Purdy,“TowardanOnlineAnomalyIntrusionDetectionSystemBasedonDeepLearning.”201615thIEEEInternationalConferenceonMachineLearningandApplications(ICMLA),Anaheim,CA,2016,pp.195-200.

Participants

ConfluenceParticipantsDavidBrumley ForAllSecureandCarnegieMellonUniversityRobertK.Cunningham MITLincolnLaboratoryChrisDalton HPInc.ErikDeBenedectis SandiaNationalLaboratoriesFlaviaDinca StockholmUniversity,SwedenWilliamG.Dubyak IBMWatsonGroupNigelEdwards HewlettPackardEnterpriseRhettHernandez U.S.DepartmentofDefenseBillHorne IntertrustTechnologies

18

BrianDavidJohnson ArizonaStateUniversityAleksandarMastilovic UniversityNoviSad,SerbiaAndreaM.Matwyshyn NortheasternUniversityAbraham(Avi)Mendelson TheTechnion–IsraelInstituteofTechnologyDejanMilojicicǂ HewlettPackardEnterpriseKatieMoussouris LutaSecurity,Inc.AdrianL.Shaw ARMLtd.BarryShoopǂ U.S.MilitaryAcademy,WestPointTrungTran LaboratoryofPhysicalSciencesMikeWalker MicrosoftCorporation

TechnicalWriters

GlennZorpette IEEE,IEEESpectrumClayMoody U.S.MilitaryAcademy,WestPointMikeLanham U.S.MilitaryAcademy,WestPointMattSherburne U.S.MilitaryAcademy,WestPointDanielHawthorne U.S.MilitaryAcademy,WestPoint

Observers

DonnaHourican IEEEProvidenceMore IEEE

Theparticipantsbroughtexpertisefromawidevarietyofsectors.DavidBrumleyandhisteamfromForAllSecurewonthe2017DARPACyberGrandChallenge.MikeWalkerwastheDARPAprogrammanagerwhodevelopedandofferedtheDARPACyberGrandChallenge.WillDubyakfromtheIBMWatsonGroupisapplyingWatson’sNPLtocybersecurity.BrianDavidJohnson,previouslyIntel’sfuturist,hasrecentlybeenapplyingfuturecastingandthreatcastingtotheareaofcybersecurity.RhettHernandezservedasthefirstcommandinggeneraloftheU.S.ArmyCyberCommand.Dr.RobertK.CunninghamchairstheIEEECybersecurityInitiativeandleadstheSecureResilientSystemsandTechnologyGroupatMIT’sLincolnLaboratory.TrungTranhasworkedforIntelandHPand,morerecently,worksforthefederalgovernmentonbuildingthenextgenerationofAI.AndreaMatwyshyn,aprofessoratNortheasternUniversity,focusesontechnologyinnovationanditslegalimplications,particularlycorporateinformationsecurityregulationandconsumerprivacy.KatieMoussourisisacomputersecurityresearcherwhocreatedthebugbountyprogramatMicrosoft,waschiefpolicyofficeratHackerOne,andwasnamedoneof“10WomeninInformationSecurityThatEveryoneShouldKnow.”ErikDeBenedectisisamemberofthetechnicalstaffatSandiaNationalLaboratories,leadingaprojecttobuildapetaflops-scalesupercomputer,andisalsodeputyprojectleadfortheASCIRedStormsupercomputer.AdrianShawisasecurityarchitectatARMwithexperienceinsecuringsoftware-definedservicestomitigatethreatsintheIoT.Abraham(Avi)MendelsonservedatIntelandMicrosoftpriortojoiningtheTechnion,wherehefocusesonoperatingsystems,computerarchitecture,high-performancecomputing,andcloudcomputing.ChrisDaltonisadistinguishedtechnologistatHPInc.andleadsthePlatformandDeviceSecurityResearchGroupwithinHPLabs.NigelEdwardsisadistinguishedtechnologistatHewlettPackardLabs,whereheleadstheSecurityResearchGroup.BillHorneisavicepresidentatIntertrustTechnologies,whereheisgeneralmanageroftheSecureSystemsDivision.FlaviaDincaisaninformationsecurityPh.D.degreestudentatStockholmUniversity,withabackgroundinthesocialimplicationsoftechnologyandpolicy.AleksandarMastilovicistheEUMarieCurieFellowattheUniversityofNoviSad,Serbia.

Thewritersfocusedoncapturingthedialoganddebateduringthecollaborationengagements.GlennZorpetteisaseniortechnicaleditorforIEEESpectrum.ClayMoody,MikeLanham,MattSherburne,andDanielHawthorneareallU.S.ArmyCyberBranchofficersandfacultyintheDepartmentofElectricalEngineeringandComputerScienceattheU.S.MilitaryAcademyatWestPoint.

DejanMilojicicisaDistinguishedTechnologistatHewlettPackardLabs,pastpresidentoftheIEEEComputerSociety,andchairoftheIEEEIndustryEngagementAdHocCommittee.BarryShoopisaprofessorandheadoftheDepartmentofElectricalEngineeringandComputerScienceattheU.S.MilitaryAcademy,WestPoint,andservedas2016IEEEpresidentandchiefexecutiveofficer.

ǂProjectsponsors.