hyrim...hybrid risk management for utility networks collaborative project deliverable 3.4 monitoring...

1

Call:FP7-SEC-2013-1Activity:SEC-2013.2.5-4:Protectionsystemsforutilitynetworks–CapabilityProject

ProjectNumber:608090

HyRiM

HybridRiskManagementforUtilityNetworks

CollaborativeProject

Deliverable3.4MonitoringFrameworkandReferenceArchitecture

Duedateofdeliverable:[30/09/2016]Actualsubmissiondate:[30/09/2016]

Startdateofproject: April1,2014 Duration:36monthsOrganisationnameofleadcontractorforthisdeliverableETRAInvestigationyDesarrolloS.A

DisseminationLevelPU Public XPP Restrictedtootherprogrammeparticipants(includingtheCommissionServices) RE Restrictedtoagroupspecifiedbytheconsortium(includingtheCommissionServices) CO Confidential,onlyformembersoftheconsortium(includingtheCommissionServices)

Deliverable 3.4 HyRiM Monitoring Framework and Reference Architecture

3

HISTORYVersion Date Reason Reviewedby00.01 27-07-2016 Firstdraftversion AntoniosGouglidis–ULANC00.02 05-08-2016 Seconddraft AntoniosGouglidis–ULANC00.03 19-08-2016 Securitystandards AnaMaríaArias–ETRAi+D00.04 08-09-2016 Internalreviewandimprovements SyedAsadAliNaqvi–ULANC01.00 30-09-2016 Finalisingthedocument AntoniosGouglidis–ULANC

SyedAsadAliNaqvi–ULANC

AUTHORSLISTOrganization NameULANC AntoniosGouglidis([email protected];phonenumber:+44(0)1524510380)ULANC SyedNoorulhassanShirazi([email protected];phonenumber:+44(0)1524510380)ULANC SyedAsadAliNaqvi([email protected];phonenumber:+44(0)1524510380)ULANC DavidHutchison([email protected];phonenumber:+44(0)1524510331)ETRAI+D AnaMaríaArias([email protected];phonenumber:+34(0)963134082)


4

TableofContentsEXECUTIVE SUMMARY ................................................................................................................................................. 61 ABBREVIATIONS ..................................................................................................................................................... 72 OBJECTIVES .............................................................................................................................................................. 83 INTRODUCTION ....................................................................................................................................................... 84 PART 1 – MONITORING APPROACHES AND REFERENCE FRAMEWORK ................................................... 94.1 RELATEDWORK....................................................................................................................................104.2 PROPOSEDFRAMEWORK........................................................................................................................114.2.1 FrameworkDesign.....................................................................................................................114.2.2 ApplicabilityoftheFramework..................................................................................................13

4.3 EVALUATIONOFADTECHNIQUES............................................................................................................154.3.1 Method......................................................................................................................................164.3.2 Dataset......................................................................................................................................164.3.3 DescriptionofAnomalies...........................................................................................................174.3.4 EvaluationMetrics.....................................................................................................................184.3.5 Analysis......................................................................................................................................18

4.4 DATADENSITY(DD)BASEDANOMALYDETECTION.....................................................................................205 PART 2 - REVIEW OF STANDARDS .................................................................................................................... 225.1 THENISTCRITICALINFRASTRUCTUREFRAMEWORK.....................................................................................235.2 NISTIR7176.SYSTEMPROTECTIONPROFILE-INDUSTRIALCONTROLSYSTEMS..............................................245.3 NISTIR7628.GUIDELINESFORSMARTGRIDCYBERSECURITY......................................................................245.4 ISA/IEC-62443...................................................................................................................................255.5 IEC62443CONFORMITYASSESSMENTPROGRAM.....................................................................................265.6 CYBERSECURITYSTRATEGYOFTHEEUROPEANUNION................................................................................275.7 CYBERSECURITYCOORDINATIONGROUP(CSCG).......................................................................................285.8 GUIDETOINCREASEDSECURITYININDUSTRIALCONTROLSYSTEMS................................................................285.9 CYBERSECURITYASSESSMENTSOFINDUSTRIALCONTROLSYSTEMS.AGOODPRACTICEGUIDE............................285.10 ENISACONTRIBUTIONSTOSTANDARDISATION.......................................................................................295.11 OTHERSTANDARDS............................................................................................................................29

6 CONCLUSIONS ....................................................................................................................................................... 31REFERENCES .................................................................................................................................................................. 32ANNEX I: DATA DENSITY BASED ANOMALY DETECTION ALGORITHM (SHIRAZI ET AL., 2016) ............ 35

IndexofFiguresFIGURE1-D2R2+DRSTRATEGY(STERBENZETAL.,2010)..........................................................................................10FIGURE2-FRAMEWORKOVERVIEW........................................................................................................................12

FIGURE3-EVALUATIONRESULTS...........................................................................................................................15

FIGURE4-IDENTIFICATIONOFANOMALIESUSINGFCMWITHK=8..............................................................................17

FIGURE5-ISA-62443WORKPRODUCTSFORIACSSECURITY.....................................................................................26


5

IndexofTablesTABLE1-THREATTOMETRICSANDOTIMAPPING....................................................................................................14

TABLE2-COMPARISONOFADTECHNIQUES(COMBINEDDATASET).............................................................................19

TABLE3-PERFORMANCEMETRICSOFADTECHNIQUESPERTYPEOFANOMALY..............................................................20

TABLE4.FIVECOREFUNCTIONSOFEFFECTIVECYBERSECURITY(GUINN,2014)..............................................................23

TABLE5.TIERSOFCYBERSECURITYMATURITY(GUINN,2014).....................................................................................24


6

EXECUTIVESUMMARY

Thisreportprovidesamonitoringframeworkandreferencearchitectureforcritical infrastructures,whichhelp in the monitoring, detection and evaluation of vulnerability-creating behaviours. We re-visit therationale for protecting against cyber-attacks andpropose a framework tomonitor, detect and evaluateanomalous behaviour within critical infrastructures. Specifically, we describe a multi-level approach forassuringresilienceincritical infrastructuresandservices,takingintoaccountorganisational,technologicaland individuals' (OTI) viewpoints. The framework supports detection of anomalies by using appropriatetechniquesatthedifferentlevelsofinfrastructureandservice.Inthecontextofourresearch,weevaluatea setofanomalydetection (AD) techniques indetectingattacksbyanalysing traffic captured inaSCADAnetwork.Forthispurpose,wehaveimplementedatoolchainwithareferenceimplementationofvariousstate-of-the-artADtechniques todetectattacks,whichmanifest themselvesasanomalies.Specifically, inorder to evaluate the AD techniques,we apply our tool chain on a dataset created from a gas pipelineSCADAsysteminMississippiStateUniversity's lab,whichincludeartefactsofbothnormaloperationsandcyber-attack scenarios. Our evaluation elaborates on several performance metrics of the examined ADtechniquessuchasprecision;recall;accuracy;F-scoreandG-score.Theresultsindicatethatdetectionratemay change significantly when considering various attack types and different detections modes (i.e.,supervisedandunsupervised). Inthiscontext,wealsodemonstratetheefficacyofourDataDensity (DD)BasedAnomalyDetectionmethodasarobustandreal-timeADtechniquetointroduceresilienceincriticalinfrastructures. Finally, through the review of security standards, we anticipate helping criticalinfrastructure owners and operators to identify, assess, and reduce cybersecurity risk by aligning withexisting resources. The included standardisation approaches have been developed with inputs fromdifferent stakeholders in industry, academia, and government and therefore are reliable tools to helputilitiesinmanagingcybersecurityrisks.


7

1 ABBREVIATIONS

Term MeaningAD AnomalyDetectionANAB ANSI-ASQNationalAccreditationBoardANSI AmericanNationalStandardsInstituteAPT AdvancedPersistentThreatBYOD BringYourOwnDeviceCEN EuropeanCommitteeforStandardizationCENELEC EuropeanCommitteeforElectrotechnicalStandardizationCOBIT ControlObjectivesforInformationandRelatedTechnologiesCOTS CommercialOff-the-shelfCRISP ConsortiumforResearchonInformationSecurityandPolicyCSCG CyberSecurityCoordinationGroupDoS DenialofServiceECA Event,ConditionandActionEDSA EmbeddedDeviceSecurityAssuranceEMGM ExpectationMaximizationforGaussianMixtureModelETSI EuropeanTelecommunicationsStandardsInstituteEUCSS CyberSecurityStrategyoftheEuropeanUnionFCM FuzzyC-meansGMM GaussianMixtureModelIA InformationAssuranceIACS IndustrialAutomationandControlSystemsIAF InternationalAccreditationForumICS IndustrialControlSystemIEC InternationalElectrotechnicalCommissionILAC InternationalLaboratoryAccreditationCooperationISA InternationalSocietyofAutomationISCI ISASecurityComplianceInstituteISF InformationSecurityForumISO InternationalOrganizationforStandardizationISRAM InformationSecurityRiskAnalysisMethodITU InternationalTelecommunicationUnionJAB JapanAccreditationBoardforConformityAssessmentMRA MutualRecognitionArrangementsMTU MasterTerminalUnitNERC-CIP NorthAmericanElectricReliabilityCorporationcriticalinfrastructureprotectionNIS NetworkandInformationSecurityNIST NationalInstituteofStandardsandTechnologyOTI Organisation,Technology,IndividualPCA PrincipalComponentAnalysisPCD ProcessControlDomainRTU RemoteTerminalUnitSCADA SupervisoryControlandDataAcquisitionSDOs StandardsDevelopingOrganizationsSP SpecialPublicationsSSA SystemSecurityAssuranceSSE-CMM SystemsSecurityEngineeringCapabilityMaturityModelSVC SupportVectorClustering


8

2 OBJECTIVES

Themainobjectivesofthisdeliverableconsistsofthefollowing:

1. Todevelopa reference framework thatwill assist in thedetection,monitoringandevaluationofvulnerability-creatingbehaviours.

2. To inspect and reviewexisting standards related to critical infrastructures, IndustrialAutomationandControlSystems(IACS),andNetworkandInformationSecurity(NIS).

3 INTRODUCTION

Theorderlyfunctioningofcritical infrastructures isessentialforthehealthofanation'seconomy,anditssocial stability. As such, critical infrastructures comprise of assets and systems that maintain societalfunctions,includinghealth,safety,security,andtheeconomicandsocialwell-beingofpeople.SupervisoryControl and Data Acquisition (SCADA) Industrial Control Systems (ICS) are particular examples of criticalinfrastructures for themonitoring,controlandautomationofoperationalplantsofvarioussorts, suchasUtility networks. In recent times, there has been a significant increase in the functional demands uponutilities,resultinginanincreasedrateofautomationinnetworkedcontrolsandinterconnections,aswellasanincreaseindependenciesbetweendiverseinfrastructures.Consequently,utilitynetworksarenowmoresusceptible to sophisticated attacks including Advanced Persistent Threats (APTs). Additionally, newchallenges arising from system complexity, overloading, unanticipated human behaviour, andvulnerabilities fromthird-partysourcesmustalsobeconsidered.Needless tosay,providingprotection intermsofsecurity,safetyandresilienceinsuchnetworksisvitallyimportant.Researchontheemergingareaofsecurity incritical infrastructureshasresultedinrules, legislationandgood-practiceguidelinesthatwewillreviewlaterinthisreport.AcommonapproachtowardsconceptuallyunderstandingSCADA-ICSistodividetheminto ‘levels’,basedontheirfunction.Often,asimplethree-levelapproachisadopted:fieldsite,controlcentre,andcorporate(Wei, Lu, Jafari, Skare,& Rohde, 2011). Specific devices, boundaries, processes, etc. are then associatedwith each level, depending on the industry and network topology in question. More detailed layeringapproaches, such as the Purdue model (Obregon, 2015), are able to provide further granularity byintroducingasix-levelviewapproach.Nevertheless,inallcasesthereisaclearindicationoftheboundariesandinterconnectionsbetweenthelevels.However,thevulnerabilitiesandthreatsaffectingSCADA-ICSareseldom confinedwithin specific functional boundaries; rather they cut acrossmultiple functional levels.Therefore, in order to support the monitoring and detection of threats at all levels; we propose theirinvestigation on the basis of three viewpoints, viz. Organisation, Technology and Individual (OTI), asdiscussedinHyRiM’sDeliverable1.1(Gouglidisetal.,2015).Bymakinguseoftheseviewpoints,weprovideindicationswith regard to different threats in ICS,which subsequently operate as input to a frameworkcapableofassuringresilienceinICS.Thedetectionofthreatswithintheframeworkisaccomplishedviatheapplicationofseveralanomalydetectiontechniques.Theapplicationof theOTIviewpointsenablesabroaderviewof thesystem, i.e.,a representationof thewholesystemfromtheperspectiveofarelatedsetofconcerns.Thishelpsinincreasingthelevelofthreatawarenessbyidentifyingpotentialvulnerability-creatingbehaviours.Specifically,theconcernsofthethreeviewpointsare:Theorganisationviewpoint isconcernedwiththegroupsofpeoplewhoworktogether inan organisedway for a shared purpose aswell as any type of policies, processes and procedures in theorganisation; The technology viewpoint, with the implemented technologies in a system including thesoftware, hardware and network components, as well as any type of communication among them; Theindividual viewpoint, with the way a single person or entity acts or behaves in a particular situation orunderparticularconditions.


9

Motivated by the importance of protecting modern and future critical infrastructures; later in thisdeliverableweoutlineamulti-levelresiliencereferenceframeworkfortheprotectionofutilitynetworks.AcriticalaimoftheproposedframeworkistousetheOTIviewpointstoprotectagainstsophisticatedattacksthatcombinetechnologyvulnerabilitiesandsocialengineeringtechniques,andmaymanifestthemselvesinvariousanomalies.

The remainder of this report is separated in two parts. The first part provides information about theproposedmonitoringapproachesandreference framework,while thesecondpartelaboratesonexistingsecurity standards related to industrial control systems. Concluding remarks are presented in the lastsectionofthereport.

4 PART1–MONITORINGAPPROACHESANDREFERENCEFRAMEWORK

Attacks on critical infrastructures have increased over the years. In particular, attacks targeting SCADAindustrialcontrolsystemsrose100%in2014comparedtothepreviousyearashighlightedinareportby(Dell, 2015). Similarly, a recent report published by the industrial control systems cyber emergencyresponseteam(ICS-CERT)showedthatwhileICSvendorshavebeentargetedbyvarioustypesofmaliciousactors, over half of the attacks reported in 2014 involved advanced persistent threats (APTs) (ICS-CERT,2015).Moreover,majorvulnerabilitiesinSCADAsystemsenabledattacksonvariouscriticalinfrastructuresin the past, which demonstrated that these systems are not as resilient, as one would expect. Stuxnet(Falliere, Murchu, & Chien, 2011) was firstly identified as a complex malware that targeted the SCADAsystemsonIran'snuclearplant.InMaroochi,Australiaadisgruntledengineerpenetratedasewagecontrolsystemand causedapproximately264,000gallonsof raw sewage to leak tonearby rivers (Slay&Miller,2007).Also,inlate2015,amajorattackonUkraine'spowergridinfrastructureresultedinapoweroutagecausedbytheBlackEnergytrojan(ESET,2016)SCADAsystemsmonitorand control infrastructures includingpowerplants,waterutility, energyandgaspipelines,whichmakesthemhighlycritical.Providingprotectionintermsofsecurity,safetyandresiliencein suchnetworks is inherently considered tobeof vital importance.Traditionally,mostof these systemswereairgapedfromothernetworks,butinseveralcases,accesstothesedevicesmaystillbeavailableoverapublicnetwork(e.g.,Internet)asarequirementtoimproveusabilityviaprovidingoperatorsthepotentialto remotely access devices. While the automation and interconnectivity contribute to increase theefficiency and reduce operational costs, they expose these systems to new threats. For instance, theexistenceofavulnerabilityinasystemonthetoplayersofthePurduemodel(Obregon,2015)mayallowattackers to exploit them and to gradually take control of systems or devices that operate in the lowerlevels,suchasSCADAsystems;thiscouldcausefailureandhenceseriousdisruptions.Therefore,itiscrucialthatSCADAsystemsareresilienttoanychallenge(whetherdeliberateoraccidental)posedtotheirorderlyfunctioning.Theterm‘resilience’hasbeenusedforseveraldecadesindifferentwaysby various fields (Science, Business, etc.) to describe the ability of materials, engineered artefacts,ecosystems, organisations, communities, etc., to adapt to changes (Hollnagel,Woods,& Leveson, 2007).The resilience strategy that we use here, entitled D2R2+DR (Defend, Detect, Remediate, Recover, andDiagnoseandRefine)definesresilienceas ‘theabilityofanetworkorsystemtoprovideandmaintainanacceptablelevelofserviceinthefaceofvariousfaultsandchallengestonormaloperation’(Sterbenzetal.,2010).TheoverallresiliencestrategyisdepictedinFigure1.The challenges faced by SCADA-ICS are somewhat different from those of conventional IT systems. Forexample, amajor concern is the intrinsicweakness of communication protocols used in SCADA systemsthat monitor and control field devices in critical infrastructure installations. The remote terminal units(RTUs) interface, which generally control and collect information that determine the system state, and


10

masterterminalunits(MTUs)whichhandlesthesupervisorycontrols,canbeattackedtospoofinformationbyexploiting the lackofauthenticationprovidedbycurrentprotocols (e.g.,Modbus,DNP3andProfibus)and leadtounexpectedbehaviours (Carcano,Fovino,Masera,&Trombetta,2008).Data injectionattacksmayalsobeusedtochangemeasurementvaluesofsomedevices,inordertohindertheoperationofthesystem(Chen,Cheng,&Chen,2012).A first step towards achieving resilience in critical infrastructuresmay be the identification of abnormalbehaviours in such environments. Anomaly detection (AD) is a technique that can be applied within aresilience framework in order to promptly provide indications and warnings about adverse events orconditionsthatmayoccur.Specifically,wedemonstratein(Gouglidis,Shirazi,Simpson,Smith,&Hutchison,2016) a resilience framework for critical infrastructures that support the detection of anomalies at thedifferentlevelsofinfrastructureandservices.Thiswasaccomplishedwithinthe“detection”processofourD2R2+DR resilience strategy, where several resilience metrics were collected and forwarded to ADinstances. The diverse nature of data in critical infrastructures' networks comparedwith data stemmingfromITsystems,andtheexistenceofmajorthreatssuchasAPTsrenderthetaskofevaluatingexistingADtechniques to be of vital importance. This evaluationwill provide indications for their effectiveness andapplicabilityinacriticalinfrastructureenvironment,andinitiatefurtherresearchinthatdirection.

Figure1-D2R2+DRstrategy(Sterbenzetal.,2010)

Anomalydetection(AD)techniqueshaveexhibitedsufficientdetectionandaccuracyforcharacterisingandidentifying challenges (A. K. Marnerides, Watson, Shirazi, Mauthe, & Hutchison, 2013; Simpson,Marnerides, Watson, Mauthe, & Hutchison, 2014; Wang et al., 2011). This is due to the fact that thestatistical models embodied in these techniques allow the robust characterisation of normal behaviourtaking into account various features (operational and network) to detect known and unknownpatterns.However,whenused in infrastructurenetworks, these techniquesaregenerallyemployed independentlyoncertainpartsofthenetwork,sthus,donotnormallyprovideaholisticviewofthesystem,asindicatedbelow.

4.1 RelatedWork

Recentresearchhasfocusedonanomalydetection(AD)techniquestoimprovetheresilienceandsecurityofcriticalinfrastructures(A.K.Marnerides,Smith,Schaeffer-Filho,&Mauthe,2015;Simpsonetal.,2014).InthecontextofSCADAsystems,a fewanomalydetectiontechniqueshavebeenadoptedandredefined(Damiani,2009;Gao,Morris,Reaves,&Richey,2010;Marton,Sánchezb,Carlosa,&Martorella,2013)andtheyarefurtherclassifiedwithrespecttotheiroperationalmode,i.e.,supervisedandunsupervised.

Diag

nose

Refine

Defen

d DetectRem

ediateRec

over


11

In(Lee,Park,Eom,&Chung,2011),theauthorsproposeamulti-levelapproachthatprovidesfastdetectionofanomalies in loggingdatafromtheunderlyingoperatingsystem. However, itrequiresmoreresourcesforitsoperationsanditisonlyspecifictodetectioninloggingdata.There is also an abundance of frameworks in the literature that have been proposed for providing pro-activedetectionof anomalousbehaviour in variousenvironments (A.Marnerides et al., 2011;Mdhaffar,Halima, Jmaiel,&Freisleben,2014;Thonnard&Dacier,2008).However, theseframeworksgenerally lackremediationandrecoverystages,whicharecrucialprocessesforaresilience-awareframework.Inaddition,theappliedtechniquesoftenrequiretheuseofcomplexstatisticalmeasures,andtheymayalsohavepoorscalability,whichmeans they are not suitable in an operational context. Having a set of objectiveswithregard to the detection of anomalous behaviour, the work conducted in (Wang, 2009) illustrates aframeworkthatanalysesmultipletypesofmetricsfromthesystemcomponents(e.g.,memoryutilisation,read/writecounts,etc.).Thepresentedresultshavepromisingscalability,butevaluationislimited.In (Dastjerdi, Bakar, & Tabatabaei, 2009) and (Garfinkel & Rosenblum, 2003), the authors propose ananomaly detection technique to detect intrusions at different layers of a cloud system. However, theproposed technique appears to be rather inflexible, and its application on an operational environmentrequires better clarification. Similarly, in (Dastjerdi et al., 2009), the authors propose a mobile-agentapproachforan intrusiondetectionsystemforcloudsystemsbasedonanomalydetection,yetscalabilityappearstobeanissueduetothehighnumberofvirtualmachinesthatarerequiredtobeattachedtotheagent.Theauthorsin(Pannu,Liu,&Fu,2012)instrumentedareal-timeanomalydetectionframeworkthatisableto detect failures through the analysis of runtime metrics using the Support Vector Machine (SVM)algorithm. The results are promising since the detector's sensitivity is demonstrated to be very high.However,themainissueraisedbythisstudyisthattheformulationofthetwo-classSVMalgorithmsuffersfromthedata-imbalanceproblem,whichleadstoseveralmis-classificationswhenitcomestonewlytestedanomalies.In (Sharma, Jayachandran, Verma, & Das, 2013), the authors propose a framework that uses a layeredapproach to dealwith frequent reconfiguration. The framework is able to identify anomalous behaviourand uses a feedback loop that allows problem remediation to be integratedwithmanagement systems.However,theemployedtechniqueisnotsuitableforreal-timeanalysisduetoanomalydiagnosisbasedonpre-computed signatures and computational overhead of pre-processing stage, and it is not clear howremediationrulesareestablished.We have referred to several cloud-specific approaches because we are interested in evaluating ADtechniquesincloudenvironments.Foramorecomprehensivepresentationandanalysisofexistingworkonanomalydetectiontechniques,wereferthereadertothetechnicalreportbyChandolaetal.in(Chandola,Banerjee,&Kumar,2009).

4.2 ProposedFramework

In this sectionwedescribeourproposed resilience framework forSCADA-ICS,and itsevaluation throughexperimentsconductedinthecontextofaEuropeanutilitynetwork.

4.2.1 FrameworkDesign

OurproposedresilienceframeworkforSCADA-ICSisdepictedinFigure2,whichconsistsoffourfunctionalplanes implemented as software components. These include the Monitoring, Detection, Analysis andManagement planes. These planes work in collaboration to provide overall resilience of infrastructuresagainst challenges that manifest themselves in the guise of various anomalies (e.g., social engineering


12

attacks,operatormisconfigurations,unanticipatedproblemsarisingfromorganisationalpolicies,etc.).Eachplanehasmultiplesub-componentstoperformlocaltasks(e.g.,pre-processing,internalstorage,etc.)andprovidesinputtoasubsequentplane.

Figure2-Frameworkoverview

With reference to the D2R2+DR strategy, our resilience framework satisfies the role of the inner loopelements by realising them as software components. These components are capable of reconfiguringdevicesinresponsetochallengesusingsuitablepolicies.Reconfigurationisnotrequiredforapplicationonthe same components in which the anomaly was detected. A policy engine is responsible for mappingdetectioneventstoreconfigurations.TheMonitoringplaneisconcernedwithworkingontheresiliencemetricsthatarereportedtoitbyasetofCollector agents. This involves pre-processing, feature extraction and selection, dimensionality reductionand transformation ofmetrics into feature vectors for subsequent AD instances that are running in theDetectionplane.TheMonitoringplaneactsasacontrollerformultipleinstancesofCollectoragentsrunningin various viewpoints: COrg for organisation, CTech for technology and CInd for individual. Similarly, themonitoring instances are also classified by the viewpoint inwhich they are operating, for exampleMOrg,MTech, MInd for organisation, technology, and individual viewpoints respectively. The classification ofinstancesinvariousviewpointshelpstoinvokemostrelevantandeffectivetechniquesforeachviewpoint.Thisisduetothefactthatdifferenttypesofpre-processingtechniquesmayberequiredfordifferenttypesofmetricsbeingcollectedforeachviewpoint.Furthermore,thelocalstoragefunctionallowsrecordingoftheincomingmetricsinadatabaseforfutureanalysis.TheDetectionplanecanbeconsideredasacorecomponentthatperformsdetectionofanomaliesbasedon metrics gathered by the Collector agents and the features processed byMonitoring instances. Thiscomponent isdesigned tooffer flexibility so thatdifferent typesofAD techniquescanbe invokedat thesametime.Thisisofspecialinterest,sincesometechniquesprovidebetterdetectioncapabilitiescomparedtoothersand,consequently,havean impacton theoverall resilienceofan infrastructure.Currently,our

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

�

�

��

��

��

��

��

��


13

tool-chainprovides reference implementations for sixdetectorsbasedonK-means,PrincipalComponentAnalysis (PCA), Wavelet, Gaussian Mixture Model (GMM), Support Vector Clustering (SVM) and NaïveBayesian techniques.Themain sub-taskof thisplane is todefine statisticalmodels toexpressnormality.Withbetter knowledgeof thedata, one can choose the appropriatedetection techniques. For example,small datametrics (i.e., data small enough for human comprehension) can be analysed and labelled byexperts. Such metrics might fall under the organisation or individual viewpoint, e.g., the monitoring ofactiveremoteconnections to the internalnetworkaspartofanorganisation'spolicy. Insuchacase, theapplication of a supervised technique might be considered more appropriate than an unsupervisedtechnique. Conversely, it is difficult and time-consuming to label large data sets coming from thetechnologyviewpoint.Subsequently, themain task of theAnalysis plane is to classify the events/messages obtained from theDetectionplane.Firstly,thesetofdataproducedisparsedandstoredinadatabaseforfutureprocessing.Thedatasetisthencomparedagainstthemodelsbuiltpreviouslytoestablishidentificationandattributionof anomalies. At the same time, coarse-grain policy actions canbe invoked, e.g., sandboxingof affectedcomponents.Theframeworkcurrentlyoffersareferenceimplementationoflimitedremediationactions(includingratelimiter, sandbox, etc.). Subsequently, theDetection plane conveys information to the Analysis plane tofurtherdecideontheremediationandrecoveryactions.TheseactionswhendecidedbytheAnalysisplaneareinvokedviatheManagementplane,whichisresponsiblefortheenforcementandexecutionofpolicyactionsthatgoverntheoverallresilience.Thisembodiestheself-organisingaspectoftheoverallsystem.ThepolicyengineintheAnalysisplaneisbasedonthe‘Event,ConditionandAction’(ECA)paradigm.Inthecontextoftheframework,eventsfromtheDetectionplaneareusedtotriggereventsforthepolicyenginethatcanperformremediationactions.Dependingonthedeployedpolicymechanisms,remediationactionscanbeperformedbyexecutionpointsinthesystemsuchasstartinganotherinstanceofthecomponentsbeingaffected.Thisisanincrementalapproachtochallengemitigation,wherebyanevolvingunderstandingof the nature of challenge is developed. The idea is to enable early remediation to protect underlyinginfrastructurefromtotalcollapse,andtosubsequentlyapplymorespecificremediationactionswhenmoredetailsareestablished.Theapproachissimilartothatproposedby(Gamer,2009).Finally,theManagementplaneisalsoresponsiblefordeployingrelevantCollectionagentsthatarecapableof gathering the monitoring metrics. Multiple instances of these agents could be instantiated for eachviewpoint,thusofferinggreaterflexibilityintheoveralldesignoftheframework.

4.2.2 ApplicabilityoftheFramework

Inthissection,weprovideanevaluationoftheproposedframeworkthroughexperimentsconductedinthecontextofaEuropeanutilitynetwork.Throughtheapplicationofasetofinterviews,ethnographicstudieswithoperatorsandtechnicalassessmentoftheutility'snetworkinfrastructure,wehaveidentifiedavarietyofpotentiallyhigh-risk threats to theutility. Theanalysisof threats in the contextof theOTI viewpointsserves as a stepping stone towards identifying several metrics. Specifically, the analysis of the utilitynetworkhelpedtodeterminethefollowingthreats:BringYourOwnDevice(BYOD)policy--theapplicationofsuchapolicyintroducesanumberofissuestotheinternalnetworkmostlyduetotheabsenceofastrictpolicyregardingtheinstallationofsoftwareandunsuperviseduseofpersonaldevices;Remoteaccess--thepotential for someemployees toaccess internaldevices remotely;Spearphishing -- ahigh likelihood foremployees to respond to spam e-mails, which might result in initiating spear phishing attacks (Green,Prince, Busby, & Hutchison, 2015);Network scanning -- the potential to deploy network scans that willresultinmappingtheinternaltopology;Malware--thepotentialforapieceofmalwaretospread(duetothe absence of appropriate security controls) and to activate itself. A detailed mapping of threats tometrics,OTIlevelsandcollectorsisdepictedinTable1.


14

Table1-ThreattometricsandOTImapping

Threat Metrics OTILevel CollectorBYOD No.ofconnectedpersonaldevices

No.ofinvalidrunningapplicationsOrganisationIndividual

COrg

CIndRemoteAccess

No.ofactiveremoteconnections Organisation COrg

SpearPhishing

No.ofspame-mail Individual CInd

NetworkScanning

No.ofpacketsNo.ofbytesNo.ofactiveflows

TechnicalTechnicalTechnical

CTech

CTech

CTechMalware Processutilisation

MemoryutilisationTechnicalTechnical

CTech

CTechHaving identified a set of threats, and corresponding metrics for their identification, we performed anexperimentwhereeachthreatscenariowassimulatedtotestthecapacityofourresilienceframeworktodetectthesechallengeswhileunderattack.Fortheexperiment,presentedinFigure3,wesetupatestbed(Watson,Marnerides,Mauthe,&Hutchison,2016)withtwohostsrunningmultipleVMsusingtheKernelVirtual Machine (KVM) hypervisor. This setup allows us to create scenarios that are relevant to ourdescribedusecase.EachVMrunsApacheHTTPdandtheclienthostrunscustomscriptstoinitiaterandomHTTP requests from theVMs.Wedid 10-minute runs,withweb traffic at fixed rate and anomalies (i.e.,ZeusmalwareandNmapnetworkscan)happeninghalfwaythrough,i.e.,after5minutes.Foreachscenario,wewroteaplug-in foraCollector agent to collect relevantdata.Thevolatility introspection library fromVolatilityFoundationwasusedtocollectrawsystemfeaturessuchasprocessandmemoryutilisation.Fornetscan,ouragentcollectedrawnetworkfeaturesthat include:Numberofpackets,numberofbytesandactiveflows.TheserawfeaturesaresenttoMonitoringinstancesthatdividethedatasetinto3-secondbinsand each bin is converted into a feature vector of the form x = (x1, x2, …, xn-1, xn) after applyingnormalisationusingZ-score.ThesefeaturevectorsarethenpassedontotheDetectionplanerunningtwodifferentinstancesofADtechniques(K-meansandPCA).ThedetectorbasedonK-meansrunsinsupervisedmode, and it seems suitable to runon small datasets (i.e., synthetically generated),whichwas easier tolabelfortraining.ThedataforthesmalldatasetswasextractedfromtheethnographicstudyperformedinHyRiM’sdeliverable3.1.For largerdatasetsas inthecaseofnetscanandmalwareweusedunsuperviseddetectors based on PCA. The features vectors are submitted to both the detectors to signal anomalouspatterns. The Anomaly Score Graph (ASG) is produced by the Analysis plane, which is a time-seriesrepresentation of the anomalous score of each bin, and indicates how anomalous each timebin is withrespect to others. The datawith regard to BYOD, remote access, and spear phishingwere syntheticallycreated,i.e.,randomlygenerated.Nevertheless,wetriedtounderstandtherangeofusualvaluesthroughtheprocessofinterviewsandethnographicstudies,e.g.,weidentifiedhowmanyemployeescanremotelyaccesstheinternalnetwork,weassumedanumberof incomingspame-mails(Greenetal.,2015),etc. Inadditiontonormalvalues,weinjecteddataintotheindividualfeaturevectorsinordertotriggeranomaliesand elaborate on our attack scenario. The red lines indicate the threshold or confidence interval abovewhichevents are generatedandpolicy engines are triggered to invokepolicies such as rate limiting andsand-boxing thevirtualmachines.Setting the thresholdorconfidence interval is considered tobeexpertknowledgeorcanbesetarbitrarily.InFigure3,weprovidetheresultsofprocessingthesetofdefinedmetricsusingthepreviouslydescribedtoolchainanddata.Theexperimentwasrunoverthreeconsecutivedays.Regarding‘Day1’,wecalculatedthegraphsprovidedonthe leftsideofFigure3.Theresultsdescribetheoutputoftheanomalydetectorand indicate anomalous behaviour related to the threats of BYOD, remote access and spear phishing.Initially,thisinformationcouldprovideusefulindicationstotheadministratorsoftheutilitynetworkabout


15

currentsignsof threats. In this scenario,weassumethatasuccessful spearphishingattackwas initiated(top-leftdiagram).This isconsideredapossiblescenarioduetothe likelihoodofoperatorsrespondingtospear-phishing e-mails (Greenet al., 2015).On ‘Day 2’, (middle graph) thedetectorwas able to identifyabnormalactivitythatwascausedbytheexecutionofamalwareapplication,whichwasintroducedtotheinternalnetworkviatheresponseofanoperatortoaspame-mail,inourattackscenario.Lastly,on‘Day3’,(right graph) the detector identified abnormal activity, which is the result of a network scan operation,initiatedby the installedmalware to subsequently exfiltrate internal network information to an externaldestination.

Figure3-Evaluationresults

This initial evaluation of our framework provides interesting insights about how it is possible to beprotected against sophisticated attacks. Although not performed in real time, the monitoring oforganisation and individual metrics were triggered, and provided indications for anomalous behaviour.Regarding themonitoringof technicalmetrics, the indicationsof anomalousbehaviourwerequite clear.Overall,wecanconcludethatthemonitoringofOTImetrics,incollaboration,providesobviousindicationsonhowandwhenanattackwasinitiatedandhowitevolved.

4.3 EvaluationofADTechniques

The evaluation of our resilience framework indicates that anomaly detection techniques performdifferentlydependingonboththecharacteristicsofthedataduringnormaloperationaswellasthenatureoftheattack.ChoosinganappropriateADtechniqueforusewithSCADAsystemsrequiresanexaminationoftheireffectivenessindetectinganomalousSCADAoperations,e.g.,trafficbetweenRTUandMTU.Fromanoperational perspective, supervised techniques require trainingdata tobuild themodel andevaluatethefitnessofthenewtestdatawithrespecttothismodel.Ontheotherhand,unsupervisedtechniquestryto partition the feature spaces into normal and anomalous regions without training data, and AD

Timebins50 100 150

Anom

aly

stat

istic

s

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

ASG for Malware (Zeus) using PCA

Timebins5 10 15 20

Anom

aly

stat

istic

s

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

ASG for Netscan using PCA

Timebins5 10 15 20

Anom

aly

stat

istic

s

0.20.40.60.8

ASG for BYOD using PCA

Timebins5 10 15 20

Anom

aly

stat

istic

s

0.20.40.60.8

ASG for SpearPhish using K-means

Timebins5 10 15 20

Anom

aly

stat

istic

s

0.2

0.4

0.6

0.8

ASG for RemoteAccess using K-means

Day 1 Day 2 Day 3


16

techniquesinthismodearemuchmoreflexibleandeasytousesincetheydonotrequireupfronthumaninterventionandtraining(Chandolaetal.,2009;Laskov,Düssel,Schäfer,&Rieck,2005).Inthefollowing,weprovideanevaluation,intheSCADA-ICScontext,ofstate-of-the-artanomalydetectiontechniques as well as a data-density (DD) basedmemory-less ADmethod (described in section 4.4) wedevelopedforusewithourresilienceframework.Specifically,thissectionpresentsinformationaboutthemethodandthedatasetusedfortheevaluation,andtheresultsofadetailedperformanceevaluationofsupervised (K-Means, Naïve Bayesian) and unsupervised (PCA, GMM, and Data-Density based (DD))anomalydetectiontechniques.Theselectionofthissubsetofstate-of-the-arttechniquesisbasedonpriorexperiencefromSECCRIT’sdeliverable4.11

4.3.1 Method

TheapproachwehavetakentoevaluatetheADtechniquesisdescribedbelow:

• Obtain themost significant features from the original dataset via pre-processingmethods usingnormalisationandprincipalcomponentanalysisofthedataset.ThesefeaturesarethenconvertedintoatimeseriesandfedintoadetectorimplementingtheADtechniques.

• We further split the dataset into 8 different traces. The first one consists of a combined trace,whichincludestheclassofnormaldataandsevenanomalousclasses.Therestoftheseventracesinclude the normal class and one of the anomalous classes. The latter is used to evaluate theefficacyoftheindividualtechniquestodetectspecifictypeofanomalies.

• Each trace is then submitted to a detector along with ground truth information to assess theapplied AD technique based on its likelihood to identify anomalies in the traces over time.Dependingontheirmode,ADtechniquesmayrequireatrainingphase(e.g.,supervisedlearning).Inthiscase,arandomselectionofthefeaturevectorisusedastrainingdata,andtherestisusedtogeneratetheanomalytimeseriesusedfortheevaluation.

• Theoutputof thedetector that includesa timeseriesofprobabilities is thencomparedwith thegroundtruth,andyieldsanevaluationoftheADtechniques.

Moredetailswithregardtothedatasetandeachoftheaforementionedstepsareprovidedinsubsequentsubsections.

4.3.2 Dataset

The dataset we used was collected using a simulation of real anomalies and normal activity on a gaspipeline. Specifically, it constitutesModbus traffic2 stemming from a serial line and including ‘read’ and‘write’commandsforaPLC3.Itcontainsthreecategoricalfeaturesincludingpayloadinformation,networkinformation and ground truth. The payload information indicates the gas pipeline's state, settings andparameters.Thenetworkinformationprovidespatternofcommunicationsandgroundtruthdetails,i.e.,ifthetransaction isnormaloranomalous. Intotal274627 instancesandtwentyrawfeaturesareprovided.We refer the reader to (T. Morris & Gao, 2014; T. H. Morris, Thornton, & Turnipseed) for a detaileddescriptionoftheindividualfeatures,datasetandtestbedarchitecturethatwasusedtocapturethedata.

1 https://www.seccrit.eu/upload/D4-1-Anomaly-Detection-Techniques-for-Cloud.pdf 2 http://www.modbus.org/ 3 https://en.wikipedia.org/wiki/Programmable_logic_controller


17

Wefirstemployapre-processingstage,which includesnormalizationofdatausingZ-score4andprincipalcomponentanalysis(PCA)toselectasubsetofrelevantfeaturesforsubsequentanalysis.ThePCAallowsusto extract new, orthogonal (independent) features that are a linear combination of the original ones.Basically,thesenewsetoffeatures,calledprincipalcomponents,areobtainedinsuchamannerthatthefirst principal component accounts for asmuchaspossibleof the variation in theoriginal data then thesecondcomponentandsoon.Weselect14principalcomponentsasnewderivedfeaturesforouranalysisbecausetheyrepresentmostvariationinoriginaldataset–therefore,theyaremostsignificant.Furthermore,weemploysoftclusteringapproachusingFuzzyC-means(FCM)(Bezdek,Ehrlich,&Full,1984)to identifynaturalgroupingsofdata.Asopposetohardclustering, inFCMthedatapointscanbelongtomore thanonecluster,andassociationwitheachof thepointsaremembershipgrades that indicate thedegreetowhichthedatapointsbelongtothedifferentcluster.Figure4illustratestheinherentstructureofthedata. Itcanbeseenthatthedata isnoteasilyseparable into8classes,but insteaditseparatestheminto4classeswherebluecolourindicatesthenormalclass.Thisisanimportantstepinunderstandingthedatasetwithrespecttothenumberofclasses.

Figure4-IdentificationofanomaliesusingFCMwithK=8

4.3.3 DescriptionofAnomalies

Intotal,thedatasetcontainssevendifferenttypesofanomaliesthataredividedintofourmaincategories.Theseanomaliesinclude‘responseinjection’,‘reconnaissance’,‘denial-of-service’,and‘commandinjection’.The response injection is further divided into naïve malicious response injection (NMRI) and complexmalicious response injection (CMRI). The former leverages the ability to inject response packets in thenetworkbut lacks informationabout theprocessbeingmonitored.The latteron theotherhand ismoresophisticatedandattempts tomask the real stateof thephysicalprocessbeingcontrolled.Similarly, thecommandinjectionisfurtherdividedintomaliciousstatecommandinjection(MSCI),maliciousparameter4 The resultof Z-scorenormalization is that the featureswillbe rescaled so that theywillhave thepropertiesofastandardnormaldistributionwithμ=0andσ=1,whereμ isthemean(average)andσ isthestandarddeviationfromthemean.


18

commandinjection(MPCI)andmaliciousfunctioncodecommandinjection(MFCI).MSCIchangesthestateoftheprocesscontrolsystemtodrivethesystemfromsafestatetocriticalstatebymaliciouscommand.MPCIchangesPLCsetpointsandMFCI injectscommandwhichmisuseprotocolnetworkparameter.DoSattack targets communication link. Each sample is labelled with its ground truth from (0-7) where 0representsnormalclassand1-7isforeachclassofanomalies.

4.3.4 EvaluationMetrics

A single metric alone is not sufficient to make a firm conclusion about performance of the underlyinganomalydetectiontechnique(Lazarevic,Ertöz,Kumar,Ozgur,&Srivastava,2003).Therefore,weevaluatedthe effectiveness of each technique using several metrics. Each input entry submitted to the detectordescribes the features of the monitored trace during a given time period (bin), and subsequently thedetector computes the deviation from normal traffic. Therefore, the performance can be assessed bydetermining thedifferencebetween the class it produces for a given input and the class it shouldhave.Correctly identified negatives are true negatives (TN), incorrectly identified negatives are false positives(FP), correctly identified positives are true positives (TP) and incorrectly identified positives are falsenegatives(FN).Fromthisoutputitallowscomputationofthetrue-positiverate(TPR,sensitivityorrecall;TP/(TP+FN)),thefalse-positiverate(FPR;FP/(FP+FN)),theprecision(TP/(TP+FP)),theaccuracy(TP+TN/ TP + TN + FP + FN), the F-score (2 x (Precision x Recall) / (Precision + Recall)), and the G-mean𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛×𝑅𝑒𝑐𝑎𝑙𝑙.

Accuracy is thedegree towhich thedetector classifiesdata samples correctly; precision is ameasureofhowmanyofthepositiveclassificationsarecorrect,i.e.,theprobabilitythatadetectedanomalyhasbeencorrectlyclassified;andrecallisameasureofthedetector'sabilitytocorrectlyidentifyananomaly,i.e.,theprobability thatananomalous samplewillbecorrectlydetected.The final twometricsare theharmonicmean(F-score)andgeometricmean(G-mean),whichprovideamoreroundedmeasureoftheperformanceofaparticulardetectorbyaccountingforalloftheoutcomestosomedegree.

4.3.5 Analysis

Oneofthemainissueswiththerawdatasetwasthatitcontainedmissingvalues,andthus,requiredfromus to perform a set of pre-processing tasks in order to make the dataset suitable for use in our ADimplementations.Otherwise,theresultsoftheanalysiswouldnotbeindicativeoftheactualperformanceof the examined AD techniques. Specifically, we pre-process the raw dataset by applying Z-score andprincipal component analysis techniques such that it remains representative of the original data,particularly in scope of the attack scenarios, while being better suited to use with AD techniques.Henceforth, we call this new derived feature-set as combined dataset since it contains artefacts of thenormaldataandallseventypesofanomalies.Subsequently,weusedthecombineddatasetasaninputtoourADimplementations.However,someoftheoperationsregardingADtechniquesrequiredanexcessiveamountof timeandmemory to completedue to the sizeof the combineddataset (275,000 rows), e.g.,normalisationofdata.Therefore,inordertoovercomethetimeandmemoryconstraints,weshuffledthedata in thecombineddatasetandselectedasubsetof it (30%) toperformthe trainingof supervisedADtechniques.Table2depicttheresultsofthebinaryclassificationforthecombineddataset.Basically,inthisapproachallanomalous classes are combined into a single anomaly class to be discriminated from the normalcommunications.Boththeprecisionandaccuracyresultsindicatethatthesupervisedtechniques(KMandNB)performbetterinclassifyinganomalieswhencomparedwithstate-of-the-artunsupervisedtechniques(PCA-SVD and GMM). However, we see that our Data Density based anomaly detection method outperforms other un-supervised techniques. Specifically, the PCA-SVD becomes less accurate in detectinganomaliessinceitmanagestoaccomplishonly17%ofaccuracy.Onthecontrary,theDDtechniqueshows


19

bothahighprecisionandaccuracylevel,i.e.,95%and72%,respectively.Infact,DDhasthebestprecisionofallthemethods,andoverallperformsatparwiththesupervisedtechniques.

Table2-ComparisonofADtechniques(combineddataset)

Method

ADT

Recall

Precision

Accuracy

F-score

G-mean

Supervised

K-means

0.5728

0.8319

0.5680

0.6751

0.6874

NB

0.7692

0.8195

0.9036

0.8595

0.8605

Un-supervised

PCA-SVD

0.2796

0.6472

0.1714

0.2710

0.3331

GMM

0.4416

0.7309

0.4516

0.5583

0.5745

DD

0.7327

0.9508

0.7257

0.8231

0.8307

Inordertofurther investigatetheperformanceoftheADtechniquesin identifyingtheindividualattacks,we created a separate set of a datasets. Each dataset included normal data and data from one of theanomalies.Eachdataset isthenusedasan inputtothedetector.AlldatasetswererunwiththeselectedfourADtechniques.Table3liststheoutputmetricsforeachtypeofanomalyandshowsthatonaveragethelevelofaccuracyofsupervisedtechniquesindetectingspecifictypeofanomalies(i.e.,notusingacombineddataset)becomesless, and better for unsupervised. This is visible when comparing the average number of accuracy forsupervised and unsupervised techniques from Table 2 and Table 3: average accuracy for supervisedtechniques is 0.7353 and that for state-of-the-art unsupervised is 0.3115; and from Table 3: averageaccuracy for supervised techniques is0.6068and that for state-of-the-artunsupervised is0.4988).Giventhediscretenatureofattacks,certainfeaturescanalsoberemovedtoimprovetheaccuracyinsupervisedmode given the fact that certain features are more revealing about attack than others. However, thefeatureselectionandtheiranalysisarebeyondthescopeofthiswork.TheoutputmetricsfortheDDmethodformtable3indicateonceagainthat,ingeneral,DDperformsatparwithsupervisedtechniques.However, there isanoutlier intheexperimentalresults:DDperformspoorlyfor the naïve malicious response injection (NMRI) anomaly. After removing the outlier, however, theaverageaccuracyforDDis0.6231fromtable3results.AnexaminationoftheoutputmetricsforDDintable3,mayexplain the reason for theoutlier.DD,beinganun-supervisedmethod,performsbadlywhen theanomalous data packets are not so different from normal traffic on the network. Specifically, theNMRIanomaly injects only response packets in the network but lacks information about the process beingmonitored.Thus,itisalesspotentattack.Conversely,foramorepotentattacksuchascomplexmaliciousresponseinjection(CMRI),wheretheattackattemptstomasktherealstateofthephysicalprocessbeingcontrolled,andsotheanomalousdatapacketsaremoredifferentfromnormaltraffic,theperformanceofDD improves dramatically, as represented in table 3. This is also the case for other un-supervisedtechniquesaswell.Anexaminationoftheprecisionandrecallresultsrevealstheexactanomalytypesthatarebeingclassified incorrectly.Theprecisionratefordenial-of-service,reconnaissance,MFCIandMSCI isover80%,butthatofNMRIandCMRI,isbelowtheacceptablelevel.Furthermore,someattacktypessuchasMFCIaredetectedwithlowrecallrateandhighprecision.AlsorecallvaluesappeartobelowerinMPCIandMSCIattacks.


20

Table3-PerformancemetricsofADtechniquespertypeofanomalyAttack Scenario ADT # of correct

normal detections

#of correct anomalous detections

#of total predicted anomalies

Recall Precision Accuracy F-score G-mean

NMRI

K-means 1465 5193 9728 0.1849 0.5338 0.1731 0.2614 0.3040 NB 6000 23168 23168 0.8102 1 0.7723 0.8715 0.8788 PCA-SVD 703 15534 20831 0.4510 0.7457 0.5178 0.6112 0.6214 GMM 6000 5011 5011 0.3059 1 0.1670 0.2863 0.4087 DD 6218 12 1794 0.3954 0.0067 0.0015 0.0025 0.0032

CMRI

K-means 3902 4554 12652 0.2013 0.3599 0.1518 0.2135 0.2338 NB 12000 23168 23168 0.8373 1 0.7723 0.8715 0.8788 PCA-SVD 2901 5193 14292 0.1927 0.3634 0.1731 0.2345 0.2508 GMM 181 13639 25458 0.3290 0.5357 0.4546 0.4919 0.4935 DD 6218 5488 7270 0.5565 0.7548 0.4210 0.5405 0.5637

MSCI

K-means 3000 23193 26193 0.7276 0.8855 0.7731 0.8255 0.8274 NB 3000 23193 26193 0.7276 0.8855 0.7731 0.8255 0.8274 PCA-SVD 5837 16520 16683 0.6210 0.9902 0.5507 0.7078 0.7384 GMM 3000 20618 23618 0.6561 0.8730 0.6873 0.7691 0.7746 DD 6218 3950 5732 0.6394 0.6891 0.5000 0.5795 0.5869

MPCI


MFCI

K-means 2000 15516 17516 0.5152 0.8858 0.5172 0.6531 0.6769 NB 4000 13639 13639 0.5188 1 0.4546 0.6251 0.6743 PCA-SVD 2000 16474 18474 0.5434 0.8917 0.5491 0.6797 0.6998 GMM 4000 6807 6807 0.3179 1 0.2269 0.3699 0.4763 DD 6218 2449 4231 0.6719 0.5788 0.5000 0.5365 0.5379

DoS

K-means 2000 15874 15874 0.5586 1 0.5291 0.6921 0.7274 NB 984 24373 25389 0.7924 0.9600 0.8124 0.8801 0.8831 PCA-SVD 1984 16492 16508 0.5774 0.9990 0.5497 0.7092 0.7411 GMM 1501 3181 3680 0.1463 0.8644 0.1060 0.1889 0.3027 DD 6218 1088 2870 0.7179 0.3790 0.5000 0.4312 0.4353

Reconnaissance


4.4 DataDensity(DD)BasedAnomalyDetection

Although, DD performs at par with supervised anomaly detection techniques; it has other distinctadvantagesasanun-supervisedanomalydetectionmethod:Supervisedtechniquesrequireanappropriatesetofdataforthetraining.Thisdatamaysometimesbemissing.ArepresentativeexamplemaybethecaseofanAPT,whereacollectionofzero-dayattacksmaybeusedtoexploitexistingvulnerabilitiesinsystems.Further, in the DD technique the density is computed recursively, so the technique ismemory-less andunsupervised,andthereforesuitableforreal-timecloudenvironments.In this section, we briefly present the DD anomaly detection technique (Shirazi, Simpson, Gouglidis,Mauthe,&Hutchison,2016).ThistechniquecanbeimplementedasaserviceinaCloudenvironment,and


21

thus,canbeusedincritical infrastructurestoprovidepromptly indicationsofanomalousbehaviours.Thetechniqueisbasedontheconceptsofdatadensity,whichusesanon-parametricCauchyfunctionthatcanbeupdatedrecursively.Duetobeingnon-parametric,onlyasmallamountofdata–onlythemeanofalldatasamplesμkandthescalarproductquantityΣkcalculatedatthecurrentmomentintimek–needstobestored in memory and updated. This has significant implications since it allows theoretically an infiniteamount of data (infinitely largemonitoringmetrics) to be processes in near real timewithout having tostorehistoricaldataitself.Letallmeasurablephysicalvariablesformthevectorx ∈ R1bedividedintoseveralclusters.Then,thelocaldensityd3ofclusterΛ,basedonEuclideandistance,isdefinedas:

d3 = 1

1 + 1N3 ||𝑥: −

<=>?@ xAB=||

C

whereN3denotesthenumberofdatasamplesassociatedwithclusterΛ.𝑓>Etransforms i(identifyingavectorcontributingtothecluster)intothedomainofk(identifyingavectorfromthecompleteset).Inthecaseofanomalydetection,xkrepresentsthefeaturevectorwithvaluesfortheinstantk.Itcanbeshown,thatthisformulacanbederivedasanexactquantity:

𝐷: =1

1 + ||𝑥: − 𝜇:||C + 𝛴: − ||𝜇:||C

where𝐷: isthedensitycalculatedforthecurrentdatasample𝑥:,andwhereboth,themean,𝜇: andthescalarproduct,𝛴: canbeupdatedrecursivelyasfollows:

𝜇: = 𝑘 − 1𝑘

𝜇:J@ +1𝑘𝑥:, 𝜇@ = 𝑥@

𝛴: = 𝑘 − 1𝑘

𝛴:J@ +1𝑘||𝑥:||C, 𝛴@ = ||𝑥@||C

Thedataiscollectedcontinuously,inon-linemodeduringthedetectionprocess.Someofthenewdatareinforceandconfirmtheinformationcontainedinthepreviousdata.Other data, however, bring new information, which could indicate a change in operating conditions,developmentofananomalyorsimplyamoresignificantchangeinthedynamicofthesystem.Inordertodetectanomalousbehaviour,thevariable{ΔDOis,then,calculatedasfollows:

𝛥𝐷: = |𝐷: − 𝐷:J@|𝐷:J@isthedensitycalculatedfortheimmediatelypreviousdatasample𝑥:[email protected]𝐷: canalsobecalculated,andcomparedtoindicatewhetherthesystemisinananomalousstate(indicatedwhen𝐷: < 𝐷:)oranormalstate.𝐷: iscalculatedasfollows:


22

𝐷: =𝑘R − 1𝑘R

𝜇S:J@ +1𝑘R𝐷: 1 − 𝛥𝐷: + 𝐷:𝛥𝐷:

wherek counts thenumberof data sampleswhich are read, andks counts thenumberof time steps inwhichthesystemremainsinthesamestatus(normal/anomalous).Sincethecomputationof𝐷: < 𝐷: isbasedentirelyontheconceptofdatadensity,itishighlysuitableandapplicable for real-time anomaly detection in cloud environments. Moreover, the time complexity forcalculatingthenewdensity𝐷: andmeandensity𝐷: fromtheirpreviousvalueisclassifiedasO(n).ThealgorithmforthedatadensityanomalydetectionalgorithmisprovidedinANNEXI.

5 PART2-REVIEWOFSTANDARDS

Standardsforcybersecurityareproceduresusuallylaiddowninreleaseddocumentsormaterialsaimedatendeavouringtoprotectthecyberatmosphereofacompanyorspecificusers5.Thisatmosphereincludesusers,networks,devices,software,processes,informationinstorageortransit,applications,services,andsystemsthatcanbeconnecteddirectlyorindirectlytonetworks.Theobjectiveistodiminishtherisks,aswell as preventing ormitigating cyber-attacks. These resources comprise collections of tools, guidelines,policies,bestpractices, riskmanagementapproaches,securityconceptsandsafeguards,actions, training,assuranceandtechnologies6.In today´s times, there isaprogressiveneedforsecurityand informationassurance (IA)giventhatmanytasksthatwereoncecarriedmanuallyarenowcarriedoutbycomputers.Standardsoncybersecurityhaveexisted for decades and both users and suppliers have collaborated inmany national and internationalforums to sketch the required capabilities, policies, andpractices - generally emerging fromwork at theStanford Consortium for Research on Information Security and Policy (CRISP)7; which was born as aresponse to growing U.S. government concern over the threat of cyber-attacks directed against criticalinfrastructures.Therearemanystandardisationresourcesavailabletoprovidehelponthedevelopmentofcybersecuritysolutionsandassistingtheengineertounderstandandmanagecomplexproblems.Thechallengeisoftentoidentifythecorrectone,whichisnotalwaysaneasytasksincetherearemany,whichoftenoverlaportackledifferentareasofthechallenge.Althoughtheworldofcybersecurityhasmanystandards,onlyfewarespecificallytargetedtowardsthemanagementofcriticalinfrastructure.Inthefollowingsubsections,wehave identified the most relevant cybersecurity standards specifically related to critical infrastructures,Industrial Automation and Control Systems (IACS) and Network and Information security (NIS). This, ofcourse, isnotacomprehensivelist.WehavechosentoelaborateonlygeneralpurposeSCADAstandards.Standardssuchas“OLFGuidelineNo.110”,whicharespecifictotheNorwegianoil&gassecurity,havenotbeenelaborated. Further, a fewSCADA-IACS standardsweredescribed inHyRiMworkpackage2, reportD2.1.Thesetoohavenotbeenelaboratedhere;thecuriousreadermayconsultHyRiMD2.1section6.Thefinalsubsectionofthispart2ofthedocumentincludesalistofgeneralcybersecuritystandards,guidelinesand regulatorydocuments thatalthoughnotbeing specific for critical infrastructures,maybeof interestwhendealingwithriskmanagementandcontrolsystemssecurity.

5 http://www.itu.int/ITU-T/recommendations/rec.aspx?rec=9136 6 https://en.wikipedia.org/wiki/Cyber_security_standards 7 http://fsi.stanford.edu/research/consortium_for_research_on_information_security_and_policy


23

5.1 TheNISTcriticalinfrastructureframework

TheNationalInstituteofStandardsandTechnology(NIST)isafederallyfundedorganisationcoveringmanyfieldsinwhichstandardisationcanbesatisfactorilyapplied.Itcoversawideassortmentofareas,fromtheprovision of the standard US time to the development of measurement techniques for differenttechnologies.The NIST Critical Infrastructure Framework (NIST, 2014), signed in February 2013, was established as aresultofPresidentObama’s(ExecutiveOrder13636)to improveCritical InfrastructureCybersecurity.Thisorder led NIST to work with different stakeholders to create a voluntary framework based on existingstandards,guidelines,andpractices.Themainobjectivewastocontributetothereductioncybersecurityrisksforcriticalinfrastructure.The framework is a simple yet powerful tool to assist critical infrastructure managers to organise andmaterialize their thoughts, and therefore improve the resilience of their operations. It is important toremarkthattheframeworkisbasedonmanyotherexistingstandardssuchastheISO27000series,COBITand ISA 62443 as it attempts to bring them together into amore coherentwhole and therefore enableuserstoimprovethesecurityandresilienceofcriticalinfrastructure.This NIST framework compiles different risk-based guidelines designed to assist organizations on theassessment of their current capabilities and draw a prioritized roadmap toward improved cybersecuritypractices.Itcomprisesdifferentseriesoffunctionsaddressingthekeyareasofcybersecurityreadiness.

• Identify:find,assessandselectassets(physicalordigital),whichrequirespecificattention.• Protect:implementsecuritycontrolsmakingitmoredifficulttoattackkeyassets.• Detect:activelymonitortheassetsinordertodiscoverwhentheyarebeingattacked.• Respond:ensureaswiftandeffectiveresponsetolimitdamagewhilstretainingsufficientforensic

evidencetobeabletounderstandthenatureandscopeoftheattack.• Recover:restoreoperationstotheiroriginalstatewithconfidencethattheattackhasindeedbeen

stopped.

Within each function, there are different categories and sub-categories giving more details of thesecurityobjectivemapped.Thecategoriesofthefivecorefunctionsareshowninthetablebelow:

Table4.Fivecorefunctionsofeffectivecybersecurity(Guinn,2014)Source:PWC

Functions Definition CategoriesIdentify Anunderstandingofhowtomanage

cybersecurityriskstosystems,assets,data,andcapabilities

Assetmanagement,businessenvironment,governance,riskassessment,riskmanagementstrategy

Protect Thecontrolsandsafeguardsnecessarytoprotectordetercybersecuritythreats

Accesscontrol,awarenessandtraining,datasecurity,dataprotectionprocesses,maintenance,protectivetechnologies

Detect Continuousmonitoringtoprovideproactiveandreal-timealertsofcybersecurity-relatedevents

Anomaliesandevents,continuousmonitoring,detectionprocesses

Respond Incident-responseactivities Responseplanning,communications,analysis,mitigation,improvements

Recover Businesscontinuityplanstomaintainresilienceandrecovercapabilitiesafteracyberbreach

Recoveryplanning,improvements,communications


24

Moreover,theframeworkalsoincludesaseriesoftiersthatassisttheevolutionoforganisationstohigherlevelsofmaturity.Thefirsttierlevelis‘partial’andthehighestoneis‘Adaptive’.

Table5.Tiersofcybersecuritymaturity(Guinn,2014)Source:pwc

Tier Levelofmaturity DefinitionTier1 Partial Riskmanagement is adhoc,with limitedawarenessof

risksandnocollaborationwithothersTier2 RiskInformed Risk-management processes and program are in place

butarenotintegratedenterprise-wide;collaborationisunderstoodbutorganizationlacksformalcapabilities

Tier3 Repeatable Formal policies for risk-management processes andprograms are in place enterprise-wide, with partialexternalcollaboration

Tier4 Adaptive Risk-management processes and programs are basedon lessons learned and embedded in culture, withproactivecollaboration

5.2 NISTIR7176.SystemProtectionProfile-IndustrialControlSystems

TheaimoftheSystemProtectionProfileforIndustrialControlSystems(SPP-ICS)istospecifyanintegratedset of requirements for the security of industrial control systems. The list of requirements includesoperating procedures and policies, requirements for information technology based system components,requirementsforinteroperabilityamongsystemelementsandinterfaces,andrequirementsforprotectionof the systemand thephysicalenvironment8.TheSPP-ICSoffersan integratedviewof the requirementsand thereforegives special emphasis todecompositionof security functionality andallocationof explicitsecurity functions to sub-components of the integrated system. Equally, the composability ordecompositionofthesecurityfunctionalityisalsoreflected.Theaimofthisaspectofanalysisanddesignistodelineatetherequirementofsecurityatthelowestlevelofsubsystemsorsystemelementsandtoretainthenecessary levelofassuranceandsecurity functionality forthe integratedsystemasawhole.TheSPPhas been written for a generic ICS as a high-level report of requirements. It provides a basis for moredetailedstatementsofrequirementsforICSwithfocusonaspecificindustry,company,orelement9

5.3 NISTIR7628.GuidelinesforSmartGridCybersecurity

TheNISTGuidelinesforSmartGridCyberSecurity includeshigh-levelsecurityrequirements,aframeworkfor assessing risks, an estimation of privacy issues at personal residences, and further information forbusinessesandorganizationstoapplyastheydesignstrategiestodefendthemodernizingpowergridfromattacks, cascading errors, malicious code, and other kind of threats. The advisory report advocates alayered—or "defense indepth"—approach to securityby recommending implementingmultiple levelsofsecurity.TheguidelinesareamajoroutputofNIST-coordinatedeffortsto identifyanddevelopstandardsrequired to transform the nation's aging electric grid into amodern and advanced, digital infrastructurewithtwo-waycapabilitiesforexchanginginformation,monitoringequipmentanddistributingenergy.Theguidelinesinclude137interfaces—pointsofdataexchangeorothertypesofinteractionswithinorbetweendifferent SmartGrid systems and subsystems. These are allocated to 1 ormore of 22 categories on thebasisofcommonorsimilarcharacteristicsorfunctionalities.Asawhole,thereportincludes189high-levelsecurity requirementsapplicable toboth theentire SmartGridand toparticular componentsof thegridandrelatedinterfacecategories.

8 http://nvlpubs.nist.gov/nistpubs/ir/2014/NIST.IR.7628r1.pdf 9 https://scadahacker.com/library/Documents/Standards/NIST%20-%20System%20Protection%20Profile%20Industrial%20Control%20Systems.pdf


25

Thereportincludes10:• Depictionoftheriskassessmentprocessappliedtoidentifytherequirements;• Discussionof technicalcryptographicandkeymanagement issuesacross thescopeofSmartGrid

systemsanddevices;• Recommendations for managing privacy risks and challenges relating to electric vehicles and

personalresidences;• OutlineoftheprocessthattheCSWGdevelopedtodefineifexistingornewstandardsthatenable

Smart Grid interoperability also satisfy the high-level security requirements included in theguidelines;

• Researchneedspresentedinasummary.

5.4 ISA/IEC-62443

ISA/IEC-62443 (formerly ISA-99) includes a range of standards, technical reports, and associatedinformation to thesecurityof IndustrialAutomationandControlSystems (IACS).ThestandardcomprisesthedefinitionofproceduresforimplementingelectronicallysecureIACSandappliestoend-users(i.e.assetowner), system integrators, security practitioners, and control systems manufacturers responsible formanufacturing, designing, implementing/managing IACS. For the elaboration of the standard, theInternational Society of Automation (ISA) organised differentworking groups, each addressing a specificaspectofIACSsecurityincludingpatchmanagement,wirelesssystemssecurity,theconvergenceofsafetyand security and technical requirements at the system and component levels. The final proposeddocumentswere11:

• ANSI/ISA-99.01.01-2007 ‘Security for Industrial Automation and Control Systems: Concepts,TerminologyandModels’.Theaimofthisfirstdocumentoftheseriesistobuildabaseofreferencefortherestoftheseries.

• ANSI/ISA-TR99.01.02-2007‘SecurityTechnologiesforManufacturingandControlSystems’.Theaimof this technical report is to reflect the different developments in the market. It includes thedescriptionofhowtoimplementandconfiguredifferentsecuritytoolsinsideICS.Thisdocumentisreviewedregularly.

• ANSI/ISA-99.02.01-2009 ‘Establishing an Industrial Automation and Control Systems SecurityProgram’. The aim of this document is to describe the required elements to implement acybersecurity management system and to provide a guide with the compilation of all therequirementsforeachelementmakingupthesystem.

• ANSI/ISA-99.02.02 ‘Operatinganindustrialautomationandcontrolsystemsecurityprogram’.Thefocusofthisdocument istheoperation,designandimplementationofthesecurityprogram.Theoperation includes different features such as the definition of metrics for measuring programeffectiveness.

• ANSI/ISA–99.03.xx‘Technicalsecurityrequirementsforindustrialautomationandcontrolsystems’.Thispartofthestandardincludedtheoryregardingdefinitionsofindustrialautomationandcontrolsystemcharacteristics thatdistinguish them from IT systems. Froma securitypoint-of-view, theydefinethesecurityrequirementsthatareuniquetothesesystems.

TheISA-62443standardsandtechnicalreportsaredividedintofourgeneralblockscalledGeneral,PoliciesandProcedures,System,andComponent.InError!Referencesourcenotfound.,allplannedandpublishedISA-62443workproductsforIACSSecurityarepresented:

10 https://www.nist.gov/news-events/news/2010/09/nist-finalizes-initial-set-smart-grid-cyber-security-guidelines-0 11 https://www.certsi.es/en/blog/iec62443-evolution-of-isa99


26

Figure5-ISA-62443workproductsforIACSSecurity12

Source:ISA99StandardsCommittee

5.5 IEC62443ConformityAssessmentProgram

The ISASecurityCompliance Institute (ISCI)13 runs the first conformityassessment scheme for IEC62443IACScybersecuritystandards.ThisprogramcertifiesCommercialOff-the-shelf(COTS)industrialautomationandcontrolsystems(IACS)productsandsystems,addressingsecuringtheIACSsupplychain.

• ISO17065andGlobalAccreditation.TheISASecure62443conformityassessmentschemeisanISO17065programwithcertificationsbodiesorlabsindependentlyaccreditedbyANSI/ANAB,JABandother global ISO 17011 accreditation bodies; and meeting the requirements established byISO17025toguaranteethereliableapplicationofcertificationrequirementsandrecognizedtools.

• Certifications.CanbefoundundertheISASecure®brand:- ISASecure - EmbeddedDeviceSecurityAssurance (EDSA): certificationof IACSproducts to

theIEC62443-4-2IACScybersecuritystandard.- ISASecure-SystemSecurityAssurance(SSA):certificationofACSsystemstotheIEC62443-

3-3IACScybersecuritystandard.- Secure Development Lifecycle Assurance (SDLA): certification of IACS development

organizations to the IEC 62443-4-1 cybersecurity standard, assuring that a given supplierhas institutionalized cybersecurity into their product development practices. ThroughMutual Recognition Arrangements (MRA)with IAF, ILAC and others, the accreditation of

12 https://en.wikipedia.org/wiki/Cyber_security_standards 13 http://www.isasecure.org


27

theISASecurelabsbytheISA17011accreditationbodiesguaranteesthatthecertificationsissuedbyanyoftheISASecurelabsarerecognizedallaroundtheworld.

• Chemicals, Oil and Gas Industries. ISCI development processes include maintenance policies toguarantee that the ISASecure certifications are in line with the evolution of the IEC 62443standards. On the one hand, the IEC 62443 standards are created to address technicalcybersecurity requirements of a cross-section of process industries in a horizontal way. On theotherhand, the ISASecure scheme’s certification requirementshavebeen vettedby agents fromtheindustriesofchemicals,oil,andgasandarereflectiveoftheirneedsintermsofcybersecurity.

• TestToolRecognition. It isaprocessused to recognize test toolsand thereforemakesure thatthese tools comply with the requirements necessary and sufficient to implement all requiredproducttestsandthattestresultswillbeconsistentamongtherecognizedtools.

5.6 CyberSecurityStrategyoftheEuropeanUnion

TheCyberSecurityStrategyoftheEuropeanUnion(EUCSS)waspublishedbytheEConFebruary4,2013(European,2013b).Thestrategyconsistsinaharmonizedframeworkfortheprogressionofthreedifferentareasofcybersecurity,whichuntil recentlyhadbeenevolving independently.Thus, itwasrecognizedbytheECthatbringingdifferentcommunitiestogethertoimprovetheapproachtocybersecurityacrosstheEUwas a requirement. Given this identified need, the EC laid the foundations for amore synchronizedapproach.Moreover,aDirectiveonNetworkandInformationSecurity(NIS)isalsoincludedinthestrategy.This directive requires Member States to have minimum NIS capabilities in place, the cooperation andinformationexchangewithinadedicatednetwork,anddemandtheprivatesectortoadoptNISenhancingactions.Following,wepresentthemainstatements:

• The EU reiterates the significance of ‘commercial and non-governmental entities, involved in theday-to-daymanagementofInternetstandards.’

• ‘Aprimefocusshouldbetocreateincentivestocarryoutappropriateriskmanagementandadoptsecurity standards and solutions, as well as possibly establishing voluntary EU-wide certificationschemesbuildingonexistingschemesintheEUandinternationally.’

• The EC will give support to the implementation of ‘security standards and assist with EU-widevoluntarycertificationschemesintheareaofcloudcomputing.’

Moreover,somerecommendationsforpublicandprivatestakeholdersarealsoprovided.Specifically,theECencouragedthemto:

• ‘Stimulate thedevelopmentandadoptionof industry-led security standards, technical normsandsecurity-by-design and privacy-by-design principles by ICT product manufacturers and serviceproviders, including cloudproviders;’ andequip ‘newgenerationsof softwareandhardwarewithstronger,embedded,anduser-friendlysecurityfeatures.’

• ‘Develop industry-led standards for companies' performance on cyber security, and improve theinformationavailabletothepublicbydevelopingsecuritylabelsorkitemarkshelpingtheconsumernavigatethemarket.’

Another important part of the Cyber Security Strategy is the proposal for a Network and InformationSecurity(NIS)Directive.ThisDirectiveaskstheMemberStatestosupportstandardizationintheareaofNIS(European,2013a):

• ‘Given the global nature of NIS problems, there is a need for closer international cooperation toimprovesecuritystandardsandinformationexchange,andpromoteacommonglobalapproachtoNISissues.’

• ‘Standardization of security requirements is a market-driven process. To ensure a convergentapplicationofsecuritystandards,MemberStatesshouldencouragecomplianceorconformitywith


28

specified standards to ensure a high level of security at the EU level. To this end, it might benecessarytodraftharmonizedstandards.’

Additionally,article16statesthecontentprovidedbelow:

• ‘Member States shall encourage the use of standards and/or specifications to networks andinformationsecurity.’

• ‘TheCommissionshalldrawup,bymeansofimplementingactsalistofspecificstandards.ThelistshallbepublishedintheOfficialJournaloftheEuropeanUnion.’

5.7 CyberSecurityCoordinationGroup(CSCG)

In 2011, The StandardsDevelopmentOrganizations ETSI, CEN andCENELECproduced theCEN-CENELEC-ETSI ‘CyberSecurityCoordinationGroup’asananswer toa requestof theEC.Theaimof thegroup is todeliverstrategicadviceinallissuesrelatedtoNetworkandInformationSecurity(NIS),ITsecurity,andcybersecurity.Inshort,themaingoalsofthegroupareto:

• CreateaEuropeanstandardizationroadmapintheareasofsecurity• Become the principal contact point for all the inquiries made by EU institutions in relation to

standardizationissues.• DesignandputforwardtoECastrategybasedonthecooperationbetweentheEUandtheUSfor

thecreationofaframeworkrelatedtocybersecuritystandardization.

Nowadays,themembersofCSCGareworkingonthecreationofawhitepaperaddressedtotheEC,withstrategicguidanceonprioritiesforR&DofEUfundedresearchinthisarea,andactionsfortheoptimizationofEUresearchthroughdirectivesforcybersecuritystandardization.

5.8 GuidetoIncreasedSecurityinIndustrialControlSystems14.

The aim of this document is to give support and increase awareness of the need for more security inindustrialcontrolsystems.ThedocumentwaselaboratedbytheMSBSwedishCivilContingenciesAgencyand first published in October 2008. The guide is periodically revised. The document delivers relevantrecommendations on security in ICS and includes tips on how to find additional useful information. It isworthhighlightingthatthisguidefocusesonelectronicsecurityinindustrialcontrolsystemsbutdoesnotprovidespecificadviceonITsecuritymatters.Theguideconsistsofthreeparts:

• PartA.Prerequisitesandgeneralrecommendations.Targetaudience:peopleworkingwithsecurityissuesatthemanagementlevel.

• Part B. Recommendations and guidelines. Target audience: people working with security inindustrialcontrolsystemsinpractice.

• PartC.Referencelistwithcomments.

5.9 CyberSecurityAssessmentsofIndustrialControlSystems.Agoodpracticeguide15.

The guide was elaborated by the Centre for the Protection of National Infrastructure (CPNI) with thepurpose of supporting asset owners to maximise the return on their investment when carrying outassessmentsof their IndustrialControl Systems.The final aim is toeducateassetownerson thegeneralprocessofacybersecuritytestandprovideinsightonspecifictestingmethodssoownerslearntoprescribeacustomassessmentthatwillmaximisetheoutputoftheirtestingbudget.

14 https://www.ccn-cert.cni.es/publico/InfraestructurasCriticaspublico/Suecia-scada_guide.pdf 15 http://www.cpni.gov.uk/documents/publications/2011/2011008-infosec-cyber_security_assessment_of_ics_viewpoint.pdf?epslanguage=en-gb


29

ThedocumentdeliversanoutlineoftheassessmentprocesstohelpusersunderstandhowtoexecuteanICS cyber security assessment. This guide also covers the planification process of an ICS cyber securityassessment,includinghowtochoosetestingareas.Thetestplanstipulatesthecorrectamountofdetailtomeet the necessities of the asset owner while retaining the flexibility to use all the skills of the teaminvolvedintheassessmenttest.Thedetailsoftheactualtestingprocessinthisguidefamiliarisetheassetownerwiththestepsandreasonsbehindthetestingprocess.ThereportingprocessfortheassessmentofICScybersecurityisalsoincludedintheguideOn top of explaining actual security testing, the advantages and disadvantages of different vulnerabilitytestingmethods for ICSsarealsoconsideredsothat testscanbetailoredtotheparticularitiesof the ICSandneedsofeachcompany.

5.10 ENISAcontributionstostandardisation

ENISAisthecentreofexpertiseforcybersecurityinEurope.Asstatedinitsfoundingregulation,oneofthekeyobjectivesofENISAisto‘trackthedevelopmentofstandardsforproductsandservicesonNetworkandInformation security’16. Since 2009, ENISA has been working on the identification and analysis of thedifferent efforts carried out by standardization bodies such as CEN, CENELEC, ISO, ETSI, ITU.One of themainresultsofthisactionwasareportincludingareviewofthestateofstandardization.Thetitleofthereportis‘Gapsinstandardisationrelatedtoresilienceofcommunicationnetworks’(Slawomiretal.,2009),whichbythat timehadnotbeenaddressedbythekeyStandardsDevelopingOrganizations (SDOs)otherthanasguidanceformanagementprocesses.Thereportsummarizeskeyfindingscoveringtheimportanceofdefiningresilienceinthecontextofstandardization,theidentificationofthemainactionsundertakenbySDOs inthefieldofsecurity,andthe identificationof furtherworkrequired inkeyareas.Thereportalsohighlightedthatastandardtaxonomyforcybersecurityidentifyingtheroleofresiliencewasmissing.Nextto this first report, ENISA released a second one including an ontology of resilience alongside andembedding a taxonomyof resilience. Two tools for understanding resilience as a network design target,andtheresultsof thosetoolswhenappliedtoresiliencewere included in thereport.Theprovidedtoolswereclassificationusing taxonomy,andrelationshipmodellingusingontologywith taxonomyat itscore.These results were taken on board by the Telecommunication and Internet converged Services andProtocols forAdvancedNetworking(TISPAN)groupofETSI forpotential future inclusion inastandard. InadditiontotheENISAworkonspecificareas,theyhavealsoempoweredcooperationamongrelevantEUactors (industry,EUorganizations,SDOs),with theaimofovercoming theweaknessesof standardizationefforts.AneffectivewaytomeetthisobjectivecouldbethroughthepromotionofbestpracticesamongEUMember States. ENISA has a specific role on this issue and acts as an intermediary between public andprivate sectors as well as interfacing with the SDOs. Specifically, working collaborations with SDOs andspecific working groups (WG), such as ISO SC27, ETSI, CEN and CENELEC, and ITU SG17 have beenestablishedbyENISA.ArecentENISAstudy(Weiss,2010)concludesthatISO27002forinformationsecuritymanagementisthemostadoptedstandard,followedbyNERC-CIP,NISTSP800-82,andfinallyISA99.

5.11 Otherstandards

This section includes a list of other standards, guidelines and regulatory documents that although notdescribedinprevioussections,couldbeof interestforanycritical infrastructurestakeholderdealingwithriskmanagementandcontrolsystemssecurity.

• ISO 15408 – The Common Criteria. Multipart standard including a set of criteria for a rigoroustechnicallybasedevaluationofICTproducts.

• ISO/IEC27000series.Securitypoliciesandsecuritymanagementstrategies.

16 https://www.enisa.europa.eu/?came_from=http%253A//www.enisa.europa.eu/activities/res-old/technologies/std/std


30

• NIST Special Publications (SP). Subseries to publish computer, cyber, information security andguidelines,recommendationsandreferencematerials.

• ETSICyberSecurityTechnicalCommittee(TCCYBER).ResponsibleforthestandardisationofCyberSecurityinternationallyandforprovidingacentreofrelevantexpertiseforotherETSIcommittees.

• StandardofGoodPractice(SoGP).PublishedbytheInformationSecurityForum(ISF).• RFC 2196 - Site Security Handbook. Published by Internet Engineering Task Force [13] for

developingsecuritypoliciesandproceduresforinformationsystemsconnectedontheInternet.• NorthAmericanElectricReliabilityCorporation(NERC).• IASME.UK-basedstandardforinformationassuranceatSMEs.• ISO31000riskmanagementprocess.Providesprinciples,aframeworkandaprocessformanaging

riskandaimsatimprovingtheidentificationofopportunitiesandthreats,andeffectivelyallocatingandusingresourcesforrisktreatment(I.ISO,2009).

• ISO31010provides informationon risk assessment concepts, processes and the selectionof riskassessmenttechniques(ISO,2009).

• ISO/IEC19790-Securityrequirementsforcryptographicmodules(similartoNISTFIPS140-2).• ISO/IECTR/19791-Securityassessmentofoperationalsystems.• ISO/IEC21827-SSEcapabilitymaturitymodel(SSE-CMM®).• ISO/IEC17799-CodeofpracticeforinformationsecurityManagement.• IEC62351–Anumbrellasetofstandardsforcommunicationsecurityinenergynetworks.• COBIT-ControlobjectivesforinformationandrelatedtechnologyincludinganITwidegovernance

model, which provides guidancewhen looking at software development, system operations andreporting.

• Configuring&managingremoteaccessforindustrialcontrolsystems.Agoodpracticeguide.• Goodpracticeguide-ProcessControlandSCADASecurity.• FirewalldeploymentforSCADAandprocesscontrolnetworks.Agoodpracticeguide.• ProcessControlDomain(PCD)–SecurityRequirementsforVendors.• VDI/VDE2182Series.• OLFGuidelineNo.104.Informationsecuritybaselinerequirementsforprocesscontrol,safetyand

supportICTsystems.• OLF Guideline No. 110. Implementation of information security in Process Control, Safety and

SupportICTSystemsduringtheengineering,procurementandcommissioningphases.• ProtectionProfilefortheGatewayofaSmartMeteringSystem.• SecurityProfileforAdvancedMeteringInfrastructure.• FieldDeviceProtectionProfileforSCADASystemsinMediumRobustnessEnvironments.• AGAReportNo.12.CryptographicProtectionofSCADACommunications.• API1164,PipelineSCADASecurity.• 21StepstoimproveCyberSecurityforSCADAsystems.• CatalogueofControlSystemsSecurity:RecommendationsforStandardsDevelopers.• SecuringyourSCADAandIndustrialControlSystems.

Asregardssecurityanalysismethodologies,Behniaetal.(Behnia,Rashid,&Chaudhry,2012)pointouttherangeofdifferencesbetweenthemostrelevantones,includingOCTAVE17,ISRAM(Karabacak&Sogukpinar,2005),CORAS(denBraberetal.,2003).TheOperationallyCriticalThreat,AssetandVulnerabilityEvaluation(OCTAVE)methodcanbeusedforidentifyingandmanaginginformationsecurityrisks.Itcontainsmethods,techniquesandtoolsforanasset-drivenevaluationapproach,focusingonsecuritypracticesandstrategicissues,andself-direction.Similarly,MageritisariskanalysisandmanagementmethodologythathasbeendevelopedinSpain(Crespo,Gómez,Candau,&Manas,2006).InasimilarmannertoOCTAVE,itisdrivenbyananalysisoftheassetsthatareassociatedwithanorganisation.TheMageritmethodologyhasformedthe

17 http://www.cert.org/octave/


31

basisforaninformationsecuritymanagementmethodthathasbeendevelopedaspartoftheEU-fundedPRECYSEproject18,whichisfocusedonindustrialcontrolsystemsecurity.

6 CONCLUSIONS

The investigation of threats in critical infrastructures using the OTI viewpoints may improve ourunderstanding of potential metrics stemming from technical and non-technical threats. Our work inprogress includes the investigation and identification of metrics required to be monitored in criticalinfrastructures as utility networks. The incorporation of these metrics in a resilience-aware framework,basedonourD2R2+DRstrategy,notonlyprovidesaneffectiveapproachforthedetectionofanomalies,butalso for invoking remediation and recovery processes to ensure the orderly functioning of a system. Bymeans of an evaluationof the proposed framework using information extracted froma Europeanutilitynetwork,we have demonstrated its applicability. In the future,we plan to deploy collectors formetricsrelated to the OTI viewpoints and to investigate further remediation and recovery processes that areapplicableinthecontextofcriticalinfrastructures.In thiswork, theperformanceofvariousADtechniquesappliedtoSCADAcommunication isevaluated intermsoftheirabilitytoidentifyvariousattacks.WehaveanalysedthecommunicationbetweenanRTUandMTU in a gas pipeline system. The data in our evaluation were developed by the Mississippi StateUniversity, and include artefacts of benign RTU transactions and various attack transactions generatedspecificallyforconductingresearchintheareaofcriticalinfrastructuresprotection.WehaveanalysedtheaccuracyoffourADtechniquesincorrectlyidentifyinganomaliesusingasetofstatisticalfeatures.Resultsfromourexperimentsindicatethatdetectionratedifferswithrespecttothetypeoftheanomalyandtherunning mode of the applied AD technique. Specifically, AD techniques that run in supervised modeappeared to perform better; however, a dataset to train a technique is not always possible to have.Therefore,weargue that there isaneed fordevelopinga robust,andpreferably real-timeADtechniquethat can work in unsupervised mode and have a better detection accuracy. The configuration modes,normalization techniques, etc. are yet more variables to consider when it comes to applying themoperationally.Withregardtothesecuritystandards,wehavereviewedtensecuritystandardisationapproachesrelatedtocritical infrastructures and Industrial Automation and Control Systems (IACS), namely the NIST criticalinfrastructure framework, TheNISTIR 7176 - System Protection Profile, theNISTIR 7628 - Guidelines forSmart Grid Cybersecurity, the ISA/IEC-62443, the IEC 62443 Conformity Assessment Program, the CyberSecurity Strategy of the European Union, the Cyber Security Coordination Group (CSCG), The Guide toIncreased Security in Industrial Control Systems, The Cyber Security Assessments of Industrial ControlSystems,andtheENISAcontributionstostandardisation.Keepinginmindthattechnologyenvironmentandcybersecurity threats are continuously evolving, the available standardsmust also continue to adapt todesign,develop,implement,maintainandimprovethecybersecuritypracticeswithincriticalinfrastructureenvironments.Inthefinalpartofthestandardsreview,generalcybersecuritystandardsarelistedprovidinginthiswayanoverviewofallexistingsecuritypoliciesandstandardisationapproaches,whichalthoughnotbeingspecificforutilities,maybeusefulwhendealingwithriskmanagementandcontrolsystemssecurity.

18 https://www.precyse.eu


32

REFERENCES

Behnia,A.,Rashid,R.A.,&Chaudhry,J.A.(2012).Asurveyof informationsecurityriskanalysismethods.SmartCR,2(1),79-94.

Bezdek, J. C., Ehrlich, R., & Full,W. (1984). FCM: The fuzzy c-means clustering algorithm. Computers &Geosciences,10(2-3),191-203.

Carcano,A.,Fovino, I.N.,Masera,M.,&Trombetta,A.(2008).ScadaMalware,aproofofConcept.PaperpresentedattheInternationalWorkshoponCriticalInformationInfrastructuresSecurity.

Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: A survey. ACM computing surveys(CSUR),41(3),15.

Chen,P.-Y.,Cheng,S.-M.,&Chen,K.-C.(2012).Smartattacksinsmartgridcommunicationnetworks.IEEECommunicationsMagazine,50(8),24-29.

Crespo,F.L.,Gómez,M.A.,Candau,J.,&Manas,J.(2006).Magerit–version2,methodologyforinformationsystems risk analysis and management, book I–the method. Ministerio de administracionespúblicas.

Damiani,E.(2009).Compositeintrusiondetectioninprocesscontrolnetworks.Dastjerdi,A.V.,Bakar,K.A.,&Tabatabaei,S.G.H. (2009).Distributed intrusiondetection incloudsusing

mobile agents. Paper presented at the Advanced Engineering Computing and Applications inSciences,2009.ADVCOMP'09.ThirdInternationalConferenceon.

Dell.(2015).DellAnnualSecurityThreatReport2015.Retrievedfromhttps://software.dell.com/docs/2015-dell-security-annual-threat-report-white-paper-15657.pdf

den Braber, F., Dimitrakos, T., Gran, B. A., Lund, M. S., Stølen, K., & Aagedal, J. Ø. (2003). The CORASmethodology:model-basedriskassessmentusingUMLandUP.UMLandtheUnifiedProcess,332-357.

ESET. (2016). The security review: BlackEnergy, Internet Explorer and Fitbit. Retrieved fromhttp://www.welivesecurity.com/2016/01/18/security-review-blackenergy-internet-explorer-fitbit/

European,C. (2013a).ConcerningMeasurestoEnsureaHighCommonLevelofNetworkand InformationSecurityAcrosstheUnion.

European,C.(2013b).CybersecurityStrategyoftheEuropeanUnion:AnOpen,SafeandSecureCyberscape.Retrievedfromhttp://eeas.europa.eu/policies/eu-cyber-security/cybsec_comm_en.pdf

Falliere,N.,Murchu,L.O.,&Chien,E.(2011).W32.stuxnetdossier.Whitepaper,SymantecCorp.,SecurityResponse,5,6.

Gamer, T. (2009). Anomaly-based identification of large-scale attacks. Paper presented at the GlobalTelecommunicationsConference,2009.GLOBECOM2009.IEEE.

Gao,W., Morris, T., Reaves, B., & Richey, D. (2010).On SCADA control system command and responseinjection and intrusion detection. Paper presented at the eCrime Researchers Summit (eCrime),2010.

Garfinkel, T., & Rosenblum,M. (2003).A VirtualMachine Introspection Based Architecture for IntrusionDetection.PaperpresentedattheNDSS.

Gouglidis,A.,Green,B.,Hutchison,D.,Arias,A.M.,Alshawish,A.,Fileppo,F.,. . .Solar,A.(2015).HyRiMDeliverable 1.1 Report on (cyber) risk trends in utility network operator requirements. Retrievedfrom

Gouglidis,A.,Shirazi,S.N.U.H.,Simpson,S.,Smith,P.,&Hutchison,D.(2016).Amulti-levelcollaborativeframeworkforcriticalinfrastructuresresilience.ICT2016.

Green, B., Prince, D., Busby, J., & Hutchison, D. (2015). The impact of social engineering on IndustrialControlSystemsecurity.PaperpresentedattheProceedingsoftheFirstACMWorkshoponCyber-PhysicalSystems-Securityand/orPrivaCy.

Guinn,J.(2014).WhyyoushouldadopttheNISTCybersecurityFramework.RetrievedfromHollnagel,E.,Woods,D.D.,&Leveson,N.(2007).Resilienceengineering:Conceptsandprecepts:Ashgate

Publishing,Ltd.


33

ICS-CERT.(2015). IncidentResponse/VulnerabilityCoordinationin2014.Retrievedfromhttps://ics-cert.us-cert.gov/sites/default/files/Monitors/ICS-CERT_Monitor_Sep2014-Feb2015.pdf

ISO.(2009).IEC31010:2009.Riskmanagement-Riskassessmenttechniques.ISO, I. (2009). 31000: 2009 Risk management–Principles and guidelines. International Organization for

Standardization,Geneva,Switzerland.Karabacak, B., & Sogukpinar, I. (2005). ISRAM: information security risk analysis method. Computers &

Security,24(2),147-159.Laskov, P., Düssel, P., Schäfer, C., & Rieck, K. (2005). Learning intrusion detection: supervised or

unsupervised?PaperpresentedattheInternationalConferenceonImageAnalysisandProcessing.Lazarevic, A., Ertöz, L., Kumar, V., Ozgur, A., & Srivastava, J. (2003). A Comparative Study of Anomaly

DetectionSchemesinNetworkIntrusionDetection.PaperpresentedattheSDM.Lee, J.-H., Park,M.-W., Eom, J.-H.,& Chung, T.-M. (2011).Multi-level intrusion detection systemand log

management in cloud computing. Paper presented at the Advanced Communication Technology(ICACT),201113thInternationalConferenceon.

Marnerides, A., James, C., Schaeffer-Filho, A., Sait, S. Y., Mauthe, A., & Murthy, H. (2011). Multi-levelnetwork resilience: traffic analysis, anomaly detection and simulation. ICTACT Journal onCommunicationTechnology,SpecialIssueonNextGenerationWirelessNetworksandApplications,2(2),2.

Marnerides,A.K., Smith,P., Schaeffer-Filho,A.,&Mauthe,A. (2015).Powerconsumptionprofilingusingenergytime-frequencydistributionsinsmartgrids.IEEECommunicationsLetters,19(1),46-49.

Marnerides,A.K.,Watson,M.R.,Shirazi,N.,Mauthe,A.,&Hutchison,D.(2013).Malwareanalysisincloudcomputing: Network and system characteristics. Paper presented at the 2013 IEEE GlobecomWorkshops(GCWkshps).

Marton, I., Sánchezb, A., Carlosa, S., & Martorella, S. (2013). Application of data driven methods forconditionmonitoringmaintenance.CHEMICALENGINEERING,33,301-306.

Mdhaffar,A.,Halima,R.B., Jmaiel,M.,&Freisleben,B. (2014).CEP4CMA:multi-layercloudperformancemonitoringandanalysisviacomplexeventprocessingNetworkedSystems(pp.138-152):Springer.

Morris,T.,&Gao,W. (2014). IndustrialControlSystemTrafficDataSets for IntrusionDetectionResearch.PaperpresentedattheInternationalConferenceonCriticalInfrastructureProtection.

Morris, T. H., Thornton, Z., & Turnipseed, I. Industrial Control System Simulation and Data Logging forIntrusionDetectionSystemResearch.

NIST.(2014).FrameworkforImprovingCriticalInfrastructureCybersecurity.Obregon, L. (2015). Secure Architecture for Industrial Control Systems. Retrieved from

https://www.sans.org/reading-room/whitepapers/ICS/secure-architecture-industrial-control-systems-36327

Pannu, H. S., Liu, J., & Fu, S. (2012). Aad: Adaptive anomaly detection system for cloud computinginfrastructures. Paper presented at the Reliable Distributed Systems (SRDS), 2012 IEEE 31stSymposiumon.

Sharma,B.,Jayachandran,P.,Verma,A.,&Das,C.R.(2013).CloudPD:Problemdeterminationanddiagnosisin shared dynamic clouds. Paper presented at the 2013 43rd Annual IEEE/IFIP InternationalConferenceonDependableSystemsandNetworks(DSN).

Shirazi,S.N.U.H.,Simpson,S.,Gouglidis,A.,Mauthe,A.U.,&Hutchison,D.(2016).Anomalydetectioninthe cloud using data density. Paper presented at the IEEE International Conference on CloudComputing.

Simpson, S.,Marnerides,A.K.,Watson,M.,Mauthe,A.,&Hutchison,D. (2014).Assessing the impactofintra-cloud live migration on anomaly detection. Paper presented at the Cloud Networking(CloudNet),2014IEEE3rdInternationalConferenceon.

Slawomir,G.,Panagiotis,S.,Demosthenes,I.,Scott,C.,Charles,d.C.,Adrian,M.,&Salvatore,D.A.(2009).Gaps in standardisation related to resilience of communication networks. Retrieved fromhttps://www.enisa.europa.eu/publications/archive/gapsstd

Slay, J., &Miller, M. (2007). Lessons learned from themaroochy water breach. Paper presented at theInternationalConferenceonCriticalInfrastructureProtection.


34

Sterbenz, J. P., Hutchison, D., Çetinkaya, E. K., Jabbar, A., Rohrer, J. P., Schöller,M.,& Smith, P. (2010).Resilience and survivability in communication networks: Strategies, principles, and survey ofdisciplines.ComputerNetworks,54(8),1245-1265.

Thonnard,O.,&Dacier,M. (2008). A framework for attack patterns' discovery in honeynet data. digitalinvestigation,5,S128-S139.

Wang, C. (2009). Ebat: online methods for detecting utility cloud anomalies. Paper presented at theProceedingsofthe6thMiddlewareDoctoralSymposium.

Wang, C., Viswanathan, K., Choudur, L., Talwar, V., Satterfield, W., & Schwan, K. (2011). Statisticaltechniques for online anomaly detection in data centers. Paper presented at the 12th IFIP/IEEEInternationalSymposiumonIntegratedNetworkManagement(IM2011)andWorkshops.

Watson, M. R., Marnerides, A. K., Mauthe, A., & Hutchison, D. (2016). Malware detection in cloudcomputinginfrastructures.IEEETransactionsonDependableandSecureComputing,13(2),192-205.

Wei, D., Lu, Y., Jafari, M., Skare, P. M., & Rohde, K. (2011). Protecting smart grid automation systemsagainstcyberattacks.IEEETransactionsonSmartGrid,2(4),782-795.

Weiss,J.(2010).Protectingindustrialcontrolsystemsfromelectronicthreats:MomentumPress.


35

ANNEXI:DATADENSITYBASEDANOMALYDETECTIONALGORITHM(SHIRAZIETAL.,2016)