continuous forensic analytics – issues and answers 1 april 14, 2015 start time: 9am us pacific /12...

Continuous Forensic Analytics – Issues and Answers


1

April 14, 2015Start Time: 9am US Pacific /12 noon US Eastern/ 5pm

London Time

TT

Sponsored by:Sponsored by:

2

#ISSAWebConf

WelcomeConference Moderator

WelcomeConference Moderator

3

April 14, 2015Start Time: 9am US Pacific

12pm US Eastern/5pm London Time

Director of Product Management, Symantec NOVA Chapter, ISSA Web Conference Committee

Matthew Mosley

Speaker IntroductionSpeaker Introduction

• Dipto Chakravarty- Altamira, Open Source Security expert

• Tyrone Wilson- Novetta Cyber Security Senior Analyst

• Remember to type in your question in the Chat area of your screen. You may need to click on the double arrows to open this function.

• Dipto Chakravarty- Altamira, Open Source Security expert

• Tyrone Wilson- Novetta Cyber Security Senior Analyst

• Remember to type in your question in the Chat area of your screen. You may need to click on the double arrows to open this function.

4

Continuous Forensic Analytics Issues and Answers

Continuous Forensic Analytics Issues and Answers

Dipto [email protected]

April 14, 2015

Dipto [email protected]

April 14, 2015

mailto:[email protected]


Topics Topics

• CFA – a new toolset emerged to accelerate the IR process and respond to threats with agility

• Forensics 101

• Analytics 101

• Continuous FA

• Call to action

• CFA – a new toolset emerged to accelerate the IR process and respond to threats with agility

• Forensics 101

• Analytics 101

• Continuous FA

• Call to action

6

Forensics 101Forensics 101

STEP 1:Preparation

Identifies the purpose and resources

STEP 2: Acquisition

Pinpoints the sources of evidence

STEP 3: Analysis

Extracts, collects and analyze evidence

STEP 4: Reporting

Documents and presents the evidence

7

Cyber Forensics is the practice of analyzing digital information in form of evidence that is legally admissible. As a case in point, Sony’s PlayStation Network underwent digital and cyber forensics to ensure the ongoing safety of its 53 million users after experiencing a DDoS incident few months ago.

Forensics 202 Forensics 202

8

Behavior Analysis

Environmental

Analysis

CodeAnalysis

MemoryAnalysis

• Network connection• Registry changes• File & Processes

• Packed code• TSL callbacks• Risky APIs

• Closed source • Mixed source• Open source

• Hidden processes• Malicious drivers• Passive shells

INSIDE RUNTIME

MA

CH

INE

US

ER

Forensic

Analysis

Analytics 101 Analytics 101

ANALYTICSANALYTICS

• What is likely to happen?

• Discovers patterns from data using ML, clustering, etc.

• What is likely to happen?

• Discovers patterns from data using ML, clustering, etc.

ANALYSISANALYSIS

• Why did something happen?

• Converts data deluge into intelligence and provides visualization

• Why did something happen?

• Converts data deluge into intelligence and provides visualization

9

A child can master pattern recognition to identify the birds, but a computer still can’t do it right consistently for simple patterns.

#1 – Eagle et. al. #2 – Swan et. al. #3 - ????

Analytics 202Analytics 202

What’s Continuous in CFAWhat’s Continuous in CFA

• Extended enterprise drives CFA

• Data Centers without walls need CFA

• Resources – internal vs. external

• Technologies – proprietary vs. open

• Services – insourced vs. outsourced

• Endpoints – de/perimeterized

• Insider threat == Outsider threat

• Continuous vs. layered forensics

• Extended enterprise drives CFA

• Data Centers without walls need CFA

• Resources – internal vs. external

• Technologies – proprietary vs. open

• Services – insourced vs. outsourced

• Endpoints – de/perimeterized

• Insider threat == Outsider threat

• Continuous vs. layered forensics

11

Threat Stages in CFAThreat Stages in CFA

• Cyber attack “kill chain” has to be watchlisted

• It has to be quarantined before it can be mitigated

• Targeted attack has distinct stages that must be understood

• Visualization is one of the precursors to continuous forensics.

• Cyber attack “kill chain” has to be watchlisted

• It has to be quarantined before it can be mitigated

• Targeted attack has distinct stages that must be understood

• Visualization is one of the precursors to continuous forensics.

Deception

Disruption

Denial

Degradation

Destruction

Steps Needed for CFASteps Needed for CFA

• Use cases for what’s taken, from where, and when

• Capture “just enough” network pcap data

• Anonymize the user & extract metadata

• Gamify to reconstruct user sessions

• Simulate the real-life scenario

• Map SIP, DIP addresses

• Payload information– Via structured queries

• Use cases for what’s taken, from where, and when

• Capture “just enough” network pcap data

• Anonymize the user & extract metadata

• Gamify to reconstruct user sessions

• Simulate the real-life scenario

• Map SIP, DIP addresses

• Payload information– Via structured queries

13

Sources: SandStorm, Altamira www.d3js.org Library

Skills Required for CFASkills Required for CFA

• Acquire new skills

• Upgrade current skills

• Implement solutions instead of tools

• Invest in training

• Acquire new skills

• Upgrade current skills

• Implement solutions instead of tools

• Invest in training

14

EFFE

CTI

VEN

ESS

TIME

Networkingfundamentals

Softwarevulnerabilities

Hackingtechniques

Securedesign

Scripting

Policyautomation

Data parsingwith regex

Performancemetrics

Security swknowledge

RT data streaming

Data mining

Malwareanalysis

Visualization

Machinelearning

Streaminganalytics

Basic skills Desired skills Innovative skills

Certification

System administrationSandbox

ProgrammaticRestoration

IA DevOps

Jun 17 09:42:30 diptoc ifup: Determining IP information for eth0...Jun 17 09:42:35 diptoc ifup: failed; no link present. Check cable?Jun 17 09:42:35 diptoc network: Bringing up interface eth0: failedJun 17 09:42:38 diptoc sendmail: sendmail shutdown succeededJun 17 09:42:38 diptoc sendmail: sm-client shutdown succeededJun 17 09:42:39 diptoc sendmail: sendmail startup succeededJun 17 09:42:39 diptoc sendmail: sm-client startup succeededJun 17 09:43:39 diptoc vmnet-dhcpd: DHCPINFORM from 172.16.48.128Jun 17 09:45:42 diptoc last message repeated 2 timesJun 17 09:45:47 diptoc vmnet-dhcpd: DHCPINFORM from 172.16.48.128Jun 17 09:56:02 diptoc vmnet-dhcpd: DHCPDISCOVER from 00:0c:29:b7:b2:47 via vmnet8Jun 17 09:56:03 diptoc vmnet-dhcpd: DHCPOFFER on 172.16.48.128 to 00:0c:29:b7:b2:47 via vmnet8Jun 17 09:56:03 diptoc vmnet-dhcpd: DHCPREQUEST for 172.16.48.128 from 00:0c:29:b7:b2:47 via vmnet8Jun 17 09:56:03 diptoc vmnet-dhcpd: DHCPACK on 172.16.48.128 to 00:0c:29:b7:b2:47 via vmnet8Jun 17 10:00:03 diptoc crond(pam_unix)[30534]: session opened for user root by (uid=0)Jun 17 10:00:10 diptoc crond(pam_unix)[30534]: session closed for user rootJun 17 10:01:02 diptoc crond(pam_unix)[30551]: session opened for user root by (uid=0)Jun 17 10:01:07 diptoc crond(pam_unix)[30551]: session closed for user rootJun 17 10:05:02 diptoc crond(pam_unix)[30567]: session opened for user idabench by (uid=0)Jun 17 10:05:05 diptoc crond(pam_unix)[30567]: session closed for user idabenchJun 17 10:13:05 diptoc portsentry[4797]: attackalert: UDP scan from host: 192.168.80.19/192.168.80.19 to UDP port: 192Jun 17 10:13:05 diptoc portsentry[4797]: attackalert: Host: 192.168.80.19/192.168.80.19 is already blocked IgnoringJun 17 10:14:09 diptoc portsentry[4797]: attackalert: UDP scan from host: 192.168.80.8/192.168.80.8 to UDP port: 68Jun 17 10:14:09 diptoc portsentry[4797]: attackalert: Host: 192.168.80.8/192.168.80.8 is already blocked IgnoringJun 17 10:14:09 diptoc portsentry[4797]: attackalert: UDP scan from host: 192.168.80.8/192.168.80.8 to UDP port: 68Jun 17 10:14:09 diptoc portsentry[4797]: attackalert: Host: 192.168.80.8/192.168.80.8 is already blocked IgnoringJun 17 10:21:30 diptoc portsentry[4797]: attackalert: UDP scan from host: 192.168.80.8/192.168.80.8 to UDP port: 68Jun 17 10:21:30 diptoc portsentry[4797]: attackalert: Host: 192.168.80.8/192.168.80.8 is already blocked IgnoringJun 17 10:28:40 diptoc vmnet-dhcpd: DHCPDISCOVER from 00:0c:29:b7:b2:47 via vmnet8Jun 17 10:28:41 diptoc vmnet-dhcpd: DHCPOFFER on 172.16.48.128 to 00:0c:29:b7:b2:47 via vmnet8Jun 17 10:28:41 diptoc vmnet-dhcpd: DHCPREQUEST for 172.16.48.128 from 00:0c:29:b7:b2:47 via vmnet8Jun 17 10:28:45 diptoc vmnet-dhcpd: DHCPACK on 172.16.48.128 to 00:0c:29:b7:b2:47 via vmnet8Jun 17 10:30:47 diptoc portsentry[4797]: attackalert: UDP scan from host: 192.168.80.8/192.168.80.8 to UDP port: 68Jun 17 10:30:47 diptoc portsentry[4797]: attackalert: Host: 192.168.80.8/192.168.80.8 is already blocked IgnoringJun 17 10:30:47 diptoc portsentry[4797]: attackalert: UDP scan from host: 192.168.80.8/192.168.80.8 to UDP port: 68Jun 17 10:30:47 diptoc portsentry[4797]: attackalert: Host: 192.168.80.8/192.168.80.8 is already blocked IgnoringJun 17 10:35:28 diptoc vmnet-dhcpd: DHCPINFORM from 172.16.48.128Jun 17 10:35:31 diptoc vmnet-dhcpd: DHCPINFORM from 172.16.48.128Jun 17 10:38:51 diptoc vmnet-dhcpd: DHCPREQUEST for 172.16.48.128 from 00:0c:29:b7:b2:47 via vmnet8Jun 17 10:38:52 diptoc vmnet-dhcpd: DHCPACK on 172.16.48.128 to 00:0c:29:b7:b2:47 via vmnet8Jun 17 10:42:35 diptoc vmnet-dhcpd: DHCPINFORM from 172.16.48.128Jun 17 10:42:38 diptoc vmnet-dhcpd: DHCPINFORM from 172.16.48.128

Visualize Analytics vs. Analysis

Graph

Find

Spaces

Map

2014-02-10+52 1 825 5536872 +52 1 877 1211498

303-301-5881

303-904-7511

Mazatlan

Mexico City

2014-02-222014-02-22Graph

Joaquin Guzman…Zarka de Mexico

Emma CoronelPatraca

Ismael Garcia

Javier Felix

Source: Lumify, Altamira

Sources: Lumify, Altamira, lumify.io Gephi, gephi.github.io

CFA – Why Now? CFA – Why Now?

16

Top 10 types of security incidents that caused breaches Forensic incident classification patterns over time

Source: Verizon DBIR 2014 report

• Reports show over 1300+ breaches from 63,000 incidents in 95 countries annually … and growing!

• Lot of alarms, some containments … few solutions

• Reports show over 1300+ breaches from 63,000 incidents in 95 countries annually … and growing!

• Lot of alarms, some containments … few solutions

Details Behind Cyber ForensicsDetails Behind Cyber Forensics

Data

17

Briefing available from ISSA. Presented on August 14, 2014

Summarizing Continuous Forensics AnalyticsSummarizing Continuous Forensics Analytics

• Assess looming behind the user activity patterns

• Analyze the data remnants in transient states

• Audit logs to unravel stealth data correlation

• Assert usage of content and patterns in context

• Answer the hard stuff:• “the known knowns” Facts• “the known unknowns” Questions• “the unknown knowns” Intuitions• “the unknown unknowns” Exploration

• Assess looming behind the user activity patterns

• Analyze the data remnants in transient states

• Audit logs to unravel stealth data correlation

• Assert usage of content and patterns in context

• Answer the hard stuff:• “the known knowns” Facts• “the known unknowns” Questions• “the unknown knowns” Intuitions• “the unknown unknowns” Exploration

18

Thank You! Thank You!

19

Dipto ChakravartyOn G+, Y!: diptoc On Tw: diptowww.linkedin.com/in/diptochakravarty [email protected]@gmp4.hbs.edu

http://www.linkedin.com/diptochakravarty


Thank YouThank You

Dipto ChakravartyBS, MS, GMP

Altamira, Open Source Security Expert


Altamira, Open Source Security Expert

20

Question and AnswerQuestion and Answer


Altamira, Open Source Security expert


Altamira, Open Source Security expert

21

22

Thank you!

Dipto ChakravartyBS, MD, GMP

Altamira, Open Source Security [email protected]




Tyrone WilsonCyber Security Senior Analyst

Novetta

Looking to the FutureLooking to the Future

Tyrone E. Wilson

Senior Security Analyst

Novetta Solutions

Tyrone E. Wilson

Senior Security Analyst

Novetta Solutions

24

What would provide perfect network security

visibility?

What would provide perfect network security

visibility?

• Start with the ground truth: network traffic

• Capture ALL PCAP data, store it in infinitely large repository, and make it instantly available for analysis

• Enable human intelligence to counter attackers in real time

• If analysts could ‘see’ everything occurring on their network instantly…• Breaches would inevitably occur• But no business damage would happen

• Goal: Get as close to the ideal solution as current technology and real-world constraints allow

• Start with the ground truth: network traffic

• Capture ALL PCAP data, store it in infinitely large repository, and make it instantly available for analysis

• Enable human intelligence to counter attackers in real time

• If analysts could ‘see’ everything occurring on their network instantly…• Breaches would inevitably occur• But no business damage would happen

• Goal: Get as close to the ideal solution as current technology and real-world constraints allow

Copyright © 2014, Novetta Solutions, LLC. All rights reserved.

What would be the benefits?What would be the benefits?

• Aggregated network phenomena created by advanced threats available immediately

• Analysts empowered to execute behavioral network analytics

• Ability to perform traffic summary roll-ups, intersections, and other advanced exploratory analysis

• Enable rapid iteration and pivoting through network data

• Dramatically accelerate the operational tempo of security teams

• Aggregated network phenomena created by advanced threats available immediately

• Analysts empowered to execute behavioral network analytics

• Ability to perform traffic summary roll-ups, intersections, and other advanced exploratory analysis

• Enable rapid iteration and pivoting through network data

• Dramatically accelerate the operational tempo of security teams


DeploymentDeployment

Analytics Hub

Internet

Router

Network

Firewall

Analytics Engine

Batch Ingest Module

Ingestion and Pre-Processing Module

Metadata

Custom Workflows

Web InterfaceAnalysts API Interface

SIEM

PCAPArchive

Packet Capture

LegacySensor

PCAP*

* PCAP is stored at sensors and is instantly retrievable when needed for deeper inspection

Metadata

Sensor

Sensor

Sensor

PCAP*

PCAP*

PCAP*SIEM

IDS/IPS

DLP

ATP

What about virtualization?What about virtualization?

• Today’s virtual taps do not scale as well as network taps– Gated by motherboard bus speeds & chip resources

• Room for creative solutions– Selective packet capture?– Hardware solutions?

• Today’s virtual taps do not scale as well as network taps– Gated by motherboard bus speeds & chip resources

• Room for creative solutions– Selective packet capture?– Hardware solutions?

What is Sessionization?What is Sessionization?• Creates sessions out of packets

• Uses 5-tuple to identify sessions: source IP, source port, destination IP, destination port, transport protocol

• Attaches locally unique sensor session ID

• Configurable sessionization logic parameters:• Maximum time before segmenting long session • Maximum time after TCP session finishes gracefully • Maximum ages of unfinished TCP and ICMP sessions

• Creates sessions out of packets

• Uses 5-tuple to identify sessions: source IP, source port, destination IP, destination port, transport protocol

• Attaches locally unique sensor session ID

• Configurable sessionization logic parameters:• Maximum time before segmenting long session • Maximum time after TCP session finishes gracefully • Maximum ages of unfinished TCP and ICMP sessions


Network PacketsNIC Software

Session 1

Session 2

Session 3

…

Session Metadata ExtractionSession Metadata Extraction

• Session information• Hosts and ports• Volume transferred• Packet counts• Protocol (TCP, UDP, ICMP, etc.)• All TCP flags (SYN, ACK, etc.)• Client/server designations

• Detected service• HTTP, FTP, SMTP, IRC, etc.• Or first 3 bytes for unknowns

• Session information• Hosts and ports• Volume transferred• Packet counts• Protocol (TCP, UDP, ICMP, etc.)• All TCP flags (SYN, ACK, etc.)• Client/server designations

• Detected service• HTTP, FTP, SMTP, IRC, etc.• Or first 3 bytes for unknowns

• DNS Headers• Query and response• Resource Record types• …and more

• HTTP Headers• Host, URI, Query string• Method, Cookie, User Agent• …and more

• RDP Headers• Host, Product ID, User

cookie• Keyboard layout• …and more

All Sessions Queryable in Columnar DatabaseAll Sessions Queryable in Columnar Database

1

Sensors

4

Analytics Engine

Security-specific MetadataFor a clean and consolidated view

of the network

Internet

SIEM

IDS/IPS

DLP

ATP

Router

Network

FirewallPCAP Data

For preprocessing

2

1%of totalPCAP data

Analysis within Analytics HubAnalysis within Analytics Hub

• Investigative analytics from automated Alerts

• Exploratory analysis

• Iterative drill-down and pivot

• Distillation

• Tagging

• PCAP export

• Investigative analytics from automated Alerts

• Exploratory analysis

• Iterative drill-down and pivot

• Distillation

• Tagging

• PCAP export


Example AnalyticsExample AnalyticsName Description

Beacon Finds beacons from infected hosts to command-and-control servers outside the enterprise

Distant Admin Uncovers remote unauthorized ‘admin-like’ access

HTTP(S) Exfiltration Finds large uploads to remote servers, indicating a potential data exfiltration

Port Scanners Finds slow, randomized port scans, part of an attacker’s reconnaissance and scanning efforts

Protocol Abuse Finds traffic utilizing backdoor/hidden access paths

RDP Keyboard Layout Uncovers sessions by un-expected keyboard types

Relay Finder Retraces an attacker’s path between hosts by finding relays (hops)

Suspicious Admin Toolkits Finds sessions where the client is using a Remote Administration Toolkit (RAT) such as Poison Ivy, Radmin, and Gh0st RAT

Two Degrees of Separation Performs traffic intersections to determine relationships between hosts

Unknown Service Returns sessions where services are unrecognized


DistillationDistillation

• Process PCAP to extract and decode embedded content

• Scripts (python, bash, etc.) follow common API – allows addition of new custom scripts

• Output: extracted content, report, md5 listing, sha1 listing, flow files, optional raw PCAP

• Process PCAP to extract and decode embedded content

• Scripts (python, bash, etc.) follow common API – allows addition of new custom scripts

• Output: extracted content, report, md5 listing, sha1 listing, flow files, optional raw PCAP


Packet Capture

File Extraction

Output

MZ.Xl8ywFuL5V3CCACLTfyLffiLAf93JP8QK3X0i00Mwf4CZosEcGYDRxBeX2aJAbABW4vlXcIIAMzMzMzMzMzMzMzMzMzMzFWL7IPsKFOL2VaJXfyLQwSLUDwD0IsD/3J4/xCLUwSL8Il14ItKPItEEXyJRdyF9gE5wAAAIXAD4TfAAAAi0YQi8uJRdiLRhSJRfSLRhi

ExtractedFiles

Report

SummarySummary

• Augmenting automated alerts, dashboards, visualizations and even machine learning with rapid queryable access to sessionized raw PCAP will be a key part of the Continuous Forensics Analytics process

• Helps to … Answer the hard stuff:• “the known knowns” Facts• “the known unknowns” Questions• “the unknown knowns” Intuitions• “the unknown unknowns” Exploration

• Augmenting automated alerts, dashboards, visualizations and even machine learning with rapid queryable access to sessionized raw PCAP will be a key part of the Continuous Forensics Analytics process

• Helps to … Answer the hard stuff:• “the known knowns” Facts• “the known unknowns” Questions• “the unknown knowns” Intuitions• “the unknown unknowns” Exploration

Thank You!Tyrone Wilson

Cyber Security Senior Analyst

Novetta

Thank You!Tyrone Wilson


Novetta

36

Question and AnswerQuestion and Answer


Novetta

To ask a question, typeyour question in the Chat area of your screen.

You may need to click on the double arrowsto open this function.

#ISSAWebConf


Novetta

To ask a question, typeyour question in the Chat area of your screen.

You may need to click on the double arrowsto open this function.

#ISSAWebConf

37

Tyrone Wilson

38

Thank you!

Tyrone WilsonCyber Security Senior Analyst

Novetta

• Dipto Chakravarty

Altamira Open Source Security Expert

• Tyrone WilsonCyber Security Senior Analyst

Novetta

• Dipto Chakravarty

Altamira Open Source Security Expert

• Tyrone WilsonCyber Security Senior Analyst

Novetta

39

#ISSAWebConf

Open Panel with Audience Q&ATo ask a question,

type your question in the Chat area of your

screen.You may need to

click on the double arrows

to open this function.

I would like to thank Dipto and Tyrone for lending their time and expertise to this ISSA Educational Program. Thank you to Novetta for sponsoring this webinar.

Thank you Citrix for donating the Webcast service.

I would like to thank Dipto and Tyrone for lending their time and expertise to this ISSA Educational Program. Thank you to Novetta for sponsoring this webinar.

Thank you Citrix for donating the Webcast service.

40

#ISSAWebConf

Closing Remarks

• Within 24 hours of the conclusion of this webcast, you will receive a link via email to a post Web Conference quiz.

• After the successful completion of the quiz you will be given an opportunity to PRINT a certificate of attendance to use for the submission of CPE credits.

• On-Demand Viewers Quiz Link• http://www.surveygizmo.com/s3/2096089/ISSA-We

b-Conference-April-14-2015-Continuous-Forensic-Analytics-Issues-and-Answers

• Within 24 hours of the conclusion of this webcast, you will receive a link via email to a post Web Conference quiz.

• After the successful completion of the quiz you will be given an opportunity to PRINT a certificate of attendance to use for the submission of CPE credits.

• On-Demand Viewers Quiz Link• http://www.surveygizmo.com/s3/2096089/ISSA-We

b-Conference-April-14-2015-Continuous-Forensic-Analytics-Issues-and-Answers41

#ISSAWebConf

CPE Credit

http://www.surveygizmo.com/s3/2096089/ISSA-Web-Conference-April-14-2015-Continuous-Forensic-Analytics-Issues-and-Answers




