Download - Data Analytics for Security Intelligence
Data Analytics for Security Intelligence
Camil Demetrescu
Dept. Computer, Control, and Management Engineering Credits: Peter Wood, First Base Technologies LLP
Data Driven Innovation Rome 2016 – Open Summit Roma Tre University, May 20 2016
Outline
• Big data
• Advanced threats – current situation
• Why big data for security?
• How can big data help?
• Big data security challenges
• Conclusions
20/5/2016 Data Driven Innovation Rome 2016 Page 2
Big data Every day, we create 2.5 quintillion bytes of data. 90% of the data in the world today has been created in the last two years alone.
http://www-01.ibm.c/software/data/bigdata/
2.5 quintillion = 2.5 exabytes = 2.5 x 1018 = 2.500.000.000.000.000.000 bytes
• Sensors used to gather climate information • Posts to social media sites • Digital pictures and videos • Purchase transaction records • Cell phone GPS signals
20/5/2016 Data Driven Innovation Rome 2016 Page 3
20/5/2016 Data Driven Innovation Rome 2016 Page 4
Outline
• Big data
• Advanced threats – current situation
• Why big data for security?
• How can big data help?
• Big data security challenges
• Conclusions
20/5/2016 Data Driven Innovation Rome 2016 Page 5
Malware events per hour
20/5/2016 Data Driven Innovation Rome 2016 Page 6
Organisations on average are experiencing malware-related activities once every three minutes. Receipt of a malicious email, a user clicking a link on an infected website, or an infected machine making a call back to a command and control server.
Fire
Eye
Adv
ance
d Th
reat
Rep
ort 2
012
How breach occurred
20/5/2016 Data Driven Innovation Rome 2016 Page 7
The Post Breach Boom, Ponemon Institute 2015 Survey of 3,529 IT and IT security practitioners
When the breach was discovered
20/5/2016 Data Driven Innovation Rome 2016 Page 8
The Post Breach Boom, Ponemon Institute 2015 Survey of 3,529 IT and IT security practitioners
Reasons for failing to prevent the breach
20/5/2016 Data Driven Innovation Rome 2016 Page 9
The
Pos
t Bre
ach
Boo
m, P
onem
on In
stitu
te 2
015
Sur
vey
of 3
,529
IT a
nd IT
sec
urity
pra
ctiti
oner
s
Extrapolated cost of breach
20/5/2016 Data Driven Innovation Rome 2016 Page 10
The
Pos
t Bre
ach
Boo
m, P
onem
on In
stitu
te 2
015
Sur
vey
of 3
,529
IT a
nd IT
sec
urity
pra
ctiti
oner
s
Outline
• Big data
• Advanced threats – current situation
• Why big data for security?
• How can big data help?
• Big data security challenges
• Conclusions
20/5/2016 Data Driven Innovation Rome 2016 Page 11
Data driven information security: examples
20/5/2016 Data Driven Innovation Rome 2016 Page 12
• Analyze system/applications log files • Analyze network traffic • Identify anomalies and suspicious activities
• Correlate multiple sources of information into a coherent view
Why do we need big data systems?
20/5/2016 Data Driven Innovation Rome 2016 Page 13
• System Log files that can grow by gigabytes per second
• Network data captures, which can grow by 10s of gigabytes per second
• Intrusion Detection/Protection log files that can grow by 10s of gigabytes per second
• Application Log files that can grow by gigabytes per second
http://www.virtualizationpractice.com/big-data-security-tools-22075/
Traditional scenarios
Traditional defences: • Signature-based anti-virus • Signature-based IDS/IDP • Firewalls and perimeter devices
Traditional approach: • Data collection for compliance • Check-list mindset • Tactical thinking
20/5/2016 Data Driven Innovation Rome 2016 Page 14
New challenges
Complex threat landscape: • Stealth malware • Targeted attacks • Social engineering
New technologies and challenges: • Social networking • Cloud • BYOD / consumerisation • Virtualisation
20/5/2016 Data Driven Innovation Rome 2016 Page 15
Conventional vs. advanced approaches
20/5/2016 Data Driven Innovation Rome 2016 Page 16
http
://w
ww
.em
c.co
m/c
olla
tera
l/ind
ustry
-ove
rvie
w/s
bic-
rpt.p
df
Outline
• Big data
• Advanced threats – current situation
• Why big data for security?
• How can big data help?
• Big data security challenges
• Conclusions
20/5/2016 Data Driven Innovation Rome 2016 Page 17
Data-driven information security: early times
20/5/2016 Data Driven Innovation Rome 2016 Page 18
• Bank fraud detection and anomaly-based intrusion detection systems.
• Credit card companies have conducted fraud detection for decades.
• Custom-built infrastructure to mine big data for fraud detection was not economical to adapt for other fraud detection uses (healthcare, insurance, etc.)
Cloud Security Alliance
Data analytics for intrusion detection
20/5/2016 Data Driven Innovation Rome 2016 Page 19
Intrusion detection systems – Security architects realized the need for layered security (e.g., reactive security and breach response) because a system with 100% protective security is impossible.
1st generation
2nd generation
Security information and event management (SIEM) – aggregate and filter alarms from many sources and present actionable information to security analysts.
3rd generation
Big data analytics in security (2nd generation SIEM) – correlating, consolidating, and contextualizing diverse security event information, correlating long-term historical data for forensic purposes
How can big data analytics help?
• Advanced persistent threat (APT) detection? • Integration of IT and physical security?
• Predictive analysis
• Real-time updates
• Behaviour models
• Correlation
• … advising the analysts?
• … active defence?
20/5/2016 Data Driven Innovation Rome 2016 Page 20
How can big data analytics help?
20/5/2016 Data Driven Innovation Rome 2016 Page 21
Outline
• Big data
• Advanced threats – current situation
• Why big data for security?
• How can big data help?
• Big data security challenges
• Conclusions
20/5/2016 Data Driven Innovation Rome 2016 Page 22
Big data security challenges
• Bigger data = bigger breaches?
• New technology = security later?
• Information classification
• Information ownership (outputs and raw data)
• Big data in cloud + BYOD = more problems?
20/5/2016 Data Driven Innovation Rome 2016 Page 23
Big data security risks
• New technology will introduce new vulnerabilities
• Attack surface of the nodes in a cluster may not have been reviewed and servers adequately hardened
• User authentication and access to data from multiple locations may not be sufficiently controlled
• Regulatory requirements may not be fulfilled, with access to logs and audit trails problematic
• Significant opportunity for malicious data input and inadequate data validation
20/5/2016 Data Driven Innovation Rome 2016 Page 24
Big data privacy concerns
• De-identifed information may be re-identified
• Possible deduction of personally identifiable information
• Risk of data breach is increased
• "Creepy" Factor: consumers may feel that companies know more about them than they are willing to volunteer
• Big brother: predictive policing and tracking potential terrorist activities. Harm individual rights or deny consumers important benefits (such as housing or employment) in lieu of credit reports.
http://www.ftc.gov/public-statements/2012/03/big-data-big-issues
Outline
• Big data
• Advanced threats – current situation
• Why big data for security?
• How can big data help?
• Big data security challenges
• Conclusions
20/5/2016 Data Driven Innovation Rome 2016 Page 26
Conclusions
20/5/2016 Data Driven Innovation Rome 2016 Page 27
• As with all new technologies, security in big data use cases seems to be an afterthought at best
• Big data breaches will be big too, with even more serious reputational damage and legal repercussions
• All organisations need to invest in research and study of the emerging big data security analytics landscape
• Big data has the potential to defend against advanced threats, but requires a big re-think of approach
• Relevant skills are key to successful deployment, only the largest organisations can invest in this now
Big data to collect
• Logs • Network traffic
• IT assets
• Sensitive / valuable information
• Vulnerabilities
• Threat intelligence
• Application behaviour
• User behaviour
20/5/2016 Data Driven Innovation Rome 2016 Page 28