data mining for computer security 1 - secappdev rieck...data mining for computer security 1 ... •...
TRANSCRIPT
![Page 1: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/1.jpg)
Data Mining for Computer Security 1Konrad Rieck Technische Universität Braunschweig, Germany
![Page 2: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/2.jpg)
Page
About me
• Professor of Computer Science at TU Braunschweig • Fun with security and machine learning for 15 years • Head of Institute of System Security (~10 people)
• More on our website: http://www.tu-bs.de/sec
2
![Page 3: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/3.jpg)
Page
Computer Security Today
• Classic security cycle • Prevention, e.g. authentication • Detection, e.g. virus scanners • Analysis, e.g. digital forensics
• Security cycle out of balance • Increasing amount and diversity of attacks • Larger attack surfaces due to system complexity • Bottleneck: manual analysis of security data
3
Prevention
Analysis
Detection
![Page 4: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/4.jpg)
Page
Our Research
• Security systems with more “intelligence” • Application of data mining and machine learning • Assistance during prevention, detection and analysis • Human out of the loop — but not without control
4
Automatisation of attacks ➠ Automatisation of defenses?
Prevention
Analysis
Detection
![Page 5: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/5.jpg)
Page
Some of Our Work
• Prevention: Discovery of vulnerabilities in software • Graph mining for finding vulnerable code patterns (S&P ’14, ’15)
• Identification of missing security checks (CCS ’13)
• Detection: Identification of attacks and malicious code • Detection of malicious Android applications (NDSS ’14) • Detection of malicious Flash animations (DIMVA ’16, Best Paper Award)
• Analysis: Understanding malicious code • Analysis of ultrasonic side channels in Android (Euro S&P ’17)
• Authorship attribution of native program code (?)
5
![Page 6: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/6.jpg)
Page
Some of Our Work
• Prevention: Discovery of vulnerabilities in software • Graph mining for finding vulnerable code patterns (S&P ’14, ’15)
• Identification of missing security checks (CCS ’13)
• Detection: Identification of attacks and malicious code • Detection of malicious Android applications (NDSS ’14) • Detection of malicious Flash animations (DIMVA ’16, Best Paper Award)
• Analysis: Understanding malicious code • Analysis of ultrasonic side channels in Android (Euro S&P ’17)
• Authorship attribution of native program code (?)
5
ML?
![Page 7: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/7.jpg)
Page
Some of Our Work
• Prevention: Discovery of vulnerabilities in software • Graph mining for finding vulnerable code patterns (S&P ’14, ’15)
• Identification of missing security checks (CCS ’13)
• Detection: Identification of attacks and malicious code • Detection of malicious Android applications (NDSS ’14) • Detection of malicious Flash animations (DIMVA ’16, Best Paper Award)
• Analysis: Understanding malicious code • Analysis of ultrasonic side channels in Android (Euro S&P ’17)
• Authorship attribution of native program code (?)
5
✓
ML?
✓
✓
✓
✓
✓
![Page 8: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/8.jpg)
Page
Let’s go …
• A generic view on learning • How learning works in general (theoretically)
• Types of machine learning • Different types of machine learning techniques
• Some learning algorithms • Implementations of machine learning
• A complete lecture condensed into two sessions. Good luck! !
6
![Page 9: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/9.jpg)
Page
Machine Learning in a Nutshell
7
![Page 10: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/10.jpg)
Page
Machine Learning?
• Machine learning = branch of artificial intelligence • Computer science intersecting with statistics • No science fiction, please! We’re talking about algorithms.
8
T-800 HAL 9000WOPR
![Page 11: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/11.jpg)
Page
Machine Learning
• Theory and practice of making computers learn • Automatic inference of dependencies from data • Generalization of dependencies; ↯ not simple memorization • Application of learned dependencies to unseen data
• Example: Handwriting recognition • Dependencies: written shapes ↔ concrete letters
9
![Page 12: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/12.jpg)
Page
Influences
• Where does machine learning come from? • Interdisciplinary branch of computer science • Close relation to artificial intelligence and data mining
• Different inspirations for learning, e.g. neurology, physics, ... • Large diversity of approaches, concepts and algorithms
10
Physics
Neurology
Statistics
Biology
Machine LearningComputer Science
Mathematics
![Page 13: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/13.jpg)
Page
Intrusion Detection
• Network intrusion detection • Detection of attacks in network payloads • Classic approach: signature-based detection • Running example in this talk
• Network packet and matching signature
11
Packet payloadHeaders
Nimda wormTCP ..%c1%9c..
GET /scripts/..%c1%9c../system32/cmd.exe... | IP | TCP
Running example
![Page 14: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/14.jpg)
Page
Feature Spaces
• Machine learning usually defined over vector spaces • Security data almost never in form of vectors • Key for learning in security → a map to a feature space
• Representation of real objects using features
12
Machine learning
Feature space
Real objects
![Page 15: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/15.jpg)
Page
Feature Extraction
13
Numerical features (Vectors)
Length 14Entropy 3.4Alpha. 12Punct. 1
Sequential features (Strings)
...GET▯
ET▯/
T▯/i
▯/in
Structural features (Trees, Graphs)
GET
indexhtml
Running example
GET index.html
Network payload
Feature extraction
x =
![Page 16: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/16.jpg)
Page
A Learning Model
• What can we learn? • Inference of functional dependencies from data (X ↔ Y) • Dependencies described by a learning model θ • Model θ parameterizes a prediction function fθ : X → Y
• A simple example • X = color × height of fruits • Y= {apple, pear} • θ = (color, height) and bias
14
fθ
![Page 17: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/17.jpg)
Page
Examples: Learning Models
15
Quadratic functions
fθ
Other non-linear functions
fθ
Decision stumps
fθ
![Page 18: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/18.jpg)
Page
Learning Function
• Learning process • Searching the space Θ for good models (functions fθ)
• Supervised learning (with labels) • Learning function g : X × Y → Θ • “You know what you are looking for”
• Unsupervised learning (without labels) • Learning function g : X → Θ • “You don’t know what you are looking for”
16
![Page 19: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/19.jpg)
Page
Learning and Errors
• Learning process guided by errors • Minimal error of learning model θ desirable • Quantification of disagreement between predictions and truth • Different strategies for reducing errors
17
+1
+1
+1
+1+1
+1
+1
+1+1
–1
–1
–1–1
–1
–1
–1
–1
3 errors
![Page 20: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/20.jpg)
Page
Learning and Errors
• Learning process guided by errors • Minimal error of learning model θ desirable • Quantification of disagreement between predictions and truth • Different strategies for reducing errors
17
+1
+1
+1
+1+1
+1
+1
+1+1
–1
–1
–1–1
–1
–1
–1
–1
3 errors
+1
+1
+1
+1+1
+1
+1
+1+1
–1
–1
–1–1
–1
–1
–1
–1
still 3 errors
![Page 21: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/21.jpg)
Page
Learning and Errors
• Learning process guided by errors • Minimal error of learning model θ desirable • Quantification of disagreement between predictions and truth • Different strategies for reducing errors
17
+1
+1
+1
+1+1
+1
+1
+1+1
–1
–1
–1–1
–1
–1
–1
–1
3 errors
+1
+1
+1
+1+1
+1
+1
+1+1
–1
–1
–1–1
–1
–1
–1
–1
still 3 errors
+1
+1
+1
+1+1
+1
+1
+1+1
–1
–1
–1–1
–1
–1
–1
–1
no errors
![Page 22: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/22.jpg)
Page
Test Data and Overfitting
• Training and test data • Model learned on training data; prediction on unseen test data • Optimizing the error on training data dangerous
18
fθ
12% training error
fθ
0% training error
Training data
![Page 23: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/23.jpg)
Page
Test Data and Overfitting
• Training and test data • Model learned on training data; prediction on unseen test data • Optimizing the error on training data dangerous
18
fθ
12% training error
fθ
0% training error
Training data
8% test error
fθ
21% test error
fθTest data
![Page 24: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/24.jpg)
Page
Regularization
• Regularization key to effective learning • Danger of adapting learning model to training data only • Balancing of training error and model complexity • Examples: Costs of SVMs, pruning in decision trees, ...
19
Risk
(erro
r)
Training error
Complexity of learning model
OverfittingUnderfitting
Test error
Complexity term
![Page 25: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/25.jpg)
Page
Types of Machine Learning
20
![Page 26: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/26.jpg)
Page
Supervised: Classification
• Learning to categorize objects into known classes • Discrimination of objects using learning model • Output domain often Y = {–1, +1} or {1,2,3...}
• Examples • Handwriting recognition • Spam filtering in emails
• Common algorithms • SVM, KNN, Neural Networks, ...
21
fθ
![Page 27: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/27.jpg)
Page
Classification
• Classification for intrusion detection • Discrimination between benign and malicious activity
22
“benign” “malicious”
GET /scripts/..%%35c../system32/cmd.exe
Data payloadHeader
... | IP | TCP
Running example
![Page 28: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/28.jpg)
Page
Classification
• Classification for intrusion detection • Discrimination between benign and malicious activity
22
“benign” “malicious”
GET /scripts/..%%35c../system32/cmd.exe
Data payloadHeader
GET /scripts/..%c1%af../system32/cmd.exe
... | IP | TCP
Running example
![Page 29: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/29.jpg)
Page
Classification
• Classification for intrusion detection • Discrimination between benign and malicious activity
22
“benign” “malicious”
GET /scripts/..%%35c../system32/cmd.exe
Data payloadHeader
GET /scripts/..%c1%af../system32/cmd.exe
GET /scripts/..%255c../system32/cmd.exe
... | IP | TCP
Running example
![Page 30: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/30.jpg)
Page
Unsupervised: Clustering
• Grouping of similar objects into clusters • Contrast to classification: clusters not known at start • Output domain Y = {1,2,3,...} (~ permutations)
• Examples • Comparison of species • Malware analysis
• Common learning algorithms • K-means, linkage clustering, ...
23
fθ
![Page 31: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/31.jpg)
Page
Clustering
• Clustering of network payloads for later analysis • Unsupervised grouping of similar payloads into clusters
24
Data payloadHeader
... | IP | TCP
Running example
GET /scripts/..%%35c../system32/cmd.exe
![Page 32: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/32.jpg)
Page
Clustering
• Clustering of network payloads for later analysis • Unsupervised grouping of similar payloads into clusters
24
Data payloadHeader
... | IP | TCP
Running example
GET /scripts/..%%35c../system32/cmd.exe
Attack Z
![Page 33: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/33.jpg)
Page
Clustering
• Clustering of network payloads for later analysis • Unsupervised grouping of similar payloads into clusters
24
Data payloadHeader
... | IP | TCP
Running example
GET /scripts/..%%35c../system32/cmd.exe
GET /php-fun/?-s
Attack Y
Attack Z
![Page 34: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/34.jpg)
Page
Clustering
• Clustering of network payloads for later analysis • Unsupervised grouping of similar payloads into clusters
24
Data payloadHeader
... | IP | TCP
Running example
GET /scripts/..%%35c../system32/cmd.exe
GET /IISWebAgentIF.dll?overflow...
Attack X
GET /php-fun/?-s
Attack Y
Attack Z
![Page 35: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/35.jpg)
Page
Unsupervised: Anomaly Detection
• Detection of deviations from learned model of normality • Generative or discriminative models of normality • Output domain often Y = [0,1] (anomaly score)
• Examples • Engine failure detection • Intrusion detection
• Common approaches • Statistics, one-class SVM, ...
25
fθ
![Page 36: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/36.jpg)
Page
Anomaly Detection
• Anomaly detection for intrusion detection • Identification of attacks as deviations from normality
26
“normal” “anomalous”
GET /scripts/..%%35c../system32/cmd.exe
Data payloadHeader
... | IP | TCP
Running example
![Page 37: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/37.jpg)
Page
Anomaly Detection
• Anomaly detection for intrusion detection • Identification of attacks as deviations from normality
26
“normal” “anomalous”
GET /scripts/..%%35c../system32/cmd.exe
Data payloadHeader
GET /scripts/..%c1%af../system32/cmd.exe
... | IP | TCP
Running example
![Page 38: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/38.jpg)
Page
Anomaly Detection
• Anomaly detection for intrusion detection • Identification of attacks as deviations from normality
26
“normal” “anomalous”
GET /scripts/..%%35c../system32/cmd.exe
Data payloadHeader
GET /scripts/..%c1%af../system32/cmd.exe
GET /scripts/..%255c../system32/cmd.exe
... | IP | TCP
Running example
![Page 39: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/39.jpg)
Page
Supervised: Regression
• Learning to predict a numerical property (score) • Approximation of observed function by learning model • Output domain usually Y = ℝ
• Examples • Temperature forecasting • Stock market prediction
• Common algorithms • Logistic & ridge regression, ...
27
fθ
swee
tnes
s
![Page 40: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/40.jpg)
Page
Dimension Reduction
• Supervised or unsupervised reduction of dimensionality • Extraction of more informative features for objects • X = ℝN and Y = ℝM with N ≫ M
• Examples • Visualisation and denoising • Vulnerability discovery
• Common learning algorithms • PCA, LLE, NMF, ...
28
fθ
![Page 41: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/41.jpg)
Page
Some Learning Algorithms
29
![Page 42: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/42.jpg)
Page
K-Nearest Neighbors
• Learning using the local neighborhood of data • Most intuitive and oldest learning algorithm • Learning = not really ...training data is just stored • Regularization = size of considered neighborhood • Prediction = labels of neighborhood
30
Neighborhood k = 4 Neighborhood k = 11
![Page 43: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/43.jpg)
Page
K-Nearest Neighbors
• Learning using the local neighborhood of data • Most intuitive and oldest learning algorithm • Learning = not really ...training data is just stored • Regularization = size of considered neighborhood • Prediction = labels of neighborhood
30
Neighborhood k = 4 Neighborhood k = 11
Green: 1 Red: 3
Green: 8 Red: 3
![Page 44: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/44.jpg)
Page
Neural Networks
• Learning using a network of artificial neurons • Classic method inspired by biological neural networks (~1940) • Learning = adaption of weights of neural network • Regularization = brain damage or weight decay • Prediction = forward pass through neural network
31
Input
Hidden layer(s)
Output
YX
WI WII
Deep Learning: Recent revival of neural networks with several different hidden layers
![Page 45: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/45.jpg)
Page
Decision Trees
• Learning by composition of simple logic predicates • Classic method inspired by decision making (~1960) • Learning = inductive composition of tree nodes • Regularization = pruning of subtrees • Prediction = top-down pass through tree
32
Xa > 3
Xa ≤ 3 Xb ≤ 1.3
Xb > 1.3
Xc > 29
Xc ≤ 29
Y = –1 Y = +1
Y = +1Y = –1
Random Forests: Ensemble of decision trees, each learned on randomly selected features
![Page 46: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/46.jpg)
Page
Support Vector Machines
• Learning using a hyperplane in a kernel feature space • Modern method inspired by learning theory (~1990) • Learning = convex problem for determining hyperplane • Regularization = softening of hyperplane for outliers • Prediction = orientation to hyperplane
33
Input space
Kernel map
Feature space
![Page 47: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/47.jpg)
Page
Support Vector Machines
• Learning using a hyperplane in a kernel feature space • Modern method inspired by learning theory (~1990) • Learning = convex problem for determining hyperplane • Regularization = softening of hyperplane for outliers • Prediction = orientation to hyperplane
33
Input space
Kernel map
Feature space
Hyperplane with maximum margin
![Page 48: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/48.jpg)
Page
Several Other Methods
• Several other learning methods • Probabilistic models • Boosting and bagging • Genetic algorithms • ...
• Several other learning concepts • Reinforcement learning • ...
34
Shawe-Taylor & Cristianini: Kernel Methods for Pattern Analysis Cambridge 2004.
Kernel Methods
Duda, Hart and Stork: Pattern Classification Wiley & Sons 2001
The Standard
![Page 49: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/49.jpg)
Page
Summary
35
![Page 50: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/50.jpg)
Page
Summary
• Current problems of computer security • Increasing automatization of attacks and malware • Large amounts of novel malicious code • Defenses involving manual analysis often ineffective
• Machine learning in computer security • Adaptive defenses using learning algorithms • Automatic detection and analysis of threats • Assisted analysis of threats, e.g. vulnerabilities
36
![Page 51: Data Mining for Computer Security 1 - SecAppDev Rieck...Data Mining for Computer Security 1 ... • Professor of Computer Science at TU Braunschweig ... • Learning by composition](https://reader031.vdocuments.mx/reader031/viewer/2022022500/5aa077c57f8b9a84178e313f/html5/thumbnails/51.jpg)
Page
Thank you! Questions?
37