指導教授:王國禎 博士 學生: 蔡薰儀 國立交通大學網路工程研究所...
DESCRIPTION
Suspicious Behavior-based Malware Detection Using Artificial Neural Network 基於可疑行為 及 類 神經 網路之 惡意軟體偵測機制. 指導教授:王國禎 博士 學生: 蔡薰儀 國立交通大學網路工程研究所 行動計算與寬頻網路實驗室. Outline. Introduction Related Work Problem Statement Background Sandboxes Design Approach Suspicious Behaviors - PowerPoint PPT PresentationTRANSCRIPT
Copyright © 2012, [email protected]
Suspicious Behavior-based Malware DetectionUsing Artificial Neural Network
基於可疑行為及類神經網路之惡意軟體偵測機制指導教授:王國禎 博士 學生:蔡薰儀
國立交通大學網路工程研究所行動計算與寬頻網路實驗室
Copyright © 2012, [email protected]
2
Outline
• Introduction• Related Work• Problem Statement• Background
– Sandboxes• Design Approach
– Suspicious Behaviors– Proposed ANN-MD System– Weight Adjusting– Malicious Degree
• Evaluation– Training Phase– Testing Phase
• Conclusion and Future Work• References
Copyright © 2012, [email protected]
3
Introduction
• In recent years, malware has been severe threats to cyber security– Viruses, Worms, Trojan horses, Botnets, …
• Drawbacks of traditional signature-based malware detection algorithms [1] [2]– Need human and time to approve– Need to update the signatures of malware frequently– Easily bypassed by obfuscation methods– Can not detect zero day malware– Increase false negative rate
Copyright © 2012, [email protected]
4
Introduction (Cont.)
• To conquer the shortcomings of signature-based malware detection algorithms, behavior-based malware detection algorithms were proposed
• Behavior-based malware detection algorithms [3] [4]– Detect unknown malware or variations of known malware– Decrease false negative rate (FNR)
• However, existing behavior-based malware detection algorithms may have a higher false positive rate (FPR)– Benign software may have some behaviors which are alike with
malware
Copyright © 2012, [email protected]
5
Introduction (Cont.)
• We proposed a behavioral artificial neural network (ANN)-based malware detection (ANN-MD) algorithm– Detect unknown malware and variations of known malware– Decrease FNR and FPR
Copyright © 2012, [email protected]
6
Related Work
• MBF [3]– File, process, network, and registry actions– Malicious Behavior Feature (MBF) MBF = <Feature_id, Mal_level, Bool_expression>– Three malicious level: high, warning, and low
• RADUX [4]– Reverse Analysis for Detecting Unsafe eXecution (RADUX)– API function call sequences, e.g. load register: RegOpenKey,
RegCreateKey, RegSetValue, RegCloseKey– Collected 9 common suspicious behaviors– Use Bayes’ theorem to compute suspicious degree (malicious
degree). Combine the appearance probability of each behavior all together not individually
Copyright © 2012, [email protected]
7
Related Work (Cont.)
Approach ANN-MD (proposed) MBF [3] RADUX [4]
Main idea Analyze suspicious behaviors Analyze malicious behavior features (MBF) Analyze API call sequences
Number of suspicious behaviors
13(Only host behaviors)
16(Host behaviors + Network behaviors)
9(Only host behaviors)
Weights of suspicious behaviors
The weights are initialized by function initnw and adjust by
ANN
Malicious level: high, warning, and low
Appearance frequency of each suspicious behavior
Calculating of malicious degree ANN Make the summation of the
mal_level of each MBF Bayes’ theorem
Pros
• Set different weights for different behaviors
• Considered each suspicious behavior
individually
• Considered the malicious level of each malicious
behavior feature
• Set different weights for different behaviors according to their
appearance probabilities
Cons • Not considered network behaviors
• Only used high, warning, and low to represent the malicious level of each
MBF
• Combined the appearance probability of each behavior all together
not individually• Not considered network
behaviors
Copyright © 2012, [email protected]
8
Problem Statement
• Given– Several sandboxes– i known malware M = {M1,M2, …, Mi} for training– j known malware N = {N1, N2, …, Nj} for testing– k benign software O = {O1, O2, …, Ok} for training– l benign software P = {P1, P2, …, Pl} for testing
• Objective– m behaviors B = {B1,B2, …, Bm} – m weights W = {ω1, ω2, …, ωm}– Malicious Degree (MD) expression
Copyright © 2012, [email protected]
9
Problem Statement (Cont.)
• Try to find the optimal MD threshold to make FPR and FNR as small as possible.
Benign Malicious
MD Threshold
Ambiguous
Number of Samples
Copyright © 2012, [email protected]
10
Background – Sandboxes
• A sandbox is a testing environment which can isolate unknown sample from make changes to the operating system
• It can interact with samples and record all the runtime behaviors of samples
• Web-based sandboxes– GFI Sandbox [5]– Norman Sandbox [6]– Anubis Sandbox [7]
Copyright © 2012, [email protected]
11
Design Approach – Suspicious Behaviors
• Choose the behaviors in the intersection of the behaviors these sandboxes investigate
• Choose the behaviors which are not in the intersection but have high appearance frequency, i.e.– Creates Mutex– Creates Hidden File– Starts EXE in System– Checks for Debugger– Starts EXE in Documents– Windows/Run Registry Key Set– Hooks Keyboard– Modifies Files in System– Deletes Original Sample– More than 5 Processes– Opens Physical Memory– Deletes Files in System– Auto Start
Copyright © 2012, [email protected]
12
Design Approach – Suspicious Behaviors (Cont.)
Ulrich Bayer et al. [8]The behaviors we chooseThe behaviors which may cause false positive rate
Observed Behavior Percentage of samplesInstallation of a Windows kernel driver 3.34%
Installation of a Windows service 12.12%Creating a file 70.78%Deleting a file 42.57%Modifying a file 79.87%Installation of an IE BHO 1.72%Installation of an IE Toolbar 0.07%Display a GUI window 33.26%Network Traffic 55.18%Writing to stderr 0.78%Writing to stdout 1.09%Modifying a registry value 74.59%Creating a registry key 64.74%Creating a process 52.19%
Copyright © 2012, [email protected]
14
Design Approach – Weight Adjusting
• Using ANN to train weights
Copyright © 2012, [email protected]
15
Design Approach – Weight Adjusting (Cont.)
• Neuron for ANN hidden layer: the first neuron
𝑓 ( 1) (𝑛1 )=𝑒𝑛1 −𝑒−𝑛 1
𝑒𝑛1+𝑒−𝑛1=𝑎1∑
𝑖=1
13
ω𝑖 ,1𝑥 𝑖+𝑏1=𝑛1
Copyright © 2012, [email protected]
16
Design Approach – Weight Adjusting (Cont.)
• Neuron for ANN output layer
∑𝑖=1
10
ω𝑖′ 𝑎𝑖+𝑏′=𝑛′ 𝑓 ( 2) (𝑛′ )=𝑒𝑛 ′ −𝑒−𝑛 ′
𝑒𝑛 ′+𝑒−𝑛 ′
Copyright © 2012, [email protected]
17
Design Approach – Weight Adjusting (Cont.)
• Delta learning process
2)(21 OdE
Expected target value:
}101|'{}101,131|{ , kjiW kji
Mean square error:
Weight set:
,W
oldnew
xE
, : learning factor; x: set of input values
𝑑={ 0 ,𝐵𝑒𝑛𝑖𝑔𝑛1,𝑀𝑎𝑙𝑖𝑐𝑖𝑜𝑢𝑠; output value: O (MD)
Copyright © 2012, [email protected]
18
Design Approach – Malicious Degree
• Malicious Degree Expression– Suspicious behaviors: – Weights: – Bias: – Transfer function: (tangent-sigmoid function)
Copyright © 2012, [email protected]
19
Evaluation (Cont.)
• Use matlab 7.11.0 to implement ANN in our system• Initial weights and bias: chosen by function initnw
to distribute the weight of each neuron in the layer evenly [9] (according to the Nguyen-Widrow initialization algorithm)
• Transfer function: tangent-sigmoid function• Learning factor η () : 0.5
Copyright © 2012, [email protected]
21
Evaluation (Cont.)
• Malicious sample sources: Blast’s Security [10] and VX Heaven [11] websites
• Benign sample sources: Portable execution files under windows XP SP2
• Training samples and testing samples Samples Malicious Benign TotalTraining 500 500 1000
Testing 500 500 1000
Copyright © 2012, [email protected]
22
Evaluation – Training Phase
• Execution time: 3 seconds (training and testing phase)• MD threshold (according to training samples)
0 0.04 0.08 0.12 0.16 0.2 0.24 0.28 0.32 0.36 0.4 0.44 0.48 0.52 0.56 0.6 0.64 0.68 0.72 0.76 0.8 0.84 0.88 0.92 0.96 10
50
100
150
200
250
300
350
400
Benign Samples Malicious Samples
Malicious Degree
Num
ber o
f Sam
ples 0.19 0.87
Range of MD Threshold
Copyright © 2012, [email protected]
23
Evaluation – Training Phase (Cont.)
• Choose MD threshold
0.19
0.22
0.25
0.28
0.31
0.34
0.37 0.4 0.4
30.4
60.4
90.5
20.5
50.5
80.6
10.6
40.6
7 0.7 0.73
0.760
0000
0000
0001
0.790
0000
0000
0001
0.820
0000
0000
0001
0.850
0000
0000
0001
0
1
2
3
4
5
6
7
8
95
95.5
96
96.5
97
97.5
98
98.5
99
FPR FNR Accuracy Rate
Malicious Degree
Fals
e R
ate
(%)
Acc
urac
y R
ate
(%)
0.5
Copyright © 2012, [email protected]
24
Evaluation – Testing Phase
• Experiment resultsTP: True positive FN: False negativeFP: False positive TN: True negative
TP TN FP FN FPR FNR Accuracy485 496 4 15 0.8% 3% 98.1%
Copyright © 2012, [email protected]
25
Evaluation – Testing Phase (Cont.)
• Distribution of testing samples
0 0.04 0.08 0.12 0.16 0.2 0.24 0.28 0.32 0.36 0.4 0.44 0.48 0.52 0.56 0.6 0.64 0.68 0.72 0.76 0.8 0.84 0.88 0.92 0.96 10
50
100
150
200
250
300
350
400
450
Benign Samples Malicious Samples
Malicious Degree
Num
ber o
f Sam
ples MD Threshold = 0.5
Copyright © 2012, [email protected]
26
Evaluation (Cont.)
ApproachTP / (TP + FN) FN / (TP + FN)
Accuracy rateFP / (FP + TN) TN / (FP + TN)
ANN-MD(proposed)
97% 3%98.1%
0.8% 99.2%
MBF [14]83% 17%
88.7%5.6% 94.4%
RADUX [19]96.6% 3.4%
91.2%14.2% 85.8%
Copyright © 2012, [email protected]
27
Evaluation (Cont.)
Weights
FPR FNR Accuracy rateAdjusting
of weightWeights in
hidden layerWeights in
output layer
With ANN
Chosen by initnw
Chosen by initnw 0.8% 3% 98.1%
Appearance frequency
Chosen by initnw 1.2% 2.8% 98%
Without ANN Appearance frequency 7.8% 4.8% 93.7%
Copyright © 2012, [email protected]
28
Conclusion and Future Work
• Conclusion– Collect 13 common behaviors of malwares– Construct Malicious Degree (MD) expression– FPR and FNR are as small as possible– Has a better effect on detecting unknown malware compared to
the related work [14] [19]• Future work
– Automate the proposed ANN-MD system– Implement PC-based sandboxes– Add more suspicious network behaviors– Classify malwares according to their typical behaviors
Copyright © 2012, [email protected]
29
References
[1] C. Mihai and J. Somesh, “Static analysis of executables to detect malicious patterns,” Proceedings of the 12th conference on USENIX Security Symposium, Vol. 12, pp. 169 - 186, Dec. 10-12, 2006.[2] J. Rabek, R. Khazan, S. Lewandowskia, and R. Cunningham, “Detection of injected, dynamically generated, and obfuscated malicious code,” Proceedings of the 2003 ACM workshop on Rapid malcode, pp. 76 - 82, Oct. 27-30, 2003.[3] W. Liu, P. Ren, K. Liu, and H. X. Duan, “Behavior-based malware analysis and detection,” Proceedings of Complexity and Data Mining (IWCDM), pp. 39 - 42, Sep. 24-28, 2011. [4] C. Wang, J. Pang, R. Zhao, W. Fu, and X. Liu, “Malware detection based on suspicious behavior identification,” Proceedings of Education Technology and Computer Science, Vol. 2, pp. 198 - 202, Mar. 7-8, 2009.[5] GFI Sandbox. http://www.gfi.com/malware-analysis-tool[6] Norman Sandbox. http://www.norman.com/security_center/security_tools[7] Anubis Sandbox. http://anubis.iseclab.org/
Copyright © 2012, [email protected]
30
References (Cont.)
[8] U. Bayer, I. Habibi, D. Balzarotti, E. Krida, and C. Kruege, “A view on current malware behaviors,” Proceedings of the 2nd USENIX Workshop on Large-Scale Exploits and Emergent Threats : botnets, spyware, worms, and more, pp. 1 - 11, Apr. 22-24, 2009.[9] Neural Network Toolbox. http://dali.feld.cvut.cz/ucebna/matlab/toolbox/nnet/initnw.html[10] Blast's Security. http://www.sacour.cn[11] VX heaven. http://vx.netlux.org/vl.php