stealthy malware detection through vmm-based “out-of-the-box” semantic view reconstruction...
TRANSCRIPT
Stealthy Malware Detection Through VMM-based “Out-of-the-Box” Semantic View Reconstruction
CCS’07, Alexandria, VA, Oct 29 – Nov 2, 2007
Xuxian Jiang, Xinyuan Wang, Dongyan Xu
George Mason UniversityPurdue University
Internet malware remains a top threat Malware: viruses, worms, rootkits, spyware,
bots…
Motivation
Motivation Recent Trend on Rootkits
Source: McAfee Avert Lab Report (April 2006)
400% growth
400% growth
Q1 of 2005
700% growth
700% growth
Viruses/worms/bots, PUPs, …
Existing Defenses (e.g., Anti-Virus Software)
Running inside the monitored system Advantages
They can see everything (e.g., files, processes,…)
Disadvantages Once compromised by advanced stealthy
malware, they may not see anything!VirusScanFirefoxIE
OS Kernel
…
Existing Defenses
Key observation Both anti-virus software and vulnerable
software are running inside the same system Hard to guarantee tamper-resistance
Solution: “Out-of-the-box” defense
FirefoxIE
OS Kernel
…
VirusScan
Virtual Machine Monitor (VMM)
The “Semantic-Gap” Challenge
What we can observe? Low-level states
Memory pages, disk blocks,… Low-level events
Privileged instructions, Interrupts, I/O access, …
What we want to observe? High-level states w/ semantic
info. Files, processes,…
high-level events w/ semantic info. System calls, context switches, …
Virtual Machine Monitor (e.g., VMware, Xen, QEMU)
Guest OSSemantic Gap
VirusScan
Main Contribution
VMwatcher: A systematic approach to bridge the semantic gap Reconstructing semantic objects and events
from low-level VMM observations
FirefoxIE
OS Kernel
…
Virtual Machine Monitor (VMM)
VMwatcher
Capability I: “Out-of-the-box” execution of
commodity anti-malware software
Capability I: “Out-of-the-box” execution of
commodity anti-malware software
Capability II: View comparison-based
stealthy malware detection
Capability II: View comparison-based
stealthy malware detection
VMwatcher: Bridging the Semantic Gap
Step 1: Procuring low-level VM states and events Disk blocks, memory pages, registers, … Traps, interrupts, …
Step 2: Reconstructing high-level semantic view Files, directories, processes, and kernel
modules,… System calls, context switches, …
VM Introspection
Guest View Casting
Step 1: VM Introspection
Raw VMM Observations
Virtual Machines (VMs)
VMware Academic Program
VM Disk Image
VM Hardware State (e.g., registers)
VM Physical Memory
VM-related low-level events (e.g., interrupts)
Step 2: Guest View Casting
Virtual Machine Monitor (VMM)
Guest OSDisk
Key observation: The guest OS already contains all necessary semantic definitions of data structures as well as
functionalities to construct the semantic view
VMwatcherSemantic
Gap
VirusScan Cross-view
Guest View Casting
Raw VMM Observations
Casted Guest Functions & Data
Structures
Reconstructed Semantic View
Device drivers, file system drivers
Memory translation,task_struct, mm_struct
CR3, MSR_SYSENTER_CS,MSR_SYSENTER_EIP/ESP
Event semantics Syscalls, Context switches, ....
Event-specific arguments…
VM Disk Image
VM Hardware State (e.g., registers)
VM Physical Memory
VM-related low-level events (e.g., interrupts)
Demo clip (3.5mins): http://www.ise.gmu.edu/~xjiang/
Guest View Casting on Memory State (Linux)
Process List
Process Memory Layout
Guest Memory Addressing
Traditional memory addressing Given a VA, MMU translates VA to PA OSes used to map with known PA
Linux: VA 0xc0000000 == PA 0x0 Windows: VA 0x80000000 == PA 0x0
VM complicates the translation
Guest virtual -> guest physical
Guest physical -> host physicalVM IntrospectionReverse Address Translation
Emulated Address Translation
Evaluation
Effectiveness Cross-view malware detection
Exp. I: Cross-view detection on volatile state Exp. II: Cross-view detection on persistent state Exp. III: Cross-view detection on both volatile and
persistent state Out-of-the-box execution of commodity anti-
malware software Exp. IV: Symantec AntiVirus Exp. V: Windows Defender
Performance Difference between internal scanning & external
scanning
Experiment Setup Guest VM: Windows XP (SP2)
Windows Fu Rootkit Host OS: Scientific Linux 4.4 VMM: VMware Server 1.0.1
Exp. I: Cross-view detection on volatile memory state
“Inside-the-box” viewVMwatcher view
Diff
Experiment Setup Guest VM: A Redhat 7.2-based honeypot
Linux SHv4 rootkit Host OS: Windows XP (SP2) VMM: VMware Server 1.0.1
Exp. II: Cross-view detection on persistent disk state
“Inside-the-box” viewVMwatcher view
Diff
Experiment (IV) Experiment Setup
Both guest OS and host OS run Windows XP (SP2) VMM: VMware Server 1.0.1
Running Symantec AntiVirus Twice Outside Inside
Hacker Defender
NTRootkit
External Scanning
Result
Internal Scanning
Result
Diff
Performance
Internal scanning time vs. external scanning time
Comparison of Scanning Time
0:00
4:48
9:36
14:24
19:12
Symantec AntiVirus10.1.396
Micorsoft Window sDefender 1.1.1592.0
Micorsoft MaliciousSoftw are Removal 1.2
Kaspersky Anti-Virus5.5
F-PROT AntiVirus 4.6.6 McAfee VirsScan4.24.0
Sophos Anti-Virus4.05.0
Scan
ning
Tim
e (m
in:s
ec)
Internal Scanning TimeExternal Scanning Time
18095 f iles
42724 f iles
10939 f iles
112556 f iles
10272 f iles 32269 f iles 11413 f iles
Internal scanning takes longer to complete !
Related Work
Enhancing security with virtualization (Livewire[Garfinkel03], IntroVirt[Joshi05],
HyperSpector[Kourai05]) Focusing on targeted attacks with specialized
IDSes Cross-view detection (Strider GhostBuster[Wang05],
RootkitRevealer/ Blacklight/IceSword/…) Either destroying the volatile state or
obtaining two internal views Secure monitors
CoPilot[Petroni04], Terra[Garfinkel03], sHype[Sailer05], SecVisor[Perrig07],TRANGO,…
Conclusions
VMwatcher – A systematic approach that bridges the semantic gap and enables two unique malware detection capabilities: Cross-view malware detection “Out-of-the-box” execution of commodity anti-
malware software
