shark : architectural support for autonomic protection against stealth by rootkit exploits
DESCRIPTION
Computer Architecture Lab at. SHARK : Architectural Support for Autonomic Protection Against Stealth by Rootkit Exploits. Vikas R. Vasisht , Hsien-Hsin S. Lee School of Electrical and Computer Engineering Georgia Tech. Rootkits. Programs used to conceal malware presence - PowerPoint PPT PresentationTRANSCRIPT
Reading Group, July 21 2009Based on slides retrieved from http://arch.ece.gatech.edu/present/micro41.pdf
SHARK: Architectural Support for Autonomic Protection Against Stealth by
Rootkit Exploits
Vikas R. Vasisht, Hsien-Hsin S. LeeSchool of Electrical and Computer Engineering
Georgia Tech
Computer Architecture Lab at
2
Rootkits• Programs used to conceal malware presence– E.g. hide keylogger or network activity
• Hard to detect– Anti-rootkit tool executes inside compromised system that
tries to hide the intrusion
3
Simple Rootkit
API Function
User Program
Library
System Administrator (e.g., “ps”, “top”)
Choose Interrupt Handler from IDT
Choose Syscall from SSDT
Syscall Function
ReturnImport Address Table
Interrupt Descriptor Table System Service Descriptor Table
Use
r Spa
ceKe
rnel
Spa
ce
Malware removed from list of
running processes
4
Existing Anti-Rootkit Tools• Signature-Based
Only works with known rootkits
• Behavior-Based– Calculate time and instructions spent on system queries
• Unusually high count indicates the O/S is manipulating response
Too many false positives
• Cross-level system view– Query system in high and low level
• E.g., compare ps response against scheduler lists
The compromised O/S can manipulate all levels
5
Sophisticated (Anti-)Rootkits• Integrity-based detection– Compare snapshot of
memory against trusted baselineCan be fooled
• Shadow-Walker– Mark all pages as unavailable– On TLB-i (execute) page fault:
present malicious code– On TLB-d (snapshot) page
fault: return ‘correct’ page
App Page Table
TLB-i View
TLB-d View
AUDIT OK!
6
Even More Sophisticated Rootkits
M2M1Mal O/S
App2App1Host O/SHardware
App2App1Host O/S
Virtual Machine MonitorHardware
• SubVirt– Install virtual machine
monitor– Downgrade host O/S to
guest O/S– Run Malicious O/S on the
side
• Blue Pill– Install tiny malicious
hypervisor on-the-fly
7
Outline• Motivation– Rootkits
• SHARK– High-level View– Page Table Encryption, Update, Decryption– Address Space Protection– Stealth Checker
• Evaluation• Security Vulnerabilities• Conclusion
8
Problem• Nearly impossible to detect all rootkits– Especially the virtualization-based ones: guest O/S is clean– All tools depend on the malicious O/S
• Proposed solution– Have the processor securely report every application that
enjoys hardware resources– Malicious apps running in different VMM will be exposed
9
SHARK: Bird’s Eye View• Hardware should record running processes– Must force O/S to give real PID to CPU
• Trusted third party audits system– Compare H/W and O/S reports– Discrepancy raises alarm
• O/S cannot hide a malicious app
• Tamper-evident address space– O/S blocked from manipulating normal apps
• Malware hijacking an app will leave a trail
• Idea: Page Table Encryption
10
Page Table Encryption: Key• App’s page table is encrypted using PID– Decrypted on-the-fly during hardware walk using same PID
• O/S assigned PIDs are vulnerable– Frequent reuse problematic for security
• SHARK provides hardware PID (HPID)– 64-bit counter: won’t overflow, nonce
• Only care for memory-resident rootkits, reset will eliminate them
• During app creation– O/S request PID and provides an initialized page table– H/W encrypts page table and returns HPID++
11
Page Table Encryption (cont)• Encrypt last level page table entries (counter mode)• Encrypt first & last level valid bits (counter mode)– Force same PID to be used during walk every time
• All updates through special H/W instruction– MODPT (modify page table)– Must provide correct PID
• Otherwise meaningless 1st level decryption
12
VTranslation Array
Valid Bit Array Encryption
V1
Translation Array
0…11…000
…
11…0Page Directory1 Page Frame
1st 128-bit Valid Bit BlockCounter = HPID||HPID
2nd 128-bit Valid Bit BlockCounter = (HPID||HPID)+1
N 128-bit Valid Bit BlockCounter = (HPID||HPID)+N-1
Counter Base (128 bit) HPID||HPID
Pipelined Counter Mode
Encryption Engine
AES-128
Hardware Secret Key: Burnt-in, unexposable
Offset
Actual Frame stored in memory
13
Page Table: Initial Encryption
VTranslation Array
…1001…
Last LevelPage Directory1 Page Frame
ADDRNULLNULLADDR
Counter (128 bit) HPID||HPID
Pipelined Counter Mode
Encryption Engine
AES-128
Hardware Secret Key: Burnt-in, unexposable
4 entries concatenated
128-bit
VTranslation Array
ADDR
Actual Frame stored in memory
14
Page Table: Entry Update
If other valid entries, decrypt and reencrypt, otherwise just encrypt
Counter (128 bit) (HPID||HPID)+k
V
Page Table Decryption – TLB update
15
V PDE
Counter Mode
DecryptionAES-128
Page DirectoryHW Key
…1001…
Normal Page Table Walk
V PDE
ADDR
Last Level
Counter (128 bit) HPID||HPID
Counter Mode
DecryptionAES-128
HW Key
V…0001…
PDE
ADDR4 entries
encrypted 128-bit
Counter (128 bit) HPID||HPID
Counter Mode
DecryptionAES-128
HW Key
4 entries decrypted
128-bit
TLB update
Address Space Protection• O/S should not tamper with address space of apps– Protect applications from swap-hijacking– Expose malware during auditing
• Shouldn’t mark as unavailable and return convenient page
• H/W signs pages and checks signatures
16
Tamper-evident swappingCaution: No check during first page mapping
Mem Page
4KB
32B
Disk
Counter (128 bit) HPID||HPID
Counter Mode
EncryptionAES-128
HW Key
SHA-256 checksum
Mem Page
4KB
32B
Disk
SHA-256 checksum
Counter Mode
EncryptionAES-128
=?
Valid/Invalid update
17
Stealth Checking• Firmware called on context switch– Record incoming PID of new process– Every 10 switches, timestamp, encrypt and call O/S to ship
Tamper-evident timestamped packet. Lost packets or O/S tampering trigger alarm.
OSContext Switch
HardwareHPID
PID1PID2PID3
Master PID List
Other SHARK H/W
P I D s …
Alarm
18
Outline• Motivation– Rootkits
• SHARK– High-level View– Page Table Encryption, Update, Decryption– Address Space Protection– Stealth Checker
• Evaluation• Security Vulnerabilities• Conclusion
19
Evaluation• Emulation using Bochs
– All installed rootkits detected
• Timing results using Simics– Close match to Core 2 processors– In-order simulation– First 2 billion instructions of SPEC apps
• Most page faults (and overhead) during initialization
20Expected: More TLB entries, less overhead
Evaluation (cont)
21
Outline• Motivation– Rootkits
• SHARK– High-level View– Page Table Encryption, Update, Decryption– Address Space Protection– Stealth Checker
• Evaluation• Security Vulnerabilities• Conclusion
22
Security Hole #1• 1st page map cannot be checked– Possible exploit by cleaner-app?
Mali-cious App’s Page Table
Name: xcalc
Audit app launched:Pagefault
Write Access
Note: cleaner-app will leave a PID trace in the stealth checker
All pages marked as
invalid, app works
through TLBs
Present malicious signature & restore
mapping, but freeze auditing
23
Security Hole #2• All page tables but the last (completely) and first
(valid bits only) are plaintext– O/S can manipulate internal mappings
VALID
VALID
VALID
VALID
View during executionView during auditing
Level 1 Level 2 Level 3 Data Pages
24
Security Hole #3• Security 101– Encryption: to keep something secret, private– Signing: to make something unmodifiable, tamper-evident
and bind message pieces together
• BAD idea to use one for the other– Creates illusion of protection
• Why encrypt the page table?– O/S already knows it, so why keep it a secret?– Signing is necessary, encryption is useless– Paper based on assumption that any tampering will result
in meaningless page table
25
Security Hole #3: Exploit
CounterH/W Key
128-bit plaintext
AES 128
⊕128-bit
Ciphertext
CounterH/W Key
128-bit plaintext
AES 128
⊕ 128-bit Ciphertext
Counter Mode Encryption Counter Mode Decryption
D(E(A)) = E(A) Cloud⊕ = A ⊕ Cloud
⊕ Cloud = A
O/S can overwrite E(A) with: (E(A), A, B known to O/S)E(A) A B = ⊕ ⊕ A Cloud ⊕ ⊕ A B = B Cloud = E(B)⊕ ⊕H/W will normally decrypt to B
E(A) = A Cloud⊕
26
Security Hole #3 (cont)• What they intended to say– The entry for virtual address V, for process PID, at level lvl,
is physical P (next level pointer or data location)
• How it should have been written– P, σΚ(P, V, PID, lvl) (σ = signature)– Bind each page table entry with where it should be located
• What was actually written– Some entries are secret; I won’t tell you
• Though you can ask me• …And you already know
27
Security Hole #4• The virtual memory system is used as a fence– (theoretically) tamper-evident page table
• O/S can simply bypass the VMM– Issue writes to physical address directly– O/S can purge malicious pages prior to auditing
• Similar to cleaner-app, but without any trace
28
Security Vulnerability #1• Stealth checker employs upgradeable firmware• Rootkit that is stored in disk can upgrade firmware– System reboot will not clean system– (Beyond the scope of this paper)
• Compromised O/S may compromise firmware– Enjoy the CPU spying on you– Impossible to detect
29
Security Vulnerability #2• BAD idea to reuse counter of counter-mode
encryption
VTranslation Array
…1001…
Last LevelPage Directory1 Page Frame
ADDRNULLNULLADDR
Counter (128 bit) HPID||HPID
Pipelined Counter Mode
Encryption Engine
AES-128
Hardware Secret Key: Burnt-in, unexposable
4 entries concatenated
128-bit
VTranslation Array
ADDR
Actual Frame stored in memory
Same for every last-level
encryption
30
Conclusion• Rootkits are a growing concern• Hardware monitoring is a good idea for protection– Malware cannot cheat hardware that easily– Good idea for rootkits also: BIOS rootkit in the works
• Paper’s implementation does more harm than good– Broken Algorithm: signing of each page table level is needed
• But O/S can bypass virtual memory manager• Cleaner-app is also a problem
• LBA should target O/S protection
31
Questions?
SYSTEM PROTECTED