VMM Based Rootkit Detection on Android

Class PresentationPete Bohman, Adam Kunk, Erik Shaw

Motivation

The Increase of Mobile Malware Variants (2004 – 2010)

• Smartphone malware on the rise• Increased security implications (compared to PC)• Sensitive information: GPS, contacts, SMS, call log• Constantly connected• Naïve users, limited use of Anti Virus

Defensive Rootkit Approaches• User mode rootkits

– Process infection, binary patching, lib hooks

• User mode integrity checkers– tripwire, chkrootkit, rkthunter, AV scanner

• Kernel mode rootkits– malicious device drivers and lkms– sys call hooking, kernel data structure manipulation

• Kernel level inspection– behavioral analysis, data structure integrity checkers, hook detection

• But… Any kernel level inspection mechanisms can be subverted by kernel level rootkits

Our Approach

• Two Pronged• KM security mechanisms • System call integrity checks• Hidden process detection• Android capability table

• VMM inspection • Ensures integrity of static KM• Isolated from host OS

• We exercise a “layer-below” level of security in which we establish trust beneath the kernel

Android Software Stack

Solution Preview (Delete Slide)Android Software Stack

Android VMM ensures integrity of static kernel module

Kernel module implements security mechanisms

Overview

• Design• VMM Design• Protected KM Design• Implementation• Results• Demo Presentation• Conclusion• Q&A

VMM Interface Design

Android VMM

Hardware (Emulator)

Linux Kernel

Trusted KM <ISR>

Libraries and Runtime

Application Framework

1. Hardware Timer Interrupt

2. Validate Protected KM 3. Raise Monitor Interrupt

4. Invoke KM

Protection KM Design

Linux KernelTrusted KMSystem Call WhitelistOriginal Sys Call Table

Libraries and Runtime

Maps

Application Framework

Contacts SMS App

Content Provider

Location Provider

Activity Manager

Open

Malicious Native

ApplicationSystem Calls

Open SocketRead GPSSQL QuerySysCall Table

…

Protection KM Design

Linux KernelTrusted KMSystem Call WhitelistOriginal Sys Call Table

Libraries and Runtime

X

MaliciousLKM

System Calls

SysCall Table

Y Z<ISR>

Android VMM

Monitor Interrupt

Implementation

• Implemented VMM security functionality in an emulated hardware device within QEMU• Protected KM data and text compiled into QEMU

emulator (VMM)

Linux Kernel Source with

Protection KMCompilation

Kernel Image

Protected Text Sect.

Protected Data Sect.

QEMU Emulator(VMM) Compilation

QEMU Emulator(VMM)Protected TextProtected Data

Implementation

• Malicious native mode application• Read contacts database• Read GPS location• Ex-filled data using sockets

• Malicious LKM• Intercept read system calls to access GPS location

Results

• We are able to detect/correct modifications to the sys_call_table• We are able to prevent malicious access to sensitive

resources• TODO Mention Malicious App and LKM

• TODO: (Insert link to demo)

Conclusion

• Layer Below Protection• Security of the Linux kernel must be rooted in a layer

below the kernel• Code contained solely in the kernel is subject to any

kernel-level attack

• Sensitive Resource Protection• Android mobile phones contain lots of sensitive

information that must be protected

App PermissionsDalvik VM IsolationApp signatures

Limits application abilities in order to prevent malicious behavior.

Virus ScannersRemote Lockout

Modified system binariesTrojan’d servicesStolen device

Linux user and group permissions

Access control

Architecture Layer Security Mechanism Threat Mitigation

Linux Kernel Source with

Protection KMCompilation

Kernel Image

Protected Text Sect.

Protected Data Sect.

QEMU Emulator(VMM) Compilation

QEMU Emulator(VMM)Protected TextProtected Data

Problem Statement

• Rootkit detection and prevention on the Android platform with specific regards to the sensitive resources Android provides.• Kapersky 2011: 1046 unique malware strains

targeting mobile platforms• Android platform built on Linux Kernel, a well known

target.• Sensitive information on smart phones• GPS, contacts, text messages, call log