1/19

Secretly Monopolizing the CPU Without Superuser

Privileges

Dan Tsafrir-IBM Watson, NY

Yoav Etsion-Hebrew University, Jerusalem

Dror Feitelson-Hebrew University, Jerusalem

2/19

In a nutshell

• We implement a “cheat” utility:

shell~/> cheat 90% program

(1) hostile aspect(2) concealment aspect

- ‘program’ would get 90% of the CPU cycles…

- …regardless on any background load

- ‘program’ appears as consuming 0%...

- …on some systems, when using tools like top, ps…

3/19

Unlike a rootkit

rootkitcheat

requires superuser access

any non-privileged user can easily do it

hostile-action (breaking in & installing

the rootkit)

concealment-capabilities (being able to secretly

use any computer resource)

concealment-capabilities (being able to secretly

use the CPU)

hostile action(being able to

monopolize the CPU)

4/19

The privileges-conflict axis

Attackers/defenders conflict often revolves around 1) Privileges of using resources, and 2) How long these privileges last

general

limited

cheatsecretly ctrl one resource

(can envision rootless rootkit)

obtain root & install rootkit: secretly ctrl all

resources

e.g. “cosmic rays attack” on JVMs [S&P

2003]privileges-conflict axis

5/19

Concealment: how?

rootkitcheat

requires superuser access

any non-privileged user can easily do it

hostile-action (breaking in & installing

the rootkit)

concealment-capabilities (being able to secretly

use any computer resource)

concealment-capabilities (being able to secretly

use the CPU)

hostile action(being able to

monopolize the CPU)

6/19

• Ticks = periodic hardware interrupts• OSs wake up HZ times a second:

• Since the 1960s (60Hz, like the grid)[Dijkstra; the “THE” system; CACM’68]

Operating System “ticks”

OSLinuxWindowsFreeBSD

2.22.42.6

HZ10010002501001000

every10ms1ms4ms10ms1ms

7/19

The tick-handler

• Tick-handler’s activities: 1) Deliver due alarm signals 2) CPU accounting 3) Initiating preemption (for multitasking) 4) ...

• Kernel’s “main loop”; its way to maintain control

8/19

Exampletick [1ms]

kernelnuclear simulator

movie player [display every 20ms]

n+1billrunsleep

n+2billrunsleep

…

n+19billrunsleep

n+20- bill - deliver alarm - switch

suspended

- display next frame- request alarm @ n+40- yield

switchrunsleepn+21billrunsleepn+22billrunseep

…

9/19

Concealment: how?

time

Run when OS not looking…(Not as easy as it sounds)

tick

n

tick

n

+1

tick

n

+2

tick

n

+3

cheater runs

cheater runs

cheater runs

10/19

Misaccounting: if we pull it off

time

The ‘honest’ process appears consuming 100%

tick

n

tick

n

+1

tick

n

+2

tick

n

+3

cheater runs

cheater runs

cheater runsh

on

est

hon

est

hon

est

hon

est

11/19

• The negative-feedback scheduling-principle - Running reduces priority to run more

• This principle guarantees 1) Fair allocation of CPU

2) Chronic sleepers (like emacs) get high priority

• Latter largely what makes editors responsive - And indeed, our cheater looks like an editor!

Monopolizing: how?

12/19

“Interactivity” makes it worse

• Turns out monopolizing is possible even without concealment capabilities…

• Scheduling for ‘interactivity’ raises ugly head: scheduling:traditionalrecent

workload:editors,

chronic sleepers

multimedia, soft realtime,

movie players, games

CPU demand:

lowhigh

prioritize by: sleep duration

(negative feedback)

sleep frequency(lots of deadlines)

13/19

Vulnerability spectrum

14/19

void cheat_attack( double fraction) {

work = fraction * cycles_per_tick();

nanosleep(0); // sync with ticktick_start = get_cycles();

while( 1 ) {now = get_cycles();

if( now - tick_start ≥ work ) {nanosleep(0); // avoid billtick_start = get_cycles();

}// do some short work here…

}}

The cheat algorithm

15/19

Evaluation

80%-cheater vs. honest-process

tick

tim

e [

ms]

time [sec]

tick

tim

e [

%]

cheaterhonestother

used CPU [ms]

bill

ed

CPU

[m

s]

16/19

‘Top’ snippet

PIDUSERPRSTAT%CPUTIMECOMMAND

5522dants21R99.30:07.79honest

5508dants16R0.30:00.04top

5246dants16S0.00:00.04csh

5259dants16S0.00:00.06csh

5509dants16S0.00:00.03bm-noklog.sh

…

5521dants15S0.00:00.00cheater

17/19

Cycle-accurate billingti

ck t

ime [

ms]

bill

ed

CPU

[m

s]

beforecheater

honest

time [sec] used CPU [ms]

after

tick

tim

e [

ms]

bill

ed

CPU

[m

s]honest

cheater

time [sec] used CPU [ms]

18/19

Curbing interactives preference

time [ms] used CPU [ms]b

illed

CPU

[m

s]

tick

tim

e [

ms]

cheater ( 50.02% )honest ( 49.97% )other ( 0.01% )

all the data zoom in

19/19

• Cheating allows unprivileged users to - seize CPU, often secretly

• To protect against cheating, one must 1) maintain accurate-enough info 2) incorporate info in scheduler (unlike Solaris, XP)

3) do it carefully (unlike Linux2.6, FreeBSD/ULE, XP)

• OS trend: prioritize based on sleep-frequency - plays straight to the hands of cheaters - probably fruitless - alternative: explicitly track relevant devices [NOSSDAV’04, TOMCCAP'06]

Conclusions

“Cheat” attack - Impact

• Linux scheduler:

Ingo Molnar, maintainer of the Linux Scheduler:

[ From the CFS-scheduler announcement people.redhat.com/mingo/cfs-scheduler/sched-design-CFS.txt ]

“Due to its design, the CFS scheduler is not prone to any of the 'attacks' that exist today'attacks' that exist today against the heuristics of the stock scheduler”

“Cheat” attack - Impact

• Amazon’s “Elastic Compute Cloud” (EC2)

– F. Zhou, M. Goel, P. Desnoyers, & R. Sundaram,“Scheduler vulnerabilities & coordinated attacks Scheduler vulnerabilities & coordinated attacks in cloud computingin cloud computing”, Northeastern U., Boston Technical Report, 2010

– Subverting Xen hypervisor with the cheat attack

– “Following the responsible disclosure model, we have reported this vulnerability to Amazon; they reported this vulnerability to Amazon; they have since implemented a fixhave since implemented a fix that we have tested and verified”

22/19

Backup…

23/19

Cheat accuracy

desired cheat rate [%]

ach

ieved

ch

eat

rate

[%

]

24/19

Effect of background load

all are honest one is 80%-cheater

number of competing processes

CPU

[%

]

all the othersone competitor

all the otherson 80% cheater

25/19

Running unmodified programs

Two approaches: 1) Cheat server 2) Binary instrumentation

26/19

The cheat-server protocol

27/19

Stealing a cluster with a P-III

cluster size [nodes; log scale]

CPU

[%

]

sum of 60%-cheaters (one per node)

sum of honest (10 per node)

28/19

Binary instrumentation

instrumentation granularity

slow

dow

n

instruction

blocktrace

function

selected function

29/19

Protecting against cheating: degrees of accuracy

• Two approaches 1) Solaris & Windows way

- Account for every kernel entry/exit (which e.g. slows down syscalls path)

2) BSD & Linux way

- Accounting by tick-handler only (inaccurate)

• Compromise - Accounting only upon a context-switch - Price: context-switch slower by 5%