Hardware Counter Driven On-the-Fly Request Signatures

Kai Shen Ming Zhong Sandhya Dwarkadas

Chuanpeng Li Christopher Stewart Xiao Zhang

University of Rochester

04/19/23 ASPLOS 2008 2

Motivation

Hardware counters on modern processors: instruction mix, rate of execution, branch prediction

accuracy, memory access behavior

Operating system utilization of hardware counter metrics

Advantages as fine-grain workload signatures: application-transparency compared to application

statistics consistent availability compared to OS software statistics free fine-grain counter maintenance compared to

software statistics in general

04/19/23 ASPLOS 2008 3

On-the-Fly Request Signatures

Identifying requests for server workloads On-the-fly: identify a request while it still executes Utilizations:

Predicting request properties to guide OS adaptations Classifying requests on-the-fly to detect anomalies

0 20 40 60 80 100

4

6

8

10

x 10-4

Cumulative request execution (in millisec)

Flo

atin

g po

int

ops

per

CP

U c

ycle

TPC-H Q4TPC-H Q3TPC-H Q13

04/19/23 ASPLOS 2008 4

Challenges

Hardware metrics as workload signatures in server system environments

fluctuating concurrency and frequent context switches⇒ unstable hardware execution characteristics

requests are fine-grain workload units

Tracking request contexts within the OS on-the-fly transparent to applications

04/19/23 ASPLOS 2008 5

Hardware Metrics As Request Signatures:Choosing Normalization Base

Acquiring stable metrics as request executes: time-normalized metrics: divide by elapsed CPU cycles progress-normalized metrics: divide by retired

instructions

Finding: time-normalization for “time duration”-style metrics

(e.g., trace cache deliver mode)

0

0.1

0.2

0.3

0.4

0.5

Me

tric

-re

qu

est

-co

rre

latio

n

Non-halt CPU ticks

L2 misses (retired ins.)

L2 references

Load/store micro-ops

Trace cache deliver mode

Time-normalizedProgress-normalized

04/19/23 ASPLOS 2008 6

Hardware Metrics As Request Signatures:Choosing Effective Metrics

Environmental dynamics: concurrent request execution in server environments hardware resource-sharing – multi-threading and multi-core

Example metrics that are significantly affected:

0

0.1

0.2

0.3

0.4

0.5

Me

tric

-re

qu

est

-co

rre

latio

n

Load/store micro-ops

L2 misses

Data TLB pagewalk misses

Trace cache misses

L1 misses (retired ins.)

Serial executionConcurrent executionHyper-threading

04/19/23 ASPLOS 2008 7

Hardware Metrics As Request Signatures

Metric effectiveness across different applications inconsistent (e.g., floating-point ops very useful for

some but useless for others)⇒ Disappointing result: difficult to find a small set of

universally effective metrics

Require application-specific calibration

04/19/23 ASPLOS 2008 8

OS Support of Request Context Tracking

On-the-fly transparent tracking of request contexts Resource containers [Banga et al.’99] – not application-

transparent Magpie [Barham et al.’04] – not on-the-fly

High-level guidance: component activities reachable through control or data flows

are semantically related, and thus likely part of one request

One case: propagate request context through message passing

tag messages with senders’ request context IDs handle asynchronous messages, clarify message boundaries in

stream-based communications

04/19/23 ASPLOS 2008 9

Example of Request Context Propagation

Multi-tier RUBiS web server application

components database

Entirely at the OS

transparent to application

TomcatServelet

JbossInvoker

Jboss RMI-Disc

RUBiSQueryHome

RUBiSItemHome

MySQLDatabase

Request

... ... ... ... ... ... ... ...

... ... ... ... ... ... ... ...

Tim

e

04/19/23 ASPLOS 2008 10

Signature-driven Request Identification

Request identification: maintain a bank of recent past requests signature is a vector of metric statistics match each new request with banked requests on-the-

fly

Property inference: infer the property of new request using the property of

matched past request

04/19/23 ASPLOS 2008 11

Prototype

Platform Linux 2.6.10/Intel Xeon processors with hyper-threading

Overhead (not yet optimized):

0

0.5%

1.0%

1.5%

2.0%

Se

rve

r th

rou

gh

pu

t de

gra

da

tion

Metric collect/10msMetric collect/1msCollect/1ms with request identification

04/19/23 ASPLOS 2008 12

Evaluation Results:Accuracy of Predicting Request CPU Time

1 2 3 4 5 6 7 8 9 10

20%

40%

60%

80%

100%

Cumulative execution (in millisec)

Mea

n pr

edic

tion

erro

r

TPC-H

Running averageHardware counter

1 2 3 4 5 6 7 8 9 10

20%

40%

60%

80%

100%

Cumulative execution (in millisec)

Mea

n pr

edic

tion

erro

r

Comparison base (running average): the average properties of recent past requests to predict future requests

04/19/23 ASPLOS 2008 13

Utilization:Shortest-Job-First Scheduling

15-27% shorter response time than running average perform similar to oracle

3.1 3.2 3.3 3.4 3.50

2

4

6

Request rate (in reqs/sec)

Mea

n r

esp

ons

e tim

e (in

se

c) TPC-H

56 58 60 620

200

400

600

800

Request rate (in reqs/sec)

Mea

n r

esp

ons

e tim

e (in

mill

isec

)

RUBiS

Running averageHardware counterOracle

04/19/23 ASPLOS 2008 14

Utilization:Request Classification and Anomaly Detection

Dots are normal TPC-H requests

Circles are anomalies (SQL injection attacks)

10-ms cumulative metrics

0 2 4 6 8

x 10-3

0

1

2

3

4

5

6

7

8

x 10-3

Trace cache misses per m-instruction

Flo

atin

g p

oin

t o

ps

pe

rm

04/19/23 ASPLOS 2008 15

Related Work

Other uses of hardware counters phase detection [Dhodapkar&Smith’02, Sherwood et al.’03] behavior prediction [Duesterwald et al.’03, Bulpin&Pratt’05] anomaly tracking [Sweeney et al.’04]⇒ we handle challenges due to dynamic server environments

Request characterization using system software metrics tracking request/response [Aguilera et al.’03] request modeling [Barham et al.’04] failure diagnosis [Chen et al.’04]⇒ hardware metrics have unique advantages: consistent

availability, free fine-grain counter maintenance

First to realize on-the-fly request signatures for server workloads.

04/19/23 ASPLOS 2008 16

Conclusion

Our contributions: investigate the effectiveness of hardware counter

metrics as request signatures in dynamic server environments

propose OS mechanism to support on-the-fly request context tracking and adaptation

demonstrate the effectiveness of request signature-enabled on-the-fly OS exploitations

High-level takeaway: OS exploitation of hardware metrics to improve

performance and dependability [HotOS’07]