Quantification of Integrity

Michael Clarkson and Fred B. SchneiderCornell University

RADICALMay 10, 2010

Clarkson: Quantification of Integrity 2

Goal

Information-theoreticQuantification of

programs’ impact onIntegrity

of Information

(relationship to database privacy)

[Denning 1982]

Clarkson: Quantification of Integrity 3

What is Integrity?Databases:

Constraints that relations must satisfyProvenance of dataUtility of anonymized data

Common Criteria:Protection of assets from unauthorized modification

Biba (1977):Guarantee that a subsystem will perform as it was

intended;Isolation necessary for protection from subversion;Dual to confidentiality

…no universal definition

Clarkson: Quantification of Integrity 4

Our Notions of Integrity

Starting Point Corruption Measure

Taint analysis ContaminationProgram correctness

Suppression

Corruption: damage to integrity

Contamination: bad information present in outputSuppression: good information lost from output

…distinct, but interact

Clarkson: Quantification of Integrity 5

Contamination

Goal: model taint analysis

Untrusted input contaminates trusted output

ProgramUser

Attacker

User

Attacker

trusted

untrusted

Clarkson: Quantification of Integrity 6

Contamination

u contaminates o

(Can’t u be filtered from o?)

o:=(t,u)

Clarkson: Quantification of Integrity 7

Quantification of ContaminationUse information theory: information is

surprise

X, Y, Z: distributions

I(X,Y): mutual information between X and Y (in bits)

I(X,Y | Z): conditional mutual information

Clarkson: Quantification of Integrity 8

Quantification of Contamination

Contamination = I(Uin,Tout | Tin)

ProgramUser

Attacker

User

Attacker

trusted

untrusted

Uin

Tin Tout

[Newsome et al. 2009]

Dual of [Clark et al. 2005, 2007]

Clarkson: Quantification of Integrity 9

Example of Contamination

o:=(t,u)

Contamination = I(U, O | T) = k bits

if U is uniform on [0,2k-1]

Clarkson: Quantification of Integrity 10

Our Notions of Integrity

Starting Point Corruption Measure

Taint analysis ContaminationProgram correctness

Suppression

Corruption: damage to integrity

Contamination: bad information present in outputSuppression: good information lost from output

Clarkson: Quantification of Integrity 11

Program Suppression

Goal: model program (in)correctness

Implementation

Sender

Attacker

Receiver

Attacker

trusted

untrusted

Sender Receiver

Information about correct output is suppressed from real output

real

correctSpecification

Clarkson: Quantification of Integrity 12

Example of Program Suppression

for (i=0; i<m; i++) { s := s + a[i]; }

for (i=1; i<m; i++) { s := s + a[i]; }

for (i=0; i<=m; i++) { s := s + a[i]; }

Spec.

Impl. 1 Impl. 2

Suppression—a[0] missing

No contamination

Suppression—a[m] added

Contamination

a[0..m-1]: trusted

Clarkson: Quantification of Integrity 13

Suppression vs. Contamination

*

Attacker

*

Attacker

Contamination

Suppression

output := input

Clarkson: Quantification of Integrity 14

Quantification of Program Suppression

Implementation

Sender

Attacker

Receiver

Attacker

trusted

untrusted

Specification

Uin

Tin

Program transmission = I(Spec , Impl)

In Spec

Impl

Sender Receiver

Clarkson: Quantification of Integrity 15

Quantification of Program SuppressionH(X): entropy (uncertainty) of XH(X|Y): conditional entropy of X given Y

Program Transmission = I(Spec, Impl) = H(Spec) − H(Spec |

Impl)

Info actually learned about

Spec by observing Impl

Total info to learn about Spec Info NOT learned

about Spec by observing Impl

Clarkson: Quantification of Integrity 16

Quantification of Program SuppressionH(X): entropy (uncertainty) of XH(X|Y): conditional entropy of X given Y

Program Transmission = I(Spec, Impl) = H(Spec) − H(Spec |

Impl)

Program Suppression = H(Spec | Impl)

Clarkson: Quantification of Integrity 17

Example of Program Suppression

for (i=0; i<m; i++) { s := s + a[i]; }

for (i=1; i<m; i++) { s := s + a[i]; }

for (i=0; i<=m; i++) { s := s + a[i]; }

Spec.

Impl. 1 Impl. 2

Suppression = H(A) Suppression ≤ H(A)A = distribution of individual array elements

Clarkson: Quantification of Integrity 18

Suppression and ConfidentialityDeclassifier: program that reveals (leaks)

some information; suppresses rest

Leakage: [Denning 1982, Millen 1987, Gray 1991, Lowe 2002, Clark et al. 2005, 2007, Clarkson et al. 2005, McCamant & Ernst 2008, Backes et al. 2009]

Thm. Leakage + Suppression is a constant What isn’t leaked is suppressed

Clarkson: Quantification of Integrity 19

Database PrivacyStatistical database anonymizes query

results:

…sacrifices utility for privacy’s sake…suppresses to avoid leakage…sacrifices integrity for confidentiality’s

sake

AnonymizerUser

Database

Userquery

response

anonymized response

Clarkson: Quantification of Integrity 20

k-anonymityDB: Every individual must be anonymous

within set of size k. [Sweeney 2002]

Programs: Every output corresponds to k inputs.

But what about background knowledge?

…no bound on leakage…no bound on suppression

Clarkson: Quantification of Integrity 21

L-diversityDB: Every individual’s sensitive information

should appear to have L (roughly) equally likely values.[Machanavajjhala et al. 2007]

Entropy L-diversity: H(anon. block) ≥ log L[Øhrn and Ohno-Machado 1999, Machanavajjhala et al. 2007]

Program: H(Tin | tout) ≥ log L (if Tin uniform)

…implies suppression ≥ log L

Clarkson: Quantification of Integrity 22

Summary

Measures of information corruption:Contamination (generalizes taint analysis)Suppression (generalizes program correctness)

Application: database privacy(model anonymizers; relate utility and privacy)

Clarkson: Quantification of Integrity 23

More Integrity Measures Channel suppression

…same as channel model from information theory, but with attacker

Attacker- and program-controlled suppression Belief-based measures [Clarkson et al. 2005]

…generalize information-theoretic measures

Granularity: Average over all executions Single executions Sequences of executions