quantification of integrity michael clarkson and fred b. schneider cornell university ieee computer...

Quantification of Integrity

Michael Clarkson and Fred B. SchneiderCornell University

IEEE Computer Security Foundations SymposiumJuly 17, 2010

Clarkson: Quantification of Integrity 2

Goal

Information-theoreticQuantification of

programs’ impact onIntegrity

of Information

[Denning 1982]


What is Integrity?Common Criteria:

Protection of assets from unauthorized modificationBiba (1977):

Guarantee that a subsystem will perform as it was intended

Isolation necessary for protection from subversionDual to confidentiality

Databases:Constraints that relations must satisfyProvenance of dataUtility of anonymized data

…no universal definition


Our Notions of Integrity

Starting Point Corruption Measure

Taint analysis ContaminationProgram correctness

Suppression

Corruption: damage to integrity





Suppression


Contamination: bad information present in outputSuppression: good information lost from output

…distinct, but interact


Contamination

Goal: model taint analysis

ProgramUser

Attacker

User

Attacker

trusted

untrusted


Contamination

Goal: model taint analysis

Untrusted input contaminates trusted output

ProgramUser

Attacker

User

Attacker

trusted

untrusted


Contamination

u contaminates o

o:=(t,u)


Contamination

u contaminates o

(Can’t u be filtered from o?)

o:=(t,u)


Quantification of ContaminationUse information theory: information is

surprise

X, Y, Z: distributions

I(X,Y): mutual information between X and Y (in bits)

I(X,Y | Z): conditional mutual information


Quantification of Contamination

ProgramUser

Attacker

User

Attacker

trusted

untrusted



ProgramUser

Attacker

User

Attacker

trusted

untrusted

Uin

Tin Tout



Contamination = I(Uin,Tout | Tin)

ProgramUser

Attacker

User

Attacker

trusted

untrusted

Uin

Tin Tout

[Newsome et al. 2009]

Dual of [Clark et al. 2005, 2007]


Example of Contamination

o:=(t,u)

Contamination = I(U, O | T) = k bits

if U is uniform on [0,2k-1]





Suppression


Contamination: bad information present in outputSuppression: good information lost from output


Program Suppression

Goal: model program (in)correctness

Sender Receivercorrect

Specification

(Specification must be deterministic)


Program Suppression


Implementation

Sender

Attacker

Receiver

Attacker

trusted

untrusted

SpecificationSender Receivercorrect

real


Program Suppression


Implementation

Sender

Attacker

Receiver

Attacker

trusted

untrusted

Sender Receiver

Implementation might suppress information about correct output from real output

real

correctSpecification


Example of Program Suppression

for (i=0; i<m; i++) { s := s + a[i]; }

Spec.

a[0..m-1]: trusted



for (i=0; i<m; i++) { s := s + a[i]; }

for (i=1; i<m; i++) { s := s + a[i]; }

Spec.

Impl. 1

Suppression—a[0] missing

No contamination

a[0..m-1]: trusted



for (i=0; i<m; i++) { s := s + a[i]; }

for (i=1; i<m; i++) { s := s + a[i]; }

for (i=0; i<=m; i++) { s := s + a[i]; }

Spec.

Impl. 1 Impl. 2

Suppression—a[0] missing

No contamination

Suppression—a[m] added

Contamination

a[0..m-1]: trusted

a[m]: untrusted


Suppression vs. Contamination

*

Attacker

*

Attacker

Contamination

Suppression

output := input


Quantification of Program Suppression

Implementation

Sender

Attacker

Receiver

Attacker

trusted

untrusted

SpecificationSender Receiver



Implementation

Sender

Attacker

Receiver

Attacker

trusted

untrusted

SpecificationIn Spec

Sender Receiver



Implementation

Sender

Attacker

Receiver

Attacker

trusted

untrusted

Specification

Uin

Tin Impl

In SpecSender Receiver



Implementation

Sender

Attacker

Receiver

Attacker

trusted

untrusted

Specification

Uin

Tin

Program transmission = I(Spec , Impl)

In Spec

Impl

Sender Receiver


Quantification of Program SuppressionH(X): entropy (uncertainty) of XH(X|Y): conditional entropy of X given Y

Program Transmission = I(Spec, Impl) = H(Spec) − H(Spec |

Impl)




Impl)Total info to learn about Spec




Impl)

Info actually learned about

Spec by observing Impl

Total info to learn about Spec




Impl)

Info actually learned about

Spec by observing Impl

Total info to learn about Spec Info NOT learned

about Spec by observing Impl




Impl)

Program Suppression = H(Spec | Impl)



for (i=0; i<m; i++) { s := s + a[i]; }

for (i=1; i<m; i++) { s := s + a[i]; }

for (i=0; i<=m; i++) { s := s + a[i]; }

Spec.

Impl. 1 Impl. 2

Suppression = H(A) Suppression ≤ H(A)A = distribution of individual array elements


Belief-based Metrics

What if user’s/receiver’s distribution on unobservable inputs is wrong?

Belief-based information flow [Clarkson et al. 2005]

Belief-based generalizes information-theoretic:

On single executions, the same In expectation, the same …if user’s/receiver’s

distribution is correct


Suppression and ConfidentialityDeclassifier: program that reveals (leaks)

some information; suppresses rest

Leakage: [Denning 1982, Millen 1987, Gray 1991, Lowe 2002, Clark et al. 2005, 2007, Clarkson et al. 2005, McCamant & Ernst 2008, Backes et al. 2009]

Thm. Leakage + Suppression is a constant What isn’t leaked is suppressed


Database PrivacyStatistical database anonymizes query

results:

…sacrifices utility for privacy’s sake

AnonymizerUser

Database

Userquery

response

anonymized response



results:

…sacrifices utility for privacy’s sake…suppresses to avoid leakage

AnonymizerUser

Database

Userquery

response

anonymized response

anon. resp. := resp.



results:

…sacrifices utility for privacy’s sake…suppresses to avoid leakage…sacrifices integrity for confidentiality’s

sake

AnonymizerUser

Database

Userquery

response

anonymized response


Database Privacy Security Conditionsk-anonymity: [Sweeney 2002] Every individual must be anonymous within set of size k. Every output corresponds to k inputs.

…no bound on leakage or suppressionL-diversity: [Øhrn and Ohno-Machado 1999, Machanavajjhala et

al. 2007] Every individual’s sensitive information should appear to have L

roughly equally likely values. Every output corresponds to L (roughly) equally likely inputs

…implies suppression ≥ log LDifferential privacy: [Dwork et al. 2006, Dwork 2006] No individual loses privacy by including data in database Output reveals almost no information about individual input

beyond what other inputs already reveal…implies almost all information about individual suppressed …quite similar to noninterference


Summary

Measures of information corruption:Contamination (generalizes taint analysis, dual

to leakage)Suppression (generalizes program correctness,

no dual)

Application: database privacy(model anonymizers; relate utility and privacy; security conditions)


More Integrity Measures Channel suppression

…same as channel model from information theory, but with attacker

Attacker- and program-controlled suppression

Granularity: Average over all executions Single executions Sequences of executions

…interaction of attacker with program

Application: Error-correcting codes

Quantification of Integrity

Michael Clarkson and Fred B. SchneiderCornell University

IEEE Computer Security Foundations SymposiumJuly 17, 2010


Beyond Contamination and Suppression? Clark–Wilson integrity policy Relational integrity constraints Software testing metrics …


Beyond Contamination & Suppression?

*

Attacker

*

Attacker


Confidentiality Dual to Suppression?

Sender

Attacker

Receiver

Attacker

Public inputs lost from public outputs

output := input

Classic duality of confidentiality and integrity is incomplete

public

secret


Value of Information

What if some bits are worth more than others?

Discrete worth: security levelsTop secret, secret, confidential, unclassified

Continuous worth: ?


Bounded Reasoning

Our agents are logically omniscient.

Bounds?Algorithmic knowledge, computational entropy, …


InformationInformation is surprise.

X: random variable on set of events {e, …}

I(e): self-information conveyed by event eI(e) = − log2 Pr[X=e] (unit is bits)


Suppression vs. Contamination

n:=rnd();o:=t xor n

o:=t xor u

o:=(t,u)

t suppressed by noiseno contamination

t suppressed by uo contaminated by u

o contaminated by uno suppression


Echo Specification

output := inputSender Receivertrusted

Tin Tin


Echo Specification

Implementation

Sender

Attacker

Receiver

Attacker

trusted

untrusted


Tin Tin

Tin Timpl

Uin


Echo Specification

Implementation

Sender

Attacker

Receiver

Attacker

trusted

untrusted


Tin Tin

Tin Tout

Uin


Echo Specification

Implementation

Sender

Attacker

Receiver

Attacker

trusted

untrusted


Tin Tin

Tin Tout

Uin

Simplifies to information-theoretic model of channels, with attacker


Channel Suppression

Channel transmission = I(Tin,Tout)Channel suppression = H(Tin | Tout)

(Tout depends on Uin)

ChannelSender

Attacker

Receiver

Attacker

trusted

untrusted

Tin Tout

Uin


Probabilistic Specifications

Correct output: distributions on outputs

Correct output distribution: distribution on distribution on outputs

Continuous distributions, differential entropy?

o := rnd(1)


Duality


Contamination vs. Leakage

ProgramUser

Attacker

User

Attacker

secret

public

ProgramUser

Attacker

User

Attacker

trusted

untrustedContamination = I(Uin,Tout | Tin)

Leakage= I(Sin,Pout | Pin)




Leakage= I(Sin,Pout | Pin)[Denning 1982, Millen 1987, Gray 1991, Lowe 2002, Clark et al. 2005,

2007, Clarkson et al. 2005, McCamant & Ernst 2008, Backes et al. 2009]




Leakage= I(Sin,Pout | Pin)[Denning 1982, Millen 1987, Gray 1991, Lowe 2002, Clark et al. 2005,

2007, Clarkson et al. 2005, McCamant & Ernst 2008, Backes et al. 2009]

Contamination is dual to leakage


Confidentiality Dual to Suppression?

No.


L-diversityEvery individual’s sensitive information

should appear to have L (roughly) equally likely values.[Machanavajjhala et al. 2007]

Entropy L-diversity: H(anon. block) ≥ log L[Øhrn and Ohno-Machado 1999, Machanavajjhala et al. 2007]

H(Tin | tout) ≥ log L (if Tin uniform)…implies suppression ≥ log L


“To Measure is to Know”

When you can measure what you are speaking about…you know something about it;

but when you cannot measure it…your knowledge is…meager and unsatisfactory… You have scarcely…advanced to the state of Science.

—Lord Kelvin

quantification of integrity michael clarkson and fred b. schneider cornell university ieee computer...

Documents