quantification of integrity michael clarkson and fred b. schneider cornell university ieee computer...

Quantification of Integrity

Michael Clarkson and Fred B. SchneiderCornell University

IEEE Computer Security Foundations SymposiumJuly 17, 2010

Clarkson: Quantification of Integrity 2

Information-theoreticQuantification of

programs’ impact onIntegrity

of Information

[Denning 1982]

What is Integrity?Common Criteria:

Protection of assets from unauthorized modificationBiba (1977):

Guarantee that a subsystem will perform as it was intended

Isolation necessary for protection from subversionDual to confidentiality

Databases:Constraints that relations must satisfyProvenance of dataUtility of anonymized data

…no universal definition

Our Notions of Integrity

Starting Point Corruption Measure

Taint analysis ContaminationProgram correctness

Suppression

Corruption: damage to integrity

Suppression

Contamination: bad information present in outputSuppression: good information lost from output

…distinct, but interact

Contamination

Goal: model taint analysis

ProgramUser

Attacker

trusted

untrusted

Contamination

Goal: model taint analysis

Untrusted input contaminates trusted output

ProgramUser

Attacker

trusted

untrusted

Contamination

u contaminates o

o:=(t,u)

Contamination

u contaminates o

(Can’t u be filtered from o?)

o:=(t,u)

Quantification of ContaminationUse information theory: information is

surprise

X, Y, Z: distributions

I(X,Y): mutual information between X and Y (in bits)

I(X,Y | Z): conditional mutual information

Quantification of Contamination

ProgramUser

Attacker

trusted

untrusted

ProgramUser

Attacker

trusted

untrusted

Tin Tout

Contamination = I(Uin,Tout | Tin)

ProgramUser

Attacker

trusted

untrusted

Tin Tout

[Newsome et al. 2009]

Dual of [Clark et al. 2005, 2007]

Example of Contamination

o:=(t,u)

Contamination = I(U, O | T) = k bits

if U is uniform on [0,2k-1]

Suppression

Contamination: bad information present in outputSuppression: good information lost from output

Program Suppression

Goal: model program (in)correctness

Sender Receivercorrect

Specification

(Specification must be deterministic)

Program Suppression

Implementation

Sender

Attacker

Receiver

Attacker

trusted

untrusted

SpecificationSender Receivercorrect

Program Suppression

Implementation

Sender

Attacker

Receiver

Attacker

trusted

untrusted

Sender Receiver

Implementation might suppress information about correct output from real output

correctSpecification

Example of Program Suppression

for (i=0; i<m; i++) { s := s + a[i]; }

a[0..m-1]: trusted

for (i=0; i<m; i++) { s := s + a[i]; }

for (i=1; i<m; i++) { s := s + a[i]; }

Impl. 1

Suppression—a[0] missing

No contamination

a[0..m-1]: trusted

for (i=0; i<m; i++) { s := s + a[i]; }

for (i=1; i<m; i++) { s := s + a[i]; }

for (i=0; i<=m; i++) { s := s + a[i]; }

Impl. 1 Impl. 2

Suppression—a[0] missing

No contamination

Suppression—a[m] added

Contamination

a[0..m-1]: trusted

a[m]: untrusted

Suppression vs. Contamination

Attacker

Contamination

Suppression

output := input

Quantification of Program Suppression

Implementation

Sender

Attacker

Receiver

Attacker

trusted

untrusted

SpecificationSender Receiver

Implementation

Sender

Attacker

Receiver

Attacker

trusted

untrusted

SpecificationIn Spec

Sender Receiver

Implementation

Sender

Attacker

Receiver

Attacker

trusted

untrusted

Specification

Tin Impl

In SpecSender Receiver

Implementation

Sender

Attacker

Receiver

Attacker

trusted

untrusted

Specification

Program transmission = I(Spec , Impl)

In Spec

Sender Receiver

Quantification of Program SuppressionH(X): entropy (uncertainty) of XH(X|Y): conditional entropy of X given Y

Program Transmission = I(Spec, Impl) = H(Spec) − H(Spec |

Impl)Total info to learn about Spec

Info actually learned about

Spec by observing Impl

Total info to learn about Spec

Info actually learned about

Spec by observing Impl

Total info to learn about Spec Info NOT learned

about Spec by observing Impl

Program Suppression = H(Spec | Impl)

for (i=0; i<m; i++) { s := s + a[i]; }

for (i=1; i<m; i++) { s := s + a[i]; }

for (i=0; i<=m; i++) { s := s + a[i]; }

Impl. 1 Impl. 2

Suppression = H(A) Suppression ≤ H(A)A = distribution of individual array elements

Belief-based Metrics

What if user’s/receiver’s distribution on unobservable inputs is wrong?

Belief-based information flow [Clarkson et al. 2005]

Belief-based generalizes information-theoretic:

On single executions, the same In expectation, the same …if user’s/receiver’s

distribution is correct

Suppression and ConfidentialityDeclassifier: program that reveals (leaks)

some information; suppresses rest

Leakage: [Denning 1982, Millen 1987, Gray 1991, Lowe 2002, Clark et al. 2005, 2007, Clarkson et al. 2005, McCamant & Ernst 2008, Backes et al. 2009]

Thm. Leakage + Suppression is a constant What isn’t leaked is suppressed

Database PrivacyStatistical database anonymizes query

results:

…sacrifices utility for privacy’s sake

AnonymizerUser

Database

Userquery

response

anonymized response

results:

…sacrifices utility for privacy’s sake…suppresses to avoid leakage

AnonymizerUser

Database

Userquery

response

anonymized response

anon. resp. := resp.

results:

…sacrifices utility for privacy’s sake…suppresses to avoid leakage…sacrifices integrity for confidentiality’s

AnonymizerUser

Database

Userquery

response

anonymized response

Database Privacy Security Conditionsk-anonymity: [Sweeney 2002] Every individual must be anonymous within set of size k. Every output corresponds to k inputs.

…no bound on leakage or suppressionL-diversity: [Øhrn and Ohno-Machado 1999, Machanavajjhala et

al. 2007] Every individual’s sensitive information should appear to have L

roughly equally likely values. Every output corresponds to L (roughly) equally likely inputs

…implies suppression ≥ log LDifferential privacy: [Dwork et al. 2006, Dwork 2006] No individual loses privacy by including data in database Output reveals almost no information about individual input

beyond what other inputs already reveal…implies almost all information about individual suppressed …quite similar to noninterference

Summary

Measures of information corruption:Contamination (generalizes taint analysis, dual

to leakage)Suppression (generalizes program correctness,

no dual)

Application: database privacy(model anonymizers; relate utility and privacy; security conditions)

More Integrity Measures Channel suppression

…same as channel model from information theory, but with attacker

Attacker- and program-controlled suppression

Granularity: Average over all executions Single executions Sequences of executions

…interaction of attacker with program

Application: Error-correcting codes

Quantification of Integrity

Michael Clarkson and Fred B. SchneiderCornell University

IEEE Computer Security Foundations SymposiumJuly 17, 2010

Beyond Contamination and Suppression? Clark–Wilson integrity policy Relational integrity constraints Software testing metrics …

Beyond Contamination & Suppression?

Attacker

Confidentiality Dual to Suppression?

Sender

Attacker

Receiver

Attacker

Public inputs lost from public outputs

output := input

Classic duality of confidentiality and integrity is incomplete

public

secret

Value of Information

What if some bits are worth more than others?

Discrete worth: security levelsTop secret, secret, confidential, unclassified

Continuous worth: ?

Bounded Reasoning

Our agents are logically omniscient.

Bounds?Algorithmic knowledge, computational entropy, …

InformationInformation is surprise.

X: random variable on set of events {e, …}

I(e): self-information conveyed by event eI(e) = − log2 Pr[X=e] (unit is bits)

Suppression vs. Contamination

n:=rnd();o:=t xor n

o:=t xor u

o:=(t,u)

t suppressed by noiseno contamination

t suppressed by uo contaminated by u

o contaminated by uno suppression

Echo Specification

output := inputSender Receivertrusted

Tin Tin

Echo Specification

Implementation

Sender

Attacker

Receiver

Attacker

trusted

untrusted

Tin Tin

Tin Timpl

Echo Specification

Implementation

Sender

Attacker

Receiver

Attacker

trusted

untrusted

Tin Tin

Tin Tout

Echo Specification

Implementation

Sender

Attacker

Receiver

Attacker

trusted

untrusted

Tin Tin

Tin Tout

Simplifies to information-theoretic model of channels, with attacker

Channel Suppression

Channel transmission = I(Tin,Tout)Channel suppression = H(Tin | Tout)

(Tout depends on Uin)

ChannelSender

Attacker

Receiver

Attacker

trusted

untrusted

Tin Tout

Probabilistic Specifications

Correct output: distributions on outputs

Correct output distribution: distribution on distribution on outputs

Continuous distributions, differential entropy?

o := rnd(1)

Duality

Contamination vs. Leakage

ProgramUser

Attacker

secret

public

ProgramUser

Attacker

trusted

untrustedContamination = I(Uin,Tout | Tin)

Leakage= I(Sin,Pout | Pin)

Leakage= I(Sin,Pout | Pin)[Denning 1982, Millen 1987, Gray 1991, Lowe 2002, Clark et al. 2005,

2007, Clarkson et al. 2005, McCamant & Ernst 2008, Backes et al. 2009]

Leakage= I(Sin,Pout | Pin)[Denning 1982, Millen 1987, Gray 1991, Lowe 2002, Clark et al. 2005,

2007, Clarkson et al. 2005, McCamant & Ernst 2008, Backes et al. 2009]

Contamination is dual to leakage

Confidentiality Dual to Suppression?

L-diversityEvery individual’s sensitive information

should appear to have L (roughly) equally likely values.[Machanavajjhala et al. 2007]

Entropy L-diversity: H(anon. block) ≥ log L[Øhrn and Ohno-Machado 1999, Machanavajjhala et al. 2007]

H(Tin | tout) ≥ log L (if Tin uniform)…implies suppression ≥ log L

“To Measure is to Know”

When you can measure what you are speaking about…you know something about it;

but when you cannot measure it…your knowledge is…meager and unsatisfactory… You have scarcely…advanced to the state of Science.

—Lord Kelvin

quantification of integrity michael clarkson and fred b. schneider cornell university ieee computer...

Documents

principal larry clarkson...education teaching clarkson...

clarkson university graduate catalog › sites › default...

venue - clarkson university

jeremy clarkson 2004 the world according to clarkson

kelly clarkson

disclaimer - clarkson university

clarkson university

clarkson public schools

clarkson lumbar company

clarkson powerpoint

aishah albarakati (clarkson university) (clarkson

kelly clarkson addicted

clarkson golden knights

quantification of integrity michael clarkson and fred b....

clarkson case study

clarkson, group imago

örnek clarkson

clarkson university physical therapy student handbook...

hyperproperties michael clarkson and fred b. schneider...

quantification of risk in norwegian stocks via the … ·...