Seminar in Foundations of Privacy

1. Adding Consistency to Differential Privacy2. Attacks on Anonymized Social Networks

Inbal TalgamMarch 2008

1. Adding Consistency to Differential Privacy

Differential Privacy

• 1977 Dalenius - The risk to one’s privacy is the same with or without access to the DB.

• 2006 Dwork & Naor – Impossibe (auxiliary info).• 2006 Dwork et al – The risk is the same with or

without participating in the DB.

Plus: Strong mechanism of Calibrated Noise to achieve DP while maintaining accuracy.

• 2007 Barak et al - Adding consistency.

Setting – Contingency Table and Marginals

k binary attributes

n participants DB

0 1 0 0 1 1 1 0

0 0 1 0 1 0 …

Terminology: Contingency table (private), marginals (public).

# # …

2k attribute settings

0…0 0…1 …

Contingency Table

8 3 …

2j attribute settings

0 9 …

2i attribute settings

Marginals

j << k

Main Contribution

• Solve following consistency problem:

• At low accuracy cost

2 0 …

Marginals

Noise NaN -0.5 …

Contingency Table

+

Outline

• Discussion of:1. Privacy

2. Accuracy & Consistency

• Key method - Fourier basis

• The algorithm– Part I– Part II

Privacy – Definition• Intuition: The risk is the same with or without

participating in the DB• Definition:

DB1 DB2Differing on 1 element

A randomized function K gives ε-differential privacy if

for all DB1, DB2 differing on at most 1 element

)exp(

)(

)(

2

1 SDBKPR

SDBKPR

Privacy - Mechanism

Noise

Pls let me know f(DB)

DB

Goal: Noise

K(DB) = f(DB)+

NoiseLaplace noise:

Pr[K(DB)=a]

exp (||f(DB) - a||1 / σ)

The Calibrated Noise Mechanismfor DP

• Main idea: Amount of noise to add to f(DB) is calibrated according to the sensitivity of f, denoted Δf.

• Definition:

• All useful functions should be insensitive…

(e.g. marginals)

For f : D → Rd, the L1-sensitivity of f is

for all DB1, DB2 differing on at most 1 element121

,)()(max

21

DBfDBffDBDB

The Calibrated Noise Mechanism – How Much Noise

• Main result: To ensure ε-differential privacy for a query of sensitivity Δf, add Laplace noise with σ = Δf/ε.

• Why does it work? Remember: Laplace: Definition:

Pr[K(DB)=a]exp (||f(DB) - a||1 / σ)

)exp(

)(

)(

2

1 SDBKPR

SDBKPR

Accuracy & Consistency

8 3 …

Contingency Table

2 0 …

Marginals

Noise+

NaN -0.5 …

New Table

• Compromise consistency

• May lead to technical problems and confusion

So smoking is one of the

leading causes of statistics?

8 3 …

Contingency Table

+

Noise

3 2 …

Marginals

• Compromise accuracy

• Non-calibrated, binomial noise Var=Θ(2k)

Key Approach

• Non-redundant representation

• Specific for required marginals

8 3 …

Contingency Table

2 0 …

Marginals

+

Small number of coefficients of the Fourier

basis

Consistency:

Any set of Fourier coefficients correspond

to a (fractional and possibly negative) contingency table.

Accuracy:

Few Fourier coefficients are needed for low-

order marginals, so low sensitivity and small

error.

Noise

+

Linear Programming +

Rounding

Accuracy – What is Guaranteed

• Let C be a set of original marginals, each on ≤ j attributes.

• Let C’ be the result marginals.

• With probability 1-δ, :

• Remark: Advantage of working in the interactive model.

Cc

DB

CCCcc j /)/log(2' 3

1

Outline

• Discussion of:1. Privacy

2. Accuracy & Consistency

• Key method - Fourier basis

• The algorithm– Part I– Part II

Notation & Preliminaries

• ||x||1 = ?

• We say α ≤ β if β has all α’s attributes (and more) e.g. 0110 ≤ 0111 but not 0110 ≤ 0101

• Introduce the linear marginal operator Cβ

β determines attributes

• Remember: xα, α ≤ β, Cβ(x), Cβ(x)γ

# # …

Contingency Table

x0…0 x0…1 xα where k}1,0{

:2kRx

:

))(( xxC2 0 …

Marginal

Cβ(x) :

The Fourier Basis

• – Orthonormal basis for space of contingency

tables x (R2k).

• Motivation: Any marginal Cβ(x) can be written as a combination of few fα’s.– How few? Depends on order of marginal.

• fα:

}}1,0{|{ kf

2/, 2/)1( kf …2/2/1 k

Writing marginals in Fourier Basis

• Theorem: 0fC

fCxffxfCxC ,,Marginal of x with

attributes β

Write x in Fourier basis

Linearity

fCxf ,

Proof. For any coordinate

:

2/, 2/)1( kfC

By definition of marginal operator and Fourier vector

Outline

• Discussion of:1. Privacy

2. Accuracy & Consistency

• Key method - Fourier basis

• The algorithm– Part I – adding calibrated noise– Part II – non-negativity by linear

programming

Algorithm – Part I

INPUT: Required marginals {Cβ}• {fα} = Fourier vectors needed to write marginals• Releasing marginals {Cβ(x)} = releasing coeffs <fα,x>

OUTPUT: Noisy coeffs {Φα}

METHOD: Add calibrated noise• Sensitivity depends on |{α}| on order of Cβ’s

8 3 …

Contingency Table

2 0 …

Marginals

+

Small number of coefficients of the Fourier

basis

Noise

+

8 3 …

Contingency Table

8 3 …

Contingency Table

2 0 …

Marginals

2 0 …

Marginals

+

Small number of coefficients of the Fourier

basis

NoiseNoise

+

fCxf ,

Part II – Non-negativity by LPINPUT: Noisy coeffs {Φα} OUTPUT: Non-negative contingency table x'METHOD: Minimize difference between Fourier coefficients

• Most entries x'γ in a vertex solution are 0 Rounding adds small error

minimize b

subject to:

x'γ ≥ 0

|Φα - <fα,x'>| ≤ b

Algorithm Summary

Input: Contingency table x, required marginals {Cβ} Output: Marginals {Cβ} of new contingency table x''

• {fα} = Fourier vectors needed to write marginals• Compute noisy Fourier coefficients {Φα}

• Find non-negative x' with nearly the correct Fourier coefficients

• Round to x''

)/(, Lapxf

',min xf

Part I

Part II

}{/)/}{log(}{22 jRounding

LP

Bound on Laplace noise per coefficient

Accuracy Guarantee - Revisited

• With probability 1-δ, 1'cc

#Coefficients

Summary & Open Questions

• Algorithm for marginals release• Guarantees privacy, accuracy & consistency

– Consistency: can reconstruct a synthetic, consistent table

– Accuracy: error increases smoothly with order of marginals

• Open questions: – Improving efficiency – Effect of noise on marginals’ statistical properties

Any Questions?