differentially private aggregation of distributed time-series
DESCRIPTION
Differentially Private Aggregation of Distributed Time-Series. Vibhor Rastogi (University of Washington) Suman Nath (Microsoft Research). Participatory Data Mining. Untrusted Aggregator. How many people visit google.com and then visit yahoo.com on day i ?. google.com private.com - PowerPoint PPT PresentationTRANSCRIPT
Differentially Private Aggregation of Distributed Time-Series
Vibhor Rastogi (University of Washington)
Suman Nath (Microsoft Research)
Participatory Data Mining
Untrusted Aggregator
Alice Bob Charlie Delta
google.comprivate.com
hateBoss.com
yahoo.comlikeBoss.com
espn.com
facebook.comfindDate.comprivate.com
google.comprivate.comfindGift.com
How many people visit google.com and then visit yahoo.com on day i ?
• Web History • Medical Info• GPS Traces
Participatory Data Mining
Untrusted Aggregator
Alice Bob Charlie Delta
Week 10Weight: 120
Cholesterol: 60
Week 10Weight: 120
Cholesterol: 60
Week 10Weight: 120
Cholesterol: 60
Week 10Weight: 120
Cholesterol: 60
How many people weigh > 200 pounds and have cholesterol > 80 in week i?
• Web History • Medical Info• GPS Traces
Participatory Data Mining
Alice Bob Charlie Delta
How many people take a particular route at 5 PM? How many people take a particular route at 5:15PM? …
Privacy Concerns!
Untrusted Aggregator
• Web History • Medical Info• GPS Traces
Goal: Enable untrusted aggregator to query users’ time-series data with formal privacy
Current State-of-the-art
Alice Bob Charlie Delta
Traffic Analyzer
How many people were at 148th Street & 36th Ave at 5PM?
Current State-of-the-art
Alice Bob Charlie Delta
Traffic Analyzer
Yes No No Yes
Trusted Server How many people were at 148th Street & 36th Ave at 5PM?
Current State-of-the-art
Alice Bob Charlie Delta
Traffic Analyzer
Trusted Server
Actual answer =2
Noisy answer = 3.6
Formal Privacy achieved for right noiseNoise still small for a single query
How many people were at 148th Street & 36th Ave at 5PM?
Alice Bob Charlie Delta
Traffic Analyzer
At 5PM, were you at 148th Street & 36th Ave?
Current State-of-the-art
Formal Privacy achieved for right noiseNoise still small for a single query
Two Main Challenges
1. Noise in each answer = O(# of queries)
2. Trusted Server required
Trusted Server
Actual answer =2
Noisy answer = 3.6How many people at 148th Street & 36th Ave at 5PM? How many people at 148th Street & 36th Ave at 5:15PM?
How many people at 148th Street & 36th Ave at 7AM?
…
Noisy answer = 203.6
???
Outline
• Background: Differential Privacy• Challenge #1: Sequence of Queries• Challenge #2: No Trusted Server• Experimental Evaluation
Background: Differential privacy[Dwork 06]
For a sequence of queries q1,q2,…,qN
Laplace random noise added to each queryWorst case: noise has to increase linearly with N
56 + Laplace Noise
How many in Bldng99 at 5?
Privacy
Algorithm[Dwork 06]
Differential Privacy[Dwork 06] : Output should be indistinguishable:
VS.
Name Location Time
….. …. ….
Smith Building 99 5:00
Alice 148 & 36 5:00
….. …. ….
Name Location Time
….. …. ….
Smith Building 99 5:00
Name Location Time
….. …. ….
Smith Building 99 5:00
Alice 148 & 36 5:00
Alice 148 & 38 5:15
…. …. ….
Outline
• Background: Differential Privacy• Challenge #1: Sequence of Queries• Challenge #2: No Trusted Server• Experimental Evaluation
Answering Sequence of Queries
q1 = # of people in 148th & Sr 520 at 5:00PMq2 = # of people in 148th & Sr 520 at 5:15PM……qN = # of people in 148th & Sr 520 at 1:25AM
Standard algorithm[Dwork et. al. 06] result in Θ(N) noise
Noise too large for long sequences!
Name Location Time
….. …. ….
Smith Building 99 5PM
Name Location Time
….. …. ….
Smith Building 99 5PM
Alice 148th & Sr 520 5PM
Alice 148th & Sr 520 5:15PM
…. …. ….
Alice 148th & Sr 520 1:25AM
Solution: Compress the sequence
q’i has some error compared to qi
– Error is small if qi has periodic nature– k/N is the compression ratio
DFT-based Compression (NOT private):
Discrete Fourier Transform (DFT):Inverse
DFTDFT
DFT Inverse DFT
q1,…,qN f1,…,fN
Reduce effective N by compressing the sequence
q1,…,qN
q1,…,qN f1,..,fk,fk+1,..,fNq’1,…,q’Nf1,..,fk,0,..,0
DFT-based Compression - Examples
qiqi’k = 20N = 2000
DFT-based Compression - Examples
qiqi’ k = 10
N = 2000
Day #
Our DFT-based Perturbation Algorithm
• Perturbation error: O(N) to O(k)– An improvement of k/N
• Additional compression error often quite small
Our Algorithm1. q1,..,qN f1,..,fk
2. Perturb ’fi’ = fi + noise
3. f1',..,fk',0,0,…,0 q’1,…,q’N
Main ResultStrong differential privacy achieved;
Error in qi’ = O(k) + Compression error
DFT
Inverse DFT
Outline
• Background: Differential Privacy• Challenge #1: Sequence of Queries• Challenge #2: No Trusted Server• Experimental Evaluation
No Trusted ServerNo known efficient technique for distributed Laplace noise
Laplace noise = combination of Gaussian noiseGaussian noise can be generated distributedly
Individual noise too small, = (total noise/m)
Use cryptographic techniques to hide individual data despite small individual noise
Distributed Paillier Cryptosystem • Homomorphic encryption: add encrypted data• Threshold decryption: many private keys distributed among users decryption requires a threshold # of users
Basic Protocol
Alice Bob Charlie Delta
Traffic Analyzer
How many were at 148th Street & 36th Ave at 5PM?
Basic Protocol (Contd.)
Alice Bob Charlie Delta
Traffic Analyzer
1 0 0 1
Trusted Server
+noise +noise +noise +noiseE(1+noise) E(0+noise) E(0+noise) E(1+noise)
Addition over encrypted data Exploiting homomorphic propertyE(sum) = E(user1) * E(user2) * …
Basic Protocol (Contd.)
Alice Bob Charlie Delta
Traffic Analyzer
E(sum)
Basic Protocol (Contd.)
Alice Bob Charlie Delta
Traffic Analyzer
D1[E(sum)] D2[E(sum)] D3[E(sum)] D4[E(sum)]Each user partially decrypts using her key
Finally combines all decryption Exploiting threshold propertySum=D1[E(sum)] * D2[E(sum)] * …
One Tricky Challenge
Alice Bob Charlie Delta
Traffic Analyzer
E(sum)
During protocol, Encrypted aggregate sent back to the users
Third-party agent can be malicious
E(Alice’s Data)
Alice’s data is breached
D1[E(Alice’s data)] … … D4[E(Alice’s data)]
Outline
• Challenge #1: Handling correlations• Challenge #2: No trusted Server• Experimental Evaluation• Conclusion
Experimental Evaluation
• Implemented both solutions on– 2.8 GHz Intel Pentium PC with 1GB RAM
• Evaluated:– Accuracy improvement by Fourier perturbation– Performance overhead in distributed noise-addition
Fourier Perturbation: Real DatasetsSource: Predestination [Krum et. al. 08]GPS Traces
Fourier-based Standard[Dwork et. al. 06]
Source: hackers.comWeight Data
Fourier-based Standard[Dwork et. al. 06]
Distributed Noise Addition: Performance Overhead
• Computation Overhead
• Space Overhead– 0.5 Kb for each user– 0.5 Kb/user for the aggregator
Conclusion
• Participator Data Mining applications require– Accurately answer sequence of queries– Distributed noise-addition
• We saw a solution based on– Fourier compression & perturbation– Cryptographic protocols
Backup slides
Current State-of-the-art
Formal Privacy achieved for right noiseNoise still small for a single query
At 5PM, were you at 148th Street & 36th Ave?
1. Noise in each answer = O(# of queries)2. Trusted Server required
At 5:15PM, were you at 148th Street & 36th Ave?
At 7AM, were you at 148th Street & 36th Ave? …
Two Main Challenges
Alice Bob Charlie Delta
Traffic Analyzer
Yes No No Yes
Trusted Server
NoNoNoYes
Two main challengesChallenge #1: Correlations in Time-Series Data
Name Age Location Time
Alice 25 Building 99 5 PM
Alice 25 36th Street 5:02 PM
Alice 25 Building 112 5:03 PM
Bob 32 Building 99 5:35 PM
Lots of tuple correlations!
Building 99
36th Street
Building 112
Current privacy techniques can’t handle tuple correlations!
Two main challenges
Challenge #2: No trusted server
Alice Bob Charlie Delta
Traffic Analyzer
Were you at 520 bridge at 5 PM?
Yes No No Yes
Trusted ServerUsers add noise individually
NoNoNoYes
Total error grows with # of users!