privacy preserving identi cation using ... -...
TRANSCRIPT
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 1 / 34
Privacy Preserving Identification UsingSparse Approximation with Ambiguization
Behrooz Razeghi, Slava Voloshynovskiy, Dimche Kostadinovand Olga Taran
Stochastic Information Processing GroupUniversity of Geneva
Switzerland
December 2017
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 1 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 2 / 34
Outline
Introduction
Proposed FrameworkMain IdeaSparse Data RepresentationAmbiguizationPrivacy-Preserving Identification
Results
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 2 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 3 / 34
Introduction
IntroductionPrivacy-preserving content identification
Biometrics
Physical object recognition and security
Medical/clinical applications
Privacy-sensitive multimedia records
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 3 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 3 / 34
Introduction
IntroductionPrivacy-preserving content identification
Biometrics
Physical object recognition and security
Recent Trends
Big Data & Distributed Applications
Services on outsourced
cloud-based systems
Medical/clinical applications
Privacy-sensitive multimedia records
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 3 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 3 / 34
Introduction
IntroductionPrivacy-preserving content identification
Biometrics
Physical object recognition and security
Recent Trends
Big Data & Distributed Applications
Services on outsourced
cloud-based systems
Medical/clinical applications
Privacy-sensitive multimedia records
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 3 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 4 / 34
Introduction
IntroductionProblem Formulation
Goal of privacy protection in outsourced services
y
x, yϕ(y)
ϕ (X)
[ϕ (x (1)) , ..., ϕ (x (M))]
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 4 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 5 / 34
Introduction
IntroductionHow do we receive a feature vector?
Encoding & Feature Aggregation
I Local/Global Descriptors
I BoW
I VLAD
I Fisher Vectors
I Triangulation Embedding
I · · ·
I Average Pooling
I Max Pooling
I Democratic Aggregation
1
N
I x(m) ∈ RN
Feature Vector
Layer 1 Layer 2 Layer L
Convolutional Neural Network
1
N
I x(m) ∈ RN
Feature Vector
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 5 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 6 / 34
Introduction
Introductionstate-of-the-art
Cryptographic Methods - Homomorphic Encryption
• Main Idea: Similarity search in the encrypted domain
• Brute force identification =⇒ huge complexity
Robust Hashing - a single hash from the whole content /local descriptors / last layer of CNN
• Main Idea: x −→ (011011100110) and believed non-invertability
• Loss in performance due to binarization
• Unauthorized database clustering
Group Testing / Memory Vectors
• Main Idea: Group testing by measuring the proximity to the group
representative
• Group representatives (memory vectors) should be stored in memory
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 6 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 6 / 34
Introduction
Introductionstate-of-the-art
Cryptographic Methods - Homomorphic Encryption
• Main Idea: Similarity search in the encrypted domain
• Brute force identification =⇒ huge complexity
Robust Hashing - a single hash from the whole content /local descriptors / last layer of CNN
• Main Idea: x −→ (011011100110) and believed non-invertability
• Loss in performance due to binarization
• Unauthorized database clustering
Group Testing / Memory Vectors
• Main Idea: Group testing by measuring the proximity to the group
representative
• Group representatives (memory vectors) should be stored in memory
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 6 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 6 / 34
Introduction
Introductionstate-of-the-art
Cryptographic Methods - Homomorphic Encryption
• Main Idea: Similarity search in the encrypted domain
• Brute force identification =⇒ huge complexity
Robust Hashing - a single hash from the whole content /local descriptors / last layer of CNN
• Main Idea: x −→ (011011100110) and believed non-invertability
• Loss in performance due to binarization
• Unauthorized database clustering
Group Testing / Memory Vectors
• Main Idea: Group testing by measuring the proximity to the group
representative
• Group representatives (memory vectors) should be stored in memory
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 6 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 7 / 34
Introduction
Introductionstate-of-the-art
Universal Quantization
• Main Idea: projection with the dimension reduction and periodic
quantization
• Binary quantization: in the region of low projected magnitudes -high Pb
• Ambiguization due to periodization of quantizer - no possibility torecover data even for the authorized users
• Server can still can cluster data - privacy leakages
• Information preservation in general - no link to R(d) and recovery is
demonstrated so far
a = ψ (Wx)
ti = [Wx]i
+1
−1
t
ψ(t)
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 7 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 8 / 34
Introduction
Introductionstate-of-the-art
Proposed approach: 3 key elements
• Sparsification• Ambiguization• Search / Identification
• Advantages:
• Fast search / memory efficient• Difficult to accurately reconstruct from probe• Server cannot reveal a structure of the database
• Main concerns addressed in our study:
• Performance• Memory (database) / complexity (identification)• Privacy-preserving with respect to:
database Aprobe y
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 8 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 8 / 34
Introduction
Introductionstate-of-the-art
Proposed approach: 3 key elements
• Sparsification• Ambiguization• Search / Identification
• Advantages:
• Fast search / memory efficient• Difficult to accurately reconstruct from probe• Server cannot reveal a structure of the database
• Main concerns addressed in our study:
• Performance• Memory (database) / complexity (identification)• Privacy-preserving with respect to:
database Aprobe y
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 8 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 8 / 34
Introduction
Introductionstate-of-the-art
Proposed approach: 3 key elements
• Sparsification• Ambiguization• Search / Identification
• Advantages:
• Fast search / memory efficient• Difficult to accurately reconstruct from probe• Server cannot reveal a structure of the database
• Main concerns addressed in our study:
• Performance• Memory (database) / complexity (identification)• Privacy-preserving with respect to:
database Aprobe y
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 8 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 9 / 34
Proposed Framework
Main Idea
Part 1:Sparse Data Representation
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 9 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 10 / 34
Proposed Framework
Main Idea
SparsificationMain Idea
1
N
I x(m) ∈ RN
I x(m) ∼ p (x)
Encoder
1
L ≥ N
I a(m) ∈ −1, 0,+1L
I ‖a(m)‖0 ≤ Sx
I Rate : R =1
Llog2
((L
Sx
)2Sx
)
Decoder
1
N
I x(m) ∈ RN
I Distortion : 1N ‖x(m)− x(m)‖22 ≤ D
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 10 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 10 / 34
Proposed Framework
Main Idea
SparsificationMain Idea
1
N
I x(m) ∈ RN
I x(m) ∼ p (x)
Encoder
1
L ≥ N
I a(m) ∈ −1, 0,+1L
I ‖a(m)‖0 ≤ Sx
I Rate : R =1
Llog2
((L
Sx
)2Sx
)
Decoder
1
N
I x(m) ∈ RN
I Distortion : 1N ‖x(m)− x(m)‖22 ≤ D
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 10 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 10 / 34
Proposed Framework
Main Idea
SparsificationMain Idea
1
N
I x(m) ∈ RN
I x(m) ∼ p (x)
Encoder
1
L ≥ N
I a(m) ∈ −1, 0,+1L
I ‖a(m)‖0 ≤ Sx
I Rate : R =1
Llog2
((L
Sx
)2Sx
)
Decoder
1
N
I x(m) ∈ RN
I Distortion : 1N ‖x(m)− x(m)‖22 ≤ D
Preservation of Information
Rate-Distortion Function
0 0.2 0.4 0.6 0.8 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 10 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 11 / 34
Proposed Framework
Main Idea
Part 2:Ambiguization
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 11 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 12 / 34
Proposed Framework
Main Idea
AmbiguizationMain Idea
1
L
I a(m) ∈ −1, 0,+1L
I ‖a(m)‖0 ≤ Sx
Ambiguization
addition ofnoisy samples
( )
1
L
I a(m)⊕
n
I Public Domain
Decoder
1
N
I x(m) ∈ RN
I ‖x(m)− x(m)‖22 →'2Nσ2x
I Prevent reconstruction from a(m) and from probe q
I Preclude server from discovering the structure of the database A
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 12 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 12 / 34
Proposed Framework
Main Idea
AmbiguizationMain Idea
1
L
I a(m) ∈ −1, 0,+1L
I ‖a(m)‖0 ≤ Sx
Ambiguization
addition ofnoisy samples
( )
1
L
I Public Domain
I a(m)⊕
n
Decoder
Even knowingtransform
1
N
I x(m) ∈ RN
I ‖x(m)− x(m)‖22 →'2Nσ2x
I Prevent reconstruction from a(m)⊕
n and from probe y
I Preclude server from discovering the structure of the database A
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 12 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 13 / 34
Proposed Framework
Main Idea
Part 3:Privacy-Preserving Identification
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 13 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 14 / 34
Proposed Framework
Main Idea
Privacy-Preserving Identification: Private SearchMain Idea: User discloses his probe completely
1
N
I y ∈ RN
1
L
I b ∈ −1, 0,+1L
I ‖b‖0 ≤ Sy
Encoder
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 14 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 14 / 34
Proposed Framework
Main Idea
Privacy-Preserving Identification: Private SearchMain Idea: User discloses his probe completely
1
N
I y ∈ RN
1
L
I b ∈ −1, 0,+1L
I ‖b‖0 ≤ Sy
Encoder
Public Sparse Storage
x(1) · · · x(M)
1
...
L
+100−10...
· · ·
· · ·
0−100+1...
ANN Decoder
List
L (y)
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 14 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 15 / 34
Proposed Framework
Main Idea
Privacy-Preserving Identification: Public SearchMain Idea: User sends only positions of interest
1
N
I y ∈ RN
1
L
I b ∈ −1, 0,+1L
I ‖b‖0 ≤ Sy
I Add Snq random positions
Encoder
Public List
Refined Encoding
List
L2 (y)
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 15 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 15 / 34
Proposed Framework
Main Idea
Privacy-Preserving Identification: Public SearchMain Idea: User sends only positions of interest
1
N
I y ∈ RN
1
L
I b ∈ −1, 0,+1L
I ‖b‖0 ≤ Sy
I Add Snq random positions
Encoder
Public Sparse Storage
x(1) · · · x(M)
1
...
L
+100−10...
· · ·
· · ·
0−100
+1...
ANN Decoderpy
Sy + Snq positions
Public List
L1 (y)
Refined Encoding
List
L2 (y)
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 15 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 15 / 34
Proposed Framework
Main Idea
Privacy-Preserving Identification: Public SearchMain Idea: User sends only positions of interest
1
N
I y ∈ RN
1
L
I b ∈ −1, 0,+1L
I ‖b‖0 ≤ Sy
I Add Snq random positions
sign (b)
Encoder
Public Sparse Storage
x(1) · · · x(M)
1
...
L
+100−10...
· · ·
· · ·
0−100
+1...
ANN Decoderpy
Sy + Snq positions
Public List
L1 (y)
Refined Decoder
Private List
L2 (y)
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 15 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 16 / 34
Proposed Framework
Main Idea
Main idea behind the proposed solutionOwner
X = x (1) , ...,x (m) , ...,x (M)
X ∈ XN×M
Encoder
a(m)=ϕx (x(m))︸ ︷︷ ︸
a(m)
⊕n
Public Storage
A = a (1) , ..., a (m) , ..., a (M)
A ∈ AL×M
x
y = x (m) + z
y = x
py (positions)
(Pubic Decoding)
List
L1 (y)
Refined Encoding
List
L2 (y)
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 16 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 16 / 34
Proposed Framework
Main Idea
Main idea behind the proposed solutionOwner
X = x (1) , ...,x (m) , ...,x (M)
X ∈ XN×M
Encoder
a(m)=ϕx (x(m))︸ ︷︷ ︸
a(m)
⊕n
Encoder
b = ϕy (y)
Public Storage
A = a (1) , ..., a (m) , ..., a (M)
A ∈ AL×M
p(y (m) | x (m))
x
Probe
Data User
y = x (m) + z
y = x
ANN Decoding
d (px (m) ,py) ≤ γL
1 ≤ m ≤M
py (positions)
(Pubic Decoding)
List
L1 (y)
Refined Encoding
List
L2 (y)
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 16 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 16 / 34
Proposed Framework
Main Idea
Main idea behind the proposed solutionOwner
X = x (1) , ...,x (m) , ...,x (M)
X ∈ XN×M
Encoder
a(m)=ϕx (x(m))︸ ︷︷ ︸
a(m)
⊕n
Encoder
b = ϕy (y)
Public Storage
A = a (1) , ..., a (m) , ..., a (M)
A ∈ AL×M
p(y (m) | x (m))
x
Probe
Data User
y = x (m) + z
y = x
ANN Decoding
d (px (m) ,py) ≤ γL
1 ≤ m ≤M
py (positions)
(Pubic Decoding)
List
L1 (y)
Refined Decoding
dsupp(b) (a (m)⊕
n,b) ≤ ζL
m ∈ L1 (y)
List
L2 (y)
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 16 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 17 / 34
Proposed Framework
Sparse Data Representation
Sparsifying TransformA Schematic Idea
1
N
1
L
1
L
ψ(Wx(m))
+1
−1
λx−λx
Linear Mapping Element-wise Non-linearity
x (m) ∈ RN Wx (m) ∈ RL a (m) ∈ −1, 0,+1L
ϕ(.)
W ψ (.)
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 17 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 18 / 34
Proposed Framework
Sparse Data Representation
Sparsifying TransformGeneral Problem Formulation
1
N
x(m) ∈ RN
Encoder
1
L ≥ N
a(m) ∈ −1, 0,+1L
Decoder
1
N
x(m) ∈ RN
Given
W
Random Learned from data
Encoder:
a (m) = ψ (Wx (m))
Decoder:
x (m) = W†a (m)
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 18 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 19 / 34
Proposed Framework
Sparse Data Representation
Encoder: as a projection problem (for a fixed W)
a(m)= arg mina(m)∈AL
‖Wx(m)−a(m)‖22 + βΩ (a(m)),∀m ∈ [M ]
- W ∈ RL×N , x(m) ∈ RN , a(m) ∈ RL
- Closed-form solution for: Ω (.) = ‖.‖0 and Ω (.) = ‖.‖1
ti
ψ(ti)
Hard-thresholding operator
Ω (.) = ‖.‖0
λx−λx ti
ψ(ti)
Soft-thresholding operator
Ω (.) = ‖.‖1
λx−λxti = [Wx]i
a (m) = ψ (Wx (m))
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 19 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 20 / 34
Proposed Framework
Sparse Data Representation
Encoder: Extra constraint on the alphabet
a (m) = ψ (Wx (m))
s.t. a (m) ∈ −1, 0,+1
ti
ψ(ti)
λx−λx ti
ψ(ti)
λx−λx
+1
−1
ti = [Wx]i
Sparse Ternary Code
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 20 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 20 / 34
Proposed Framework
Sparse Data Representation
Encoder: Extra constraint on the alphabet
a (m) = ψ (Wx (m))
s.t. a (m) ∈ −1, 0,+1
ti
ψ(ti)
λx−λx ti
ψ(ti)
λx−λx
+1
−1
ti = [Wx]i
Sparse Ternary Code
Remark:
Binary hashing (like LSH) is the special case of our ψ(.) for λx = 0.
+1
−1
ti
ψ(ti)
λx = 0
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 20 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 21 / 34
Proposed Framework
Sparse Data Representation
Comparison of Three Encoding Schemes
Binary Hashing
+1
−1
t
ψ(t)
Quantized Embeddings
+1
−1
t
ψ(t)
Sparse Ternary Coding
+1
−1
t
ψ(t)
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 21 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 22 / 34
Proposed Framework
Sparse Data Representation
Learning Sparsifying TransformGeneral Formulation: joint learning(
W, A)
= arg min(W,A)
‖WX−A‖2F + βWΩW (W) + βAΩA(A)
I Sparse Coding Step (Fixed W):
A = arg minA‖WX−A‖2F + βAΩA (A)
a (m) = ψ (Wx (m))
I Transform Update Step (Fixed A):
W = arg minW‖WX−A‖2F + βWΩW (W)
Linear Regression : W = AXT(XXT + βW IN
)−1
(with quadratic regularizer)
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 22 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 23 / 34
Proposed Framework
Ambiguization
Ambiguization SchemeMain Idea
Add noise to non-zero components of sparse representation
0 20 40 60 80 100 120-1.5
-1
-0.5
0
0.5
1
1.5
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 23 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 23 / 34
Proposed Framework
Ambiguization
Ambiguization SchemeMain Idea
Add noise to non-zero components of sparse representation
0 20 40 60 80 100 120-1.5
-1
-0.5
0
0.5
1
1.5
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 23 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 24 / 34
Proposed Framework
Privacy-Preserving Identification
Desired property of mapping schemeDistance preservation in the desired radius
Privacy
Preserving Region
y εN
x
εN
εN
‖x− y‖2
‖ϕ(x)− ϕ(y)‖2
Presereved Distances
Uniform Distances
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 24 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 25 / 34
Results
Impact of Ambiguization at Server Side
Goal: The server should not distinguish distances ‖ (ψ (Wx)⊕
n)− ψ (Wy) ‖2
Distances are computed in the full length.
0 2 4 6 80
2
4
6
8
10
12
14 ψ (Wx)
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 25 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 25 / 34
Results
Impact of Ambiguization at Server Side
Goal: The server should not distinguish distances ‖ (ψ (Wx)⊕
n)− ψ (Wy) ‖2
Distances are computed in the full length.
0 2 4 6 80
2
4
6
8
10
12
14 ψ (Wx)
ψ (Wx)⊕
n
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 25 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 25 / 34
Results
Impact of Ambiguization at Server Side
Goal: The server should not distinguish distances ‖ (ψ (Wx)⊕
n)− ψ (Wy) ‖2
Distances are computed in the full length.
0 2 4 6 80
2
4
6
8
10
12
14 ψ (Wx)
ψ (Wx)⊕
n
Server Side
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 25 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 26 / 34
Results
Impact of Ambiguization at Client Side
Goal: The client should distinguish distances(‖ (ψ (Wx)
⊕n)− ψ (Wy) ‖2
)supp
0 1 2 3 4 5 6 7 8 90
2
4
6
8
10
12
14 Owner’s Data: ψ (Wx)
Server’s Data: ψ (Wx)⊕
n
Client’s Data: ψ (Wy)
Distances are computed in the non-zero components of probe.
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 26 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 27 / 34
Results
Reconstruction: Authorized User I x = W†ax : i.i.d. Gaussian, with each sample Xn ∼ N (0, 1), N
L = 1Sx : Sparsity Level
0 0.2 0.4 0.6 0.8 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 27 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 28 / 34
Results
Reconstruction: Unauthorized User I x = W† (a⊕
n)
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
Half Ambiguization: Sns = 12(L− Sx) Full Ambiguization: Sns = (L− Sx)
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 28 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 29 / 34
Results
ClusteringGenerate Structured Data
I Generate:
- Four 512-dimensional i.i.d. vectors with distribution N (0,1)- 1000 512-dimensional i.i.d. vectors with distribution N (0,0.1)
I Add each 250 (out of 1000) low variance vectors to the fourhigh variance ones
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 29 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 29 / 34
Results
ClusteringGenerate Structured Data
I Generate:
- Four 512-dimensional i.i.d. vectors with distribution N (0,1)- 1000 512-dimensional i.i.d. vectors with distribution N (0,0.1)
I Add each 250 (out of 1000) low variance vectors to the fourhigh variance ones
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 29 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 30 / 34
Results
Clustering
Goal: Hide structure of database
Original Domain Public Domain
ψ (Wx (m))⊕
n
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 30 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 31 / 34
Results
ClusteringIntroduce Measure for Evaluation
Define:
I αx = SxL
, Sx : Sparsity level
Denote:
I Pintra : PDF of ‘intra-cluster’ distances
I Pinter : PDF of ‘inter-cluster’ distances
Define:
I P1 = αx Pintra + (1− αx)Pinter, 0 ≤ αx ≤ 1
Denote:
I P2 ∼ N(µ2, σ2
2
), fit to P1
Define:
I Privacy Leak Measure:
D (P1‖P2) = αxD (Pintra‖P2) + (1− αx)D (Pinter‖P2)
= EP1
[log
P1
P2
]
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 31 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 31 / 34
Results
ClusteringIntroduce Measure for Evaluation
Define:
I αx = SxL
, Sx : Sparsity level
Denote:
I Pintra : PDF of ‘intra-cluster’ distances
I Pinter : PDF of ‘inter-cluster’ distances
Define:
I P1 = αx Pintra + (1− αx)Pinter, 0 ≤ αx ≤ 1
Denote:
I P2 ∼ N(µ2, σ2
2
), fit to P1
Define:
I Privacy Leak Measure:
D (P1‖P2) = αxD (Pintra‖P2) + (1− αx)D (Pinter‖P2)
= EP1
[log
P1
P2
]
P1
intra inter
P2
D (P1‖P2) ↑
Clear distinguishability based on inter&intra-distances
P1
P2
D (P1‖P2) → 0
Not distinguishable
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 31 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 32 / 34
Results
Clustering: How much ambiguization should be added tohave indistinguishability for the server?Evaluation of Our Scheme: αx = Sx
L, βx =
SnsL
, Sns : # of noise components for theserver
0 5 10 15 20 25 30 35Distances
0
0.5
1
1.5
Est
imate
dPD
F
,x : 0:01
-x = 0-x = 0:03125-x = 0:125-x = 0:5-x = 1! ,x
D(P1kP2) = 0:1312
D(P1kP2) = 1:4778
D(P1kP2) = 0:0088
D(P1kP2) = 0:0132
0 5 10 15 20 25 30 35Distances
0
0.5
1
1.5
Est
imate
dPD
F
,x : 0:02
-x = 0-x = 0:03125-x = 0:125-x = 0:5-x = 1! ,x
D(P1kP2) = 0:0098
D(P1kP2) = 0:0321
D(P1kP2) = 9:5504 D(P1kP2) = 1:5719
0 5 10 15 20 25 30 35Distances
0
0.5
1
1.5
Est
imate
dPD
F
,x : 0:045
-x = 0-x = 0:03125-x = 0:125-x = 0:5-x = 1! ,x
D(P1kP2) = 0:5390
D(P1kP2) = 0:0363
D(P1kP2) = 17:4059 D(P1kP2) = 7:7313
0 5 10 15 20 25 30 35Distances
0
0.5
1
1.5Est
imate
dPD
F
,x : 0:09
-x = 0-x = 0:03125-x = 0:125-x = 0:5-x = 1! ,x
D(P1kP2) = 0:3152
D(P1kP2) = 3:4178
D(P1kP2) = 17:4405
D(P1kP2) = 12:4624
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 32 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 33 / 34
Results
Conclusions:
Preserve distances up to the desired radius
Ensure the reconstruction of data for authorized users
Preclude the curious server to cluster or reconstruct thesamples in the database
Public decoding scheme
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 33 / 34
Privacy Preserving Identification Using Sparse Approximation with Ambiguization 34 / 34
Results
Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 34 / 34