Bootstrapping Privacy Bootstrapping Privacy Compliance in Big Data Compliance in Big Data SystemSystem

Shayak Sen, Saikat Guha et alCarnegie Mellon UniversityMicrosoft Research

Presenter: Cheng Li

We have your everythingWe have your everythingYour bank account

Your mobile

Your social network

Your shopping account

We will keep it as a secretWe will keep it as a secret

This is how we workThis is how we work

Legal team craft privacy policy

Privacy Champion interprets policy

Developer writes code

Audit Team verifies compliance

Life could be much easierLife could be much easier

encode

refine

code analysis

OutlineOutlineIntroductionLEGALEASE

◦Goal◦Syntax◦Domain-Specific Attribute◦Formal Semantics◦Properties

GROKValidationDiscussionConclusion

LEGALEASELEGALEASEGoal

◦Usability: Policy clauses are structured very similarly to clauses in English language policy.

◦Expressivity: Clauses are built around an attribute abstraction that allows the language to evolve as policy evolves.

◦Compositional Reasoning: LEGALEASE provides meaningful syntactic restrictions to allow compositional reasoning.

OutlineOutlineIntroductionLEGALEASE

◦Goal◦Syntax◦Domain-Specific Attribute◦Formal Semantics◦Properties

GROKValidationDiscussionConclusion

LEGALEASELEGALEASESyntax

Domain-Specific attributes are defined in concept lattice

LEGLEASE Policies are checked at each node in the data dependency graph.Each node is labeled with attr’s name and set of values.ALLOW: permits node labeled with subset of values.DENY: forbids node labeled with sets that overlaps the attribute values.

LEGALEASELEGALEASEExample

◦ Full IP address will not be used for advertising. IP address may be used for detecting abuse. In such cases it will not be combined with account information.

◦ DENY DataType IPAddress UseForPurpose AdvertisingEXCEPTALLOW DataType IPAddress:TruncatedALLOW DataType IPAddress UseForPurpose AbuseDetect EXCEPT DENY DataType IPAddress, AccountInfo

OutlineOutlineIntroductionLEGALEASE

◦Goal◦Syntax◦Domain-Specific Attribute◦Formal Semantics◦Properties

GROKValidationDiscussionConclusion

LEGALEASELEGALEASEDomain-specific Attribute

◦Attribute values are organized as a concept lattice.

◦Advantages of concept lattice: Abstracts away semantics. The lattice structure allows users to

concisely define sets of elements through their least upper bound.

The lattice structure allows us to statically check the policy for certain classes of errors.

LEGALEASELEGALEASEAttribute define in the

implementation◦InStore attribute: encode certain

policies around collection and storage of data.

LEGALEASELEGALEASEAttribute define in the

implementation◦UseForPurpose attribute: Encode the

data usage.

LEGALEASELEGALEASEAttribute define in the

implementation◦AccessByRole attribute: For encoding

internal access-control based policies.

LEGALEASELEGALEASEAttribute define in the

implementation◦DataType attribute:

Policy datatypes: types of data

LEGALEASELEGALEASEAttribute define in the

implementation◦DataType attribute:

Policy datatypes: Category of data types Limited typestate: A limited way of

tracking history.

LEGALEASELEGALEASEAttribute define in the

implementation◦DataType attribute:

Combining policy datatypes and typestates:

t:s where t is policy datatypes and s is typestates.

OutlineOutlineIntroductionLEGALEASE

◦Goal◦Syntax◦Domain-Specific Attribute◦Formal Semantics◦Properties

GROKValidationDiscussionConclusion

LEGALEASELEGALEASEFormal Semantics

◦Notions: T – a vector of sets of latice elements. Tx – the value of attribute x in T. TG – Graph node. TC – Policy clause vector.

LEGALEASELEGALEASEFormal Semantics

◦ where is ALLOW TC applies to a graph node TG if TG

⊑TC

◦ is for each x,

DENY TC applies to TG if

LEGALEASELEGALEASEFormal Semantics

◦A graph node is allowed by an ALLOW clause if and only if the clause applies and is allowed by each exception.

LEGALEASELEGALEASEFormal Semantics

◦A graph node is denied by an DENY clause if and only if the clause applies and is denied by each exception.

OutlineOutlineIntroductionLEGALEASE

◦Goal◦Syntax◦Domain-Specific Attribute◦Formal Semantics◦Properties

GROKValidationDiscussionConclusion

LEGALEASELEGALEASEProperties

◦Totality: C should either allow T or deny it.

◦Unicity: C cannot allow T and deny T at the same time.

◦Monotonicity: If C1 C2, then for any TG, C1 allows TG implies that C2 allows TG and C2;C2 denies TG implies C1 denies TG.

OutlineOutlineIntroductionLEGALEASEGROKValidationDiscussionConclusion

GROKGROKGROK SystemNodes are labeled with

attribute

Confidence value

Different granularity

GROKGROKData Flow Edges and Labeling

Nodes◦Log Analysis: Use log to bootstrap

the coarse-grained data flow graph Label file nodes with InStore attribute,

entity nodes with AccessByRole attribute. (high confidence)

Label UseForPurpose attribute for each job. (low confidence)

Log Analysis

GROKGROKData Flow Edges and Labeling

Nodes◦Syntactic Analysis: Label Datatype

attr by syntactically analyzing the source code of the job that read or wrote data. (low confidence)

Syntactic Analysis

GROKGROKData Flow Edges and Labeling

Nodes◦Semantic Analysis: Refine file nodes

to a collection of column nodes. Refine job nodes to a sub-graph of nodes.

Semantic Analysis

GROKGROKData Flow Analysis

◦Copy DataType attribute of one node to all nodes that data flows to.

◦Join two attributes that has the same confidence value.

◦If data flow through UDF(user defined function), check whether typestate has been modified. If it does, assign low confidence value.

GROKGROKVerifying Labels

◦Attributes verified by developers are assigned with high confidence value.

low = IPAddress

low confidence attribute

related source file related low confidence

attribute

low = IPAddresslow = UserAgent …

source file

reverse mapping Contact

the developer with highest-ranking source file

GROKGROKImplementation

GROK

static semantic analyzer

data flow analyzer

processes individual jobs from the cluster log into the nodes and edges in data dependency graph without attr

collates all the graph node, syntactic analysis and conservative data flow analysis, augmented with attrs.

OutlineOutlineIntroductionLEGALEASEGROKValidationDiscussionConclusion

ValidationValidationScale

◦ 100 day period, 77 thousand jobs each day, submitted by over 7 thousand entities in over 300 functional units.

◦ 1.1 million unique lines of code, 21% changes on a day-to-day basis.

ValidationValidationCoverage

simulate syntactic analyses on real-

world DDG

add dataflow analysis

add manual verification

ValidationValidationUsability

◦Online survey◦12 participants from Microsoft

privacy champions.◦Majority of participants were able to

use LEGALEASE to code policy clauses

ValidationValidationExpressiveness

OutlineOutlineIntroductionLEGALEASEGROKValidationDiscussionConclusion

DiscussionDiscussionExpressiveness: LEGALEASE cannot

express policies based on first-order temporal-logic. However, LEGALEASE is enough to express privacy policies.

Infer sensitive data: Unless explicitly labeled, GROK cannot detect inference from non-sensitive data to sensitive data.

Precision: Major source of precision comes from overly conservative treatment of UDF.

DiscussionDiscussionFalse Negatives: The authors are

unable to characterize the exact nature of false negatives in the system due to lack of ground truth.

Assurance: The system can not guarantee the result in face of adversarial developers’ behavior.

OutlineOutlineIntroductionLEGALEASEGROKValidationDiscussionConclusion

ConclusionConclusionAutomated privacy compliance

checking◦LEGALEASE: stating privacy policies as a

form of restrictions on information flows.◦GROK: data inventory that maps low level

data types in code to high level policy concepts.

Evaluation results show that◦LEGALEASE is expressive enough to capture

real-world privacy policies.◦GROK could bootstrap labeling the graph

with LEGALEASE at massive scale.