cse 544: lecture 5-6 theory

78
1 CSE 544: Lecture 5-6 Theory Wednesday, April 10/12, 2006

Upload: slone

Post on 06-Feb-2016

44 views

Category:

Documents


0 download

DESCRIPTION

CSE 544: Lecture 5-6 Theory. Wednesday, April 10/12, 2006. Conjunctive Queries. A subset of FO queries Correspond to SELECT-DISTINCT-FROM-WHERE Most queries in practice are conjunctive Some optimizers handle only conjunctive queries - break larger queries into many CQs - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: CSE 544: Lecture 5-6 Theory

1

CSE 544: Lecture 5-6Theory

Wednesday, April 10/12, 2006

Page 2: CSE 544: Lecture 5-6 Theory

2

Conjunctive Queries

• A subset of FO queries• Correspond to SELECT-DISTINCT-FROM-WHERE

• Most queries in practice are conjunctive

• Some optimizers handle only conjunctive queries - break larger queries into many CQs

• CQ’s have more positive theoretical properties than arbitrary queries

Page 3: CSE 544: Lecture 5-6 Theory

3

Conjunctive Queries

• Definition A conjunctive query is defined by:

• missing are , ,

• CQ FO

::= R(t1, ..., tar(R)) | ti = tj | ’ | x. ::= R(t1, ..., tar(R)) | ti = tj | ’ | x.

Page 4: CSE 544: Lecture 5-6 Theory

4

Conjunctive Queries, CQ

• Example of CQ

• Examples of non-CQ:

q(x,y) = z.(R(x,z) u.(R(z,u) R(u,y)))

q(x) = z.u.(R(x,z) R(z,u) R(u,y))

q(x,y) = z.(R(x,z) u.(R(z,u) R(u,y)))

q(x) = z.u.(R(x,z) R(z,u) R(u,y))

q(x,y) = z.(R(x,z) R(y,z))

q(x) = T(x) z.S(x,z)

q(x,y) = z.(R(x,z) R(y,z))

q(x) = T(x) z.S(x,z)

Page 5: CSE 544: Lecture 5-6 Theory

5

Conjunctive Queries

• Any CQ query can be written as:

(i.e. all quantifiers are at the beginning)

• Same in Datalog notation:

q(x1,...,xn) = y1.y2...yp.(R1(t11,...,t1m) ... Rk(tk1,...,tkm))q(x1,...,xn) = y1.y2...yp.(R1(t11,...,t1m) ... Rk(tk1,...,tkm))

q(x1,...,xn) :- R1(t11,...,t1m), ... , Rk(tk1,...,tkm))q(x1,...,xn) :- R1(t11,...,t1m), ... , Rk(tk1,...,tkm))

head body

Datalog rule

Page 6: CSE 544: Lecture 5-6 Theory

6

Examples

Employee(x), ManagedBy(x,y), Manager(y)

• Find all employees having the same manager as “Smith”:

A(x) :- ManagedBy(“Smith”,y), ManagedBy(x,y)A(x) :- ManagedBy(“Smith”,y), ManagedBy(x,y)

Page 7: CSE 544: Lecture 5-6 Theory

7

Examples

Employee(x), ManagedBy(x,y), Manager(y)

• Find all employees having the same director as Smith:

A(x) :- ManagedBy(“Smith”,y), ManagedBy(y,z), ManagedBy(x,u), ManagedBy(u,z)A(x) :- ManagedBy(“Smith”,y), ManagedBy(y,z), ManagedBy(x,u), ManagedBy(u,z)

CQs are useful in practice

Page 8: CSE 544: Lecture 5-6 Theory

8

CQ and SQL

select distinct m2.namefrom ManagedBy m1, ManagedBy m2where m1.name=“Smith” AND m1.manager=m2.manager

select distinct m2.namefrom ManagedBy m1, ManagedBy m2where m1.name=“Smith” AND m1.manager=m2.manager

A(x) :- ManagedBy(“Smith”,y), ManagedBy(x,y)A(x) :- ManagedBy(“Smith”,y), ManagedBy(x,y)

CQ:

SQL: Notice“distinct”

Page 9: CSE 544: Lecture 5-6 Theory

9

CQ and SQL

• Are CQ queries precisely the SELECT-DISTINCT-FROM-WHERE queries ?

Page 10: CSE 544: Lecture 5-6 Theory

10

CQ and RA

Relational Algebra:• CQ correspond precisely to

C, A, (missing: , –)

$2.name

name=“Smith”

ManagedBy ManagedBy

$1.manager=$2.manager

A(x) :- ManagedBy(“Smith”,y), ManagedBy(x,y)A(x) :- ManagedBy(“Smith”,y), ManagedBy(x,y)

Page 11: CSE 544: Lecture 5-6 Theory

11

Extensions of CQ

CQ

A(y) :- ManagedBy(x,y), ManagedBy(z,y), xzA(y) :- ManagedBy(x,y), ManagedBy(z,y), xz

Find managers that manage at least 2 employees

Page 12: CSE 544: Lecture 5-6 Theory

12

Extensions of CQ

CQ<

A(y) :- ManagedBy(x,y), Salary(x,u), Salary(y,v), u>v)A(y) :- ManagedBy(x,y), Salary(x,u), Salary(y,v), u>v)

Find employees earning more than their manager:

Page 13: CSE 544: Lecture 5-6 Theory

13

Extensions of CQ

CQ

A(y) :- Office(“Alice”,u), Office(y,u), ManagedBy(“Alice”,x), ManagedBy(x,y)A(y) :- Office(“Alice”,u), Office(y,u), ManagedBy(“Alice”,x), ManagedBy(x,y)

Find people sharing the same office with Alice, butnot the same manager:

Page 14: CSE 544: Lecture 5-6 Theory

14

Extensions of CQ

UCQ

A(name) :- Employee(name, dept, age, salary), age > 50A(name) :- RetiredEmployee(name, address)A(name) :- Employee(name, dept, age, salary), age > 50A(name) :- RetiredEmployee(name, address)

Union of conjuctive queries

Datalog notation is very convenient at expressing unions(no need for )

Datalog:

Page 15: CSE 544: Lecture 5-6 Theory

15

Extensions of CQ

• If we extend too much, we capture FO

• Theoreticians need to be careful: small extensions may make a huge difference on certain theoretical properties of CQ

Page 16: CSE 544: Lecture 5-6 Theory

16

Query Equivalence and Containment

•Justified by optimization needs

•Intensively studied since 1977

Page 17: CSE 544: Lecture 5-6 Theory

17

Query Equivalence

• Queries q1 and q2 are equivalent if for every database D, q1(D) = q2(D).

• Notation: q1 q2

Page 18: CSE 544: Lecture 5-6 Theory

18

Query Equivalence

SELECT DISTINCT x.name, x.managerFROM Employee x, Employee yWHERE x.dept = ‘Sales’ and x.office = y.office and x.floor = 5 and y.dept = ‘Sales’

SELECT DISTINCT x.name, x.managerFROM Employee x, Employee yWHERE x.dept = ‘Sales’ and x.office = y.office and x.floor = 5 and y.dept = ‘Sales’

Hmmmm…. Is there a simple way to write that ?

Page 19: CSE 544: Lecture 5-6 Theory

19

Query Containment

• Query q1 is contained in q2 if for every database D, q1(D) q2(D).

• q1 and q2 are equivalent if for every database D, q1(D) = q2(D)

• Notation: q1 q2, q1 q2

• Obviously: q1 q2 and q2 q1 iff q1 q2

• Conversely: q1 q2 q1 iff q1 q2

We will study the containment problem only.

Page 20: CSE 544: Lecture 5-6 Theory

20

Examples of Query Containments

q1(x) :- R(x,u), R(u,v), R(v,w)q2(x) :- R(x,u), R(u,v)

q1(x) :- R(x,u), R(u,v), R(v,w)q2(x) :- R(x,u), R(u,v)

Is q1 q2 ?

Page 21: CSE 544: Lecture 5-6 Theory

21

Examples of Query Containments

q1(x) :- R(x,u), R(u,v), R(v,x)q2(x) :- R(x,u), R(u,x)

q1(x) :- R(x,u), R(u,v), R(v,x)q2(x) :- R(x,u), R(u,x)

Is q1 q2 ?

Page 22: CSE 544: Lecture 5-6 Theory

22

Examples of Query Containments

q1(x) :- R(x,u), R(u,u)q2(x) :- R(x,u), R(u,v), R(v,w)

q1(x) :- R(x,u), R(u,u)q2(x) :- R(x,u), R(u,v), R(v,w)

Is q1 q2 ?

Page 23: CSE 544: Lecture 5-6 Theory

23

Examples of Query Containments

q1(x) :- R(x,u), R(u,”Smith”)q2(x) :- R(x,u), R(u,v)

q1(x) :- R(x,u), R(u,”Smith”)q2(x) :- R(x,u), R(u,v)

Is q1 q2 ?

Page 24: CSE 544: Lecture 5-6 Theory

24

Query Containment

• Theorem Query containment for FO is undecidable

• Theorem Query containment for CQ is decidable and NP-complete.

Page 25: CSE 544: Lecture 5-6 Theory

25

Query Containment Algorithm

How to check q1 q2

• Canonical database for q1 is: Dq1 = (D, R1

D, …, RkD)

– D = all variables and constants in q1

– R1D, …, Rk

D = the body of q1

• Canonical tuple for q1 is: tq1 (the head of q1)

Page 26: CSE 544: Lecture 5-6 Theory

26

Examples of Canonical Databases

• Canonical database: Dq1 = (D, RD)– D={x,y,u,v}– RD =

• Canonical tuple: tq1 = (x,y)

x u

v u

v y

q1(x,y) :- R(x,u),R(v,u),R(v,y)q1(x,y) :- R(x,u),R(v,u),R(v,y)

Page 27: CSE 544: Lecture 5-6 Theory

27

Examples of Canonical Databases

• Dq1 = (D, R)– D={x,u,”Smith”,”Fred”}– R =

• tq1 = (x)

x u

u“Smith”

u“Fred

u u

q1(x) :- R(x,u), R(u,”Smith”), R(u,”Fred”), R(u, u)q1(x) :- R(x,u), R(u,”Smith”), R(u,”Fred”), R(u, u)

Page 28: CSE 544: Lecture 5-6 Theory

28

Checking ContainmentTheorem: q1 q2 iff tq1 q2(Dq1).

Example:q1(x,y) :- R(x,u),R(v,u),R(v,y)q2(x,y) :- R(x,u),R(v,u),R(v,w),R(t,w),R(t,y)

• D={x,y,u,v}• R = tq1 = (x,y)

• Yes, q1 q2

x u

v u

v y

Page 29: CSE 544: Lecture 5-6 Theory

29

Query Homomorphisms

• A homomorphism f : q2 q1 is a function f: var(q2) var(q1) const(q1)such that:– f(body(q2)) body(q1)

– f(tq1) = tq2

The Homomorphism Theorem q1 q2 iff there exists a homomorphism f : q2 q1

Page 30: CSE 544: Lecture 5-6 Theory

30

Example of Query Homeomorphism

var(q1) = {x, u, v, y}

var(q2) = {x, u, v, w, t, y}

q1(x,y) :- R(x,u),R(v,u),R(v,y)

q2(x,y) :- R(x,u),R(v,u),R(v,w),R(t,w),R(t,y)

Therefore q1 q2

Page 31: CSE 544: Lecture 5-6 Theory

31

Example of Query Homeomorphism

var(q1) const(q1) = {x,u, “Smith”}

var(q2) = {x,u,v,w}

q1(x) :- R(x,u), R(u,”Smith”), R(u,”Fred”), R(u, u)

q2(x) :- R(x,u), R(u,v), R(u,”Smith”), R(w,u)

Therefore q1 q2

Page 32: CSE 544: Lecture 5-6 Theory

32

The Homeomorphism Theorem

• Theorem Conjunctive query containment is:

(1) decidable (why ?)(2) in NP (why ?)(3) NP-hard

• Short: it is NP-complete

Page 33: CSE 544: Lecture 5-6 Theory

33

Query Containment for UCQ

q1 q2 q3 . . . . q1’ q2’ q3’

. . . .

Notice: q1 q2 q3 . . . . q iff q1 q and q2 q and q3 q and ….

Theorem q q1’ q2’ q3’ . . . . Iff there exists some k such that q qk’

It follows that containment for UCQ is decidable, NP-complete.

Page 34: CSE 544: Lecture 5-6 Theory

34

Query Containment for CQ<

q1() :- R(x,y), R(y,x)q2() :- R(x,y), x <= y

q1() :- R(x,y), R(y,x)q2() :- R(x,y), x <= y

q1 q2 although there is no homomorphism !

To check containment do this:-Consider all possible orderings of variables in q1-For each of them check containment of q1 in q2-If all hold, then q1 q2

Still decidable, but harder than NP: now in p2

Page 35: CSE 544: Lecture 5-6 Theory

35

Query Minimization

Definition A conjunctive query q is minimal if for every other conjunctive query q’ s.t. q q’, q’ has at least as many predicates (‘subgoals’) as q

Are these queries minimal ?q(x) :- R(x,y), R(y,z), R(x,x)q(x) :- R(x,y), R(y,z), R(x,x)

q(x) :- R(x,y), R(y,z), R(x,’Alice’)q(x) :- R(x,y), R(y,z), R(x,’Alice’)

Page 36: CSE 544: Lecture 5-6 Theory

36

Query Minimization

• Query minimization algorithm

Choose a subgoal g of qRemove g: let q’ be the new queryWe already know q q’ (why ?)If q’ q then permanently remove g

• Notice: the order in which we inspect subgoals doesn’t matter

Page 37: CSE 544: Lecture 5-6 Theory

37

Query Minimization In Practice

• No database system today performs minimization !!!

• Reason:– It’s hard (NP-complete)– Users don’t write non-minimal queries

• However, non-minimal queries arise when using views intensively

Page 38: CSE 544: Lecture 5-6 Theory

38

Query Minimization for Views

CREATE VIEW HappyBoaters

SELECT DISTINCT E1.name, E1.manager FROM Employee E1, Employee E2 WHERE E1.manager = E2.name and E1.boater=‘YES’ and E2.boater=‘YES’

CREATE VIEW HappyBoaters

SELECT DISTINCT E1.name, E1.manager FROM Employee E1, Employee E2 WHERE E1.manager = E2.name and E1.boater=‘YES’ and E2.boater=‘YES’

This query is minimal

Page 39: CSE 544: Lecture 5-6 Theory

39

Query Minimization for Views

SELECT DISTINCT H1.nameFROM HappyBoaters H1, HappyBoaters H2WHERE H1.manager = H2.name

SELECT DISTINCT H1.nameFROM HappyBoaters H1, HappyBoaters H2WHERE H1.manager = H2.name

Now compute the Very-Happy-Boaters

What happens in SQL when we run a query ona view ?

This query is also minimal

Page 40: CSE 544: Lecture 5-6 Theory

40

Query Minimization for Views

SELECT DISTINCT E1.nameFROM Employee E1, Employee E2, Employee E3, Empolyee E4WHERE E1.manager = E2.name and E1.boater = ‘YES’ and E2.boater = ‘YES’ and E3.manager = E4.name and E3.boater = ‘YES’ and E4.boater = ‘YES’ and E1.manager = E3.name

SELECT DISTINCT E1.nameFROM Employee E1, Employee E2, Employee E3, Empolyee E4WHERE E1.manager = E2.name and E1.boater = ‘YES’ and E2.boater = ‘YES’ and E3.manager = E4.name and E3.boater = ‘YES’ and E4.boater = ‘YES’ and E1.manager = E3.name

View Expansion

This query is no longer minimal !

E1E3 E4

E2 E2 is redundant

Page 41: CSE 544: Lecture 5-6 Theory

41

Monotone Queries

Definition A query q is monotone if:For every two databases D, D’if D D’ then q(D) q(D’)

Which queries below are monotone ?

x.R(x,x) x.R(x,x)

x.y.z.u.(R(x,y) R(y,z) R(z,u)) x.y.z.u.(R(x,y) R(y,z) R(z,u))

x.y.R(x,y) x.y.R(x,y)

Page 42: CSE 544: Lecture 5-6 Theory

42

Monotone Queries

• Theorem. Every conjunctive query is monotone

• Stronger: every UCQ query is monotone

Page 43: CSE 544: Lecture 5-6 Theory

43

How To Impress Your Students Or Your Boss

• Find all drinkers that like some beer that is not served by the bar “Black Cat”

• Can you write as a simple SELECT-FROM-WHERE (I.e. without a subquery) ?

SELECT L.drinkerFROM Likes LWHERE L.beer not in (SELECT S.beer FROM Serves S WHERE S.bar = ‘Black Cat’)

SELECT L.drinkerFROM Likes LWHERE L.beer not in (SELECT S.beer FROM Serves S WHERE S.bar = ‘Black Cat’)

Page 44: CSE 544: Lecture 5-6 Theory

44

Expressive Power of FO

• The following queries cannot be expressed in FO:

• Transitive closure: x.y. there exists x1, ..., xn s.t.R(x,x1) R(x1,x2) ... R(xn-1,xn) R(xn,y)

• Parity: the number of edges in R is even

Page 45: CSE 544: Lecture 5-6 Theory

45

Datalog

• Adds recursion, so we can compute transitive closure

• A datalog program (query) consists of several datalog rules:

P1(t1) :- body1

P2(t2) :- body2

.. . .Pn(tn) :- bodyn

Page 46: CSE 544: Lecture 5-6 Theory

46

Datalog

Terminology:• EDB = extensional database predicates– The database predicates

• IDB = intentional database predicates– The new predicates constructed by the program

Page 47: CSE 544: Lecture 5-6 Theory

47

Datalog

Employee(x), ManagedBy(x,y), Manager(y)

HMngr(x) :- Manager(x), ManagedBy(y,x), ManagedBy(z,y)Answer(x) :- HMngr(x), Employee(x)

HMngr(x) :- Manager(x), ManagedBy(y,x), ManagedBy(z,y)Answer(x) :- HMngr(x), Employee(x)

All higher level managers that are employees: EDBs

IDBs

Page 48: CSE 544: Lecture 5-6 Theory

48

Datalog

Employee(x), ManagedBy(x,y), Manager(y)

Person(x) :- Manager(x) Person(x) :- Employee(x)

Person(x) :- Manager(x) Person(x) :- Employee(x)

All persons:

Manger Employee

Page 49: CSE 544: Lecture 5-6 Theory

49

Unfolding non-recursive rules

Graph: R(x,y)

P(x,y) :- R(x,u), R(u,v), R(v,y)A(x,y) :- P(x,u), P(u,y)

P(x,y) :- R(x,u), R(u,v), R(v,y)A(x,y) :- P(x,u), P(u,y)

Can “unfold” it into:

A(x,y) :- R(x,u), R(u,v), R(v,w), R(w,m), R(m,n), R(n,y)A(x,y) :- R(x,u), R(u,v), R(v,w), R(w,m), R(m,n), R(n,y)

Page 50: CSE 544: Lecture 5-6 Theory

50

Unfolding non-recursive rules

Graph: R(x,y)P(x,y) :- R(x,y) P(x,y) :- R(x,u), R(u,y)A(x,y) :- P(x,y)

P(x,y) :- R(x,y) P(x,y) :- R(x,u), R(u,y)A(x,y) :- P(x,y)

Now the unfolding has a union:

A(x,y) :- R(x,y) u(R(x,u) R(u,y))A(x,y) :- R(x,y) u(R(x,u) R(u,y))

Page 51: CSE 544: Lecture 5-6 Theory

51

Recursion in Datalog

Graph: R(x,y)

P(x,y) :- R(x,y)P(x,y) :- P(x,u), R(u,y)

P(x,y) :- R(x,y)P(x,y) :- P(x,u), R(u,y)

Transitive closure:

P(x,y) :- R(x,y)P(x,y) :- P(x,u), P(u,y)

P(x,y) :- R(x,y)P(x,y) :- P(x,u), P(u,y)

Transitive closure:

Page 52: CSE 544: Lecture 5-6 Theory

52

Recursion in Datalog

Boolean trees:Leaf0(x), Leaf1(x),AND(x, y1, y2), OR(x, y1, y2),Root(x)

• Write a program that computes:Answer() :- true if the root node is 1

Page 53: CSE 544: Lecture 5-6 Theory

53

Recursion in Datalog

One(x) :- Leaf1(x)One(x) :- AND(x, y1, y2), One(y1), One(y2)One(x) :- OR(x, y1, y2), One(y1)One(x) :- OR(x, y1, y2), One(y2)Answer() :- Root(x), One(x)

One(x) :- Leaf1(x)One(x) :- AND(x, y1, y2), One(y1), One(y2)One(x) :- OR(x, y1, y2), One(y1)One(x) :- OR(x, y1, y2), One(y2)Answer() :- Root(x), One(x)

Page 54: CSE 544: Lecture 5-6 Theory

54

Exercise

Boolean trees:Leaf0(x), Leaf1(x),AND(x, y1, y2), OR(x, y1, y2), Not(x,y),Root(x)

• Write a datalog program that computes the set of nodes that are “true” or “one”.

• Hint: compute both One(x) and Zero(x)here you need to use Leaf0

Page 55: CSE 544: Lecture 5-6 Theory

55

Variants of Datalog

without recursion

with recursion

without

Non-recursive Datalog

= UCQ (why ?)Datalog

with Non-recursive

Datalog

= FODatalog

Page 56: CSE 544: Lecture 5-6 Theory

56

Non-recursive Datalog

• Union of Conjunctive Queries = UCQ– Containment is decidable, and NP-complete

• Non-recursive Datalog– Is equivalent to UCQ– Hence containment is decidable here too

– Is it still NP-complete ?

Page 57: CSE 544: Lecture 5-6 Theory

57

Non-recursive Datalog

• A non-recursive datalog:

• Its unfolding as a CQ:

• How big is this query ?

T1(x,y) :- R(x,u), R(u,y)T2(x,y) :- T1(x,u), T1(u,y) . . .Tn(x,y) :- Tn-1 (x,u), Tn-1 (u,y)Answer(x,y) :- Tn(x,y)

T1(x,y) :- R(x,u), R(u,y)T2(x,y) :- T1(x,u), T1(u,y) . . .Tn(x,y) :- Tn-1 (x,u), Tn-1 (u,y)Answer(x,y) :- Tn(x,y)

Anser(x,y) :- R(x,u1), R(u1, u2), R(u2, u3), . . . R(um, y)Anser(x,y) :- R(x,u1), R(u1, u2), R(u2, u3), . . . R(um, y)

Page 58: CSE 544: Lecture 5-6 Theory

58

Query Complexity

• Given a query in FO

• And given a model D = (D, R1D,

…, RkD)

• What is the complexity of computing the answer (D)

Page 59: CSE 544: Lecture 5-6 Theory

59

Query Complexity

Vardi’s classification:

Data Complexity:• Fix . Compute (D) as a function of |D|

Query Complexity:• Fix D. Compute (D) as a function of ||

Combined Complexity:• Compute (D) as a function of |D| and ||

Which is the most important in databases ?

Page 60: CSE 544: Lecture 5-6 Theory

60

Example

(x) u.(R(u,x) y.(v.S(y,v) R(x,y)))(x) u.(R(u,x) y.(v.S(y,v) R(x,y)))

3 8

7 5

0 8

09 7

6 9

7 6

89 8

98 7

4 0

43 4

5 58

8 6

9 79

6 67

4 7

6 8

R = S =

How do we proceed ?

Page 61: CSE 544: Lecture 5-6 Theory

61

General Evaluation Algorithm

for every subexpression i of , (i = 1, …, m)

compute the answer to i as a table Ti(x1, …, xn)

return Tm

for every subexpression i of , (i = 1, …, m)

compute the answer to i as a table Ti(x1, …, xn)

return Tm

Theorem. If has k variables then one can compute (D) in time O(||*|D|k)

Data Complexity = O(|D|k) = in PTIMEQuery Complexity = O(||*ck) = in EXPTIME

Page 62: CSE 544: Lecture 5-6 Theory

62

General Evaluation Algorithm

Example:

1(u,x) R(u,x) 2(y,v) S(y,v) 3(x,y) R(x,y)4(y) v.2(y,v) 5(x,y) 4(y) 3(x,y) 6(x) y. 5(x,y) 7(u,x) 1(u,x) 6(x) 8(x) u. 7(u,x) (x)

1(u,x) R(u,x) 2(y,v) S(y,v) 3(x,y) R(x,y)4(y) v.2(y,v) 5(x,y) 4(y) 3(x,y) 6(x) y. 5(x,y) 7(u,x) 1(u,x) 6(x) 8(x) u. 7(u,x) (x)

(x) u.(R(u,x) y.(v.S(y,v) R(x,y)))(x) u.(R(u,x) y.(v.S(y,v) R(x,y)))

Page 63: CSE 544: Lecture 5-6 Theory

63

Complexity

Theorem. If has k variables then one can compute (D) in time O(||*|D|k)

Remark. The number of variables matters !

Page 64: CSE 544: Lecture 5-6 Theory

64

Paying Attention to Variables

• Compute all chains of length m

• We used m+1 variables• Can you rewrite it with fewer variables ?

Chainm(x,y) :- R(x,u1), R(u1, u2), R(u2, u3), . . . R(um-1, y)Chainm(x,y) :- R(x,u1), R(u1, u2), R(u2, u3), . . . R(um-1, y)

Page 65: CSE 544: Lecture 5-6 Theory

65

Counting Variables

• FOk = FO restricted to variables x1,…, xk

• Write Chainm in FO3:Chainm(x,y) :- u.R(x,u)(x.R(u, x)(u.R(x,u)…(u. R(u, y)…))Chainm(x,y) :- u.R(x,u)(x.R(u, x)(u.R(x,u)…(u. R(u, y)…))

Page 66: CSE 544: Lecture 5-6 Theory

66

Query Complexity

• Note: it suffices to investigate boolean queries only– If non-boolean, do this:

for a1 in D, …, ak in D

if (a1, …, ak) in (D) /* this is a boolean query */

then output (a1, …, ak)

for a1 in D, …, ak in D

if (a1, …, ak) in (D) /* this is a boolean query */

then output (a1, …, ak)

Page 67: CSE 544: Lecture 5-6 Theory

67

ComputationalComplexity Classes

Recall computational complexity classes:• AC0

• LOGSPACE• NLOGSPACE• PTIME• NP• PSPACE• EXPTIME• EXPSPACE• (Kalmar) Elementary Functions• Turing Computable functions

Page 68: CSE 544: Lecture 5-6 Theory

68

Data Complexity of Query Languages

AC0

PTIME

PSPACE

FO = non-rec datalog

FO(LFP) = datalog

FO(PFP) = datalog,*

Important: the more complex a QL, the harder it is to optimize

Paper: On the Unusual Effectiveness of Logic in Computer Science

Page 69: CSE 544: Lecture 5-6 Theory

69

Views

Employee(x), ManagedBy(x,y), Manager(y)L(x,y) :- ManagedBy(x,u), ManagedBy(u,y)L(x,y) :- ManagedBy(x,u), ManagedBy(u,y)

Q(x,y) :- ManagedBy(x,u), ManagedBy(u,v), ManagedBy(v,w), ManagedBy(w,y), Employee(y)

Q(x,y) :- ManagedBy(x,u), ManagedBy(u,v), ManagedBy(v,w), ManagedBy(w,y), Employee(y)

E(x,y) :- ManagedBy(x,y), Employee(y)E(x,y) :- ManagedBy(x,y), Employee(y)

How can we answer Q if we only have L and E ?

Views

Query

Page 70: CSE 544: Lecture 5-6 Theory

70

Views

• Query rewriting using views (when possible):

• Query answering:– Sometimes we cannot express it in CQ or FO, but we can still answer it

Q(x,y) :- L(x,u), L(u,y), E(v,y)Q(x,y) :- L(x,u), L(u,y), E(v,y)

Page 71: CSE 544: Lecture 5-6 Theory

71

Views

Applications:• Using advanced indexes• Using replicated data• Data integration [Ullman’99]

Page 72: CSE 544: Lecture 5-6 Theory

72

Finding a Rewriting

Theorem Given views V1, …, Vn and query Q, the problem whether Q has a complete rewriting in terms of V1, …, Vn is NP complete

Page 73: CSE 544: Lecture 5-6 Theory

73

Certain Answers

• Sometimes we can’t answer, but we can get close

V1(x,y) :- E(x,u), E(u,v), E(v,y)V2(x,y) :- E(x,u), E(u,y), Black(y)

V1(x,y) :- E(x,u), E(u,v), E(v,y)V2(x,y) :- E(x,u), E(u,y), Black(y)

Q(x) :- E(x,u), E(u,v), E(v,w), E(w,s)Q(x) :- E(x,u), E(u,v), E(v,w), E(w,s)

Can’t really answer Q, but we can find approximations….

Page 74: CSE 544: Lecture 5-6 Theory

74

Certain Answers

V1(x,y) :- E(x,u), E(u,v), E(v,y)V2(x,y) :- E(x,u), E(u,y), Black(y)

V1(x,y) :- E(x,u), E(u,v), E(v,y)V2(x,y) :- E(x,u), E(u,y), Black(y)

Q(x) :- E(x,u), E(u,v), E(v,w), E(w,s)Q(x) :- E(x,u), E(u,v), E(v,w), E(w,s)

Q(x) :- V2(x,u), V2(u,v)Q(x) :- V1(x,u), V2(u,v)Q(x) :- V1(x,u), V1(u,v)

Q(x) :- V2(x,u), V2(u,v)Q(x) :- V1(x,u), V2(u,v)Q(x) :- V1(x,u), V1(u,v)

All these return ‘certain’answers…

Page 75: CSE 544: Lecture 5-6 Theory

75

Certain Answers

Definition. Given V1, …, Vn, their answers A1, …, An and a query Q, a tuple t is a certain tuple for Q iff for every database instance D:if A1=V1(D) and … and An =Vn(D) then t Q(D)if A1=V1(D) and … and An =Vn(D) then t Q(D)

if A1V1(D) and … and AnVn(D) then t Q(D)if A1V1(D) and … and AnVn(D) then t Q(D)

CWD (Closed World Assumption)

OWD (Open World Assumption)

Page 76: CSE 544: Lecture 5-6 Theory

76

Computing Certain Answers Under OWD

V1(x,y) :- E(x,u), E(u,v), E(v,y)V2(x,y) :- E(x,u), E(u,y), Black(y)

V1(x,y) :- E(x,u), E(u,v), E(v,y)V2(x,y) :- E(x,u), E(u,y), Black(y)

Q(x) :- E(x,u), E(u,v), E(v,w), E(w,s)Q(x) :- E(x,u), E(u,v), E(v,w), E(w,s)

E(x,f(x,y)) :- V1(x,y)E(f(x,y),g(x,y)) :- V1(x,y)E(g(x,y),y) :- V1(x,y)E(x,h(x,y)) :- V2(x,y)E(h(x,y),y) :- V2(x,y)Black(y) :- V2(x,y)

E(x,f(x,y)) :- V1(x,y)E(f(x,y),g(x,y)) :- V1(x,y)E(g(x,y),y) :- V1(x,y)E(x,h(x,y)) :- V2(x,y)E(h(x,y),y) :- V2(x,y)Black(y) :- V2(x,y)

Inverse rules

Combined datalog program

Page 77: CSE 544: Lecture 5-6 Theory

77

Computing Certain Answers Under OWD

Next, we have two options

• Run the combined “datalog” program– It is actually a Prolog program– Notice: data complexity is PTIME

• Transform the datalog program first, so Q returns only values that are not Skolem Terms

Page 78: CSE 544: Lecture 5-6 Theory

78

Computing Answer Under CWD Is Different

V1(x) :- R(x,u)V2(y) :- R(v,y)Q(x,y) :- R(x,y)

V1(x) :- R(x,u)V2(y) :- R(v,y)Q(x,y) :- R(x,y)

A1 = {a}A2 = {b}

A1 = {a}A2 = {b}

Certain answers for Q under OWD: none

Certain answers for Q under CWD: (a,b)

Why ?

Certain answers for Q under OWD: none

Certain answers for Q under CWD: (a,b)

Why ?