answering top-k queries using views
DESCRIPTION
Answering Top-k Queries Using Views. By: Gautam Das (Univ. of Texas), Dimitrios Gunopulos (Univ. of California Riverside), Nick Koudas (Univ. of Toronto), Dimitris Tsirogiannis (Univ. of Toronto) Presented By: - PowerPoint PPT PresentationTRANSCRIPT
Answering Top-k Queries Using Views
By:Gautam Das (Univ. of Texas), Dimitrios Gunopulos (Univ. of California Riverside), Nick Koudas (Univ. of Toronto), Dimitris Tsirogiannis (Univ. of Toronto)
Presented By: Kushal Shah Lipsa Patel
Views
• Definition: Views
• Declaring Views
• Advantages of using Views
Views
• A view may be thought of as a table, that is derived from one or more underlying base table.
• Two kinds:1. Virtual: Not stored in the database; just a
query for constructing the relation.2. Materialized: Actually constructed and
stored.
Declaring Views
• Materialized: CREATE [MATERIALIZED] VIEW <name> AS <query>;
• Virtual: Default
Advantages of using Views
• If we have several tables in a DB and we want to view only specific columns from specific tables we can go for views.
• Suffice the needs of security: Sometimes allowing specific users to see only specific columns based on the permission that we can configure on the views.
Answering Top-k Queries Using Views
By:Gautam Das (Univ. of Texas), Dimitrios Gunopulos (Univ. of California Riverside), Nick Koudas (Univ. of Toronto), Dimitris Tsirogiannis (Univ. of Toronto)
Presented By: Kushal Shah Lipsa Patel
Top-k Query
• Top-k Query Processing – Definition
• Top-k Example
• Algorithms for Top-k Query Processing
Top-k Query Processing
Top-k query processing =
Finding k objects that have the highest overall Score
Top-k Example
R
• Users preferences regarding the ordering of the tuples of a relation can be expressed as a scoring functions on the attributes of a relation, eg
• fq = 3x1 + 2x2 + 5x3• The top-k problem is to find the k tuples with the highest
score according to a given scoring function.
tid X1 X2 X3
1 82 1 59
2 53 19 83
3 29 99 15
4 80 45 8
5 28 32 39
fQ
tid Score
2 612
1 543
4 370
3 360
5 343
Algorithms for Top-k Query Processing
• How? Which algorithms? – Related Work How we complement existing approaches?
• TA [Fagin]• PREFER [Hristidis] Stores the multiple copies of a relation and each
copy is ordered according to a different scoring function.
In order to answer a top-k query the algorithm utilizes a single copy with a scoring function which is closest to the scoring function of the query.
(a, 0.9)
(b, 0.8)
(c, 0.72)
(d, 0.6)
.
.
.
.
Sorted L1
(d, 0.9)
(a, 0.85)
(b, 0.7)
(c, 0.2)
.
.
.
.
N
a
b
c
d
.
.
.
.
ObjectID
0.9
0.8
0.72
0.6
.
.
.
.
Attribute 1
0.85
0.2
0.9
.
.
.
.
Attribute 2
0.7
M
Sorted L2
Example – Simple Database model
ID A1 A2 Min(A1,A2)
Step 1: - parallel sorted access to each list
(a, 0.9)
(b, 0.8)
(c, 0.72)
(d, 0.6)
.
.
.
.
L1 L2
(d, 0.9)
(a, 0.85)
(b, 0.7)
(c, 0.2)
.
.
.
.
a
d
0.9
0.9
0.85 0.85
0.6 0.6
For each object seen: - get all grades by random access - determine Min(A1,A2) - amongst 2 highest seen ? keep in buffer
Example – Threshold Algorithm
ID A1 A2 Min(A1,A2)a: 0.9
b: 0.8
c: 0.72
d: 0.6
.
.
.
.
L1 L2
d: 0.9
a: 0.85
b: 0.7
c: 0.2
.
.
.
.
Step 2: - Determine threshold value based on objects currently seen under sorted access. T = min(L1, L2)
a
d
0.9
0.9
0.85 0.85
0.6 0.6
T = min(0.9, 0.9) = 0.9
- 2 objects with overall grade ≥ threshold value ? stop else go to next entry position in sorted list and repeat step 1
Example – Threshold Algorithm
ID A1 A2 Min(A1,A2)
Step 1 (Again): - parallel sorted access to each list
(a, 0.9)
(b, 0.8)
(c, 0.72)
(d, 0.6)
.
.
.
.
L1 L2
(d, 0.9)
(a, 0.85)
(b, 0.7)
(c, 0.2)
.
.
.
.
a
d
0.9
0.9
0.85 0.85
0.6 0.6
For each object seen: - get all grades by random access - determine Min(A1,A2) - amongst 2 highest seen ? keep in buffer
b 0.8 0.7 0.7
Example – Threshold Algorithm
ID A1 A2 Min(A1,A2)a: 0.9
b: 0.8
c: 0.72
d: 0.6
.
.
.
.
L1 L2
d: 0.9
a: 0.85
b: 0.7
c: 0.2
.
.
.
.
Step 2 (Again): - Determine threshold value based on objects currently seen. T = min(L1, L2)
a
b
0.9
0.7
0.85 0.85
0.8 0.7
T = min(0.8, 0.85) = 0.8
- 2 objects with overall grade ≥ threshold value ? stop else go to next entry position in sorted list and repeat step 1
Example – Threshold Algorithm
c
ID A1 A2 Min(A1,A2)a: 0.9
b: 0.8
c: 0.72
d: 0.6
.
.
.
.
L1 L2
d: 0.9
a: 0.85
b: 0.7
c: 0.2
.
.
.
.
Situation at stopping condition
a
b
0.9
0.7
0.85 0.85
0.8 0.7
T = min(0.72, 0.7) = 0.7
Example – Threshold Algorithm
0.72 0.2 0.2
Related Work for Top-k Query Processing
• TA: Sequential as well as Random Access
• PREFER
Approach for Top-k Query Processing
• Top-k Query Answering using Views
• Views are Materialized (incurring space overhead)
• Advantages of using views: increased performance because views are small in size
• Space-Performance tradeoff
Example Views
R tid X1 X2 X3
1 82 1 59
2 53 19 83
3 29 99 15
4 80 45 8
5 28 32 39Three attribute relation R
V1 tid Score
3 553
4 385
5 216
2 201
1 169Top-5 query using function f1 = 2x1 + 5x2
V2 tid Score
2 351
1 237
5 177
3 159
4 88Top-5 query using function f2 = x2 + 2x3
• Top-k ranking queries in SQL-like syntax: SELECT TOP[k] FROM R ORDER BY Score(q) Score(q) - function that assigns numeric score to any tuple t
• Ranking Views: Views only aim to rank• A ranking view is the materialized result of a previously asked top-k query.
Can we answer new top-k queries efficiently using ranking views? Let’s see
Formal Definitions
• Ranking Queries
• Ranking Views
Ranking Queries
• Ranking Queries: Top-k ranking queries in SQL-like syntax: Select Top[k] from R where Range(q) Order By Score(q)
• A ranking query may be expressed as a triple Q = (Score(q), k, Range(q)), where
Score(q)= Function that assigns numeric score to any tuple t Range(q) = defines selection condition for the tuples of R• Semantics: Retrieve the k tuples with the top scores
satisfying the selection condition.
Ranking Views
Materialized Ranking View V: for a previously executed query Q1 = (ScoreQ
1, k1, RangeQ1),
the corresponding materialized ranking view is a set of k(tid, scoreQ(tid)) pairs,
ordered by decreasing values of scoreQ(tid).
Problems we are going to solve
• Top-k Query Answer using Views
• View Selection
Top-k Query Answer using Views
Given: Set U of views Query Q
Obtain an answer to Q combining all the information conveyed by the views in U
Solution: Algorithm named LPTA
Problems we are going to solve
• Top-k Query Answer using Views
• View Selection
View Selection
• Problem: Given a collection of views V={V1…Vr} – base views and a query Q, determine the most efficient subset U of V to execute Q on.
• Input to LPTA: subset U• Obtaining an answer to ranking query: Running TA on base
views.• Find the subset U that when utilized by LPTA 1. Provide answer to query 2. Provide answer faster than running TA on the base
views V
Outline
• LPTA Algorithm
• View Selection Problem
LPTA: Linear Programming Adaptation of the Threshold Algorithm
1. Scoring function of Query: Q - fQ = 3x1 + 10x2
2. Scoring function of Views: V1 – fv1 = 2x1 + 5x2
Subset of Views U V2 – fv2 = x1 + 2x2
LPTA for Top-k Query Answer using Views
Top-1 Query• View is a set of pairs of (tuple identifier, score). • The LPTA algorithm requires sorted access on each view in non-increasing order of that score.
LPTA Example
tid x1 x2 x3
1 82 1 592 53 19 833 29 1 24 80 22 905 28 8 876 12 55 827 16 99 428 18 42 679 42 1 23
10 23 21 88
R tid Score7 5276 2994 2708 2462 201
V1
Top-5 Query f1 = 2x1 + 5x2
tid Score6 2194 202
10 197
Top-3 Query f2 = x2 + 2x3
V2
Answer Top-2 Query using LPTA
LPTA Setting
• The algorithm initializes the top-k buffer to empty.
Top-2 Buffer tid Score7 5276 2994 2708 2462 201
tid Score6 2194 20210 197
V1 V2
7 16 99 42
For each tid read, random access on R to retrieve tuple and compute score acc to query function f3 = 3x1 + 10x2 + 5x3
6 12 55 82 (7,1248)(6,996)
Top-2 Buffer
Check for stopping Condition
Check for Stopping Condition• The unseen tuples in the view have satisfy the following inequalities:
The domain of each attribute of R [1,100]0<=X1,X2,X3<=100---------------------------(1)2x1 + 5x2 <= sd1-------------------------------(2)x2 + 2x3 <= sd2---------------------------------(3)sd1 = 527 and sd2 = 219
• Unseenmax = Solution to the linear program where we maximize the function f3 = 3x1 + 10x2 + 5x3 subject to these inequalities.
• The solution of the linear program gives the maximum score of any unseen tuple.
• Unseenmax = the maximum possible score (with respect to the ranking query’s scoring function) of any tuple not yet visited in the views.
• The algorithm terminates when the top-k buffer is full and Unseenmax <= topkmin
Calculating Unseenmax
• Unseenmax = Solution to the linear program where we maximize the function f3 = 3x1 + 10x2 + 5x3 subject to these inequalities.
• A linear programming problem may be defined as the problem of maximizing or minimizing a linear function subject to linear constraints. The constraints may be equalities or inequalities. Here is a simple example.
• Find numbers x1 and x2 that maximize the sum x1 + x2 subject to the constraints
x1 ≥ 0, x2 ≥ 0, and x1 + 2x2 ≤ 4
4x1 + 2x2 ≤ 12−x1 + x2 ≤ 1
Objective Function
Maximize the function
Convex region
This system of inequalities defines a convex region.
Occasionally, the maximum occurs along an entire edge or face of the constraint set, but then the maximum occurs at a corner point as well.
LPTA - Example
tid11
s11
tid21
tid31
tid41
tid51
s21
s31
s41
s51
tid12
s12
tid22
tid32
tid42
tid52
s22
s32
s42
s52
V1 V2
tid11
tid12
Top-1 queryV1
V2
Qstoppingcondition
X1
X2
R(X1, X2)
O (0,0)
P (1,0)
R (1,1)
T (0,1)
Normalized Domain[0,1]
Views and top-k query represented by vectors denoting the direction of increasing score
Sweeping line perpendicular to V1 from infinity to origin
Score of a tuple with respect to the query: project that tuple to the vector of the query
Score of a tuple with respect to a view: project that tuple to the the vector of the view
Max posssible score of any tuple not yet visited in the views with respect to the scoring func of query UNSEENMAX
LPTA - Example (cont’)
tid11
s11
tid21
tid31
tid41
tid51
s21
s31
s41
s51
tid12
s12
tid22
tid32
tid42
tid52
s22
s32
s42
s52
V1 V2
tid11
tid12
tid21
tid22
Top-1 V1
V2
Qstoppingcondition
X1
X2
R(X1, X2)
O (0,0)
P (1,0)
R (1,1)
T (0,1)
The algorithm will stop early if the scoring function of the views is “similar” to the scoring function of the query.
LPTA Algorithm – Pseudo Code
• There is Sequential as well as Random Access.• Sequential access on views• Random Access on base table to find the tuple
Comparison of LPTA with TA
• LPTA becomes TA when the set of views U = set of base views
• Execution cost: Both have Sequential as well as Random Access
These I/O Operations play a significant role – overshadow the costs of CPU operations such as updated top-k buffer, testing for stopping condition & so on.
Determining Factor for performance LPTA versus TA
• Highly correlated: every sequential access incurs a random access.
• As a result the determining factor for the performance is (distance from the beginning of the view each algorithm has to traverse (read sequentially) before coming into a halt with the correct answer) X (the number of views participating in the process).
• d=number of lock-step r = no of views• Running Cost: O(dr)
Outline
• LPTA Algorithm
• View Selection Problem
View Selection Problem
• Given a collection of views V = {V1,…,Vr} and a Query Q, determine the most efficient subset U C V to execute Q on.
• Conceptual discussion of View SelectionTwo attribute relation (in two dimension)Multi attribute relation (for any dimension)
• Domain of each attribute is normalized to [0,1]• M-attribute relation is refer as m-dimension
View Selection – Two Dimension(same side)
Min top-k tupleQ
V1
V2
O (0,0)
T (0,1)
P (1,0)
R (1,1)
X
Y
Square OPRT
Two views V1 and V2 and Query Q are represented by vectors. Both the view vectors are to the same side (clockwise) of the query vector
A
B
B1B2
M
AB 1 Q passes through M & intersect unit squareABR – Top-k tuplesABPOT – Remaining tuples
Sorted access to V1 – sweeping line 1 to V1 from infinity to origin
Stopping condition for V1: sweepline crosses AB1 b’coz convex polygon AB1POT – unseen tuples and score(unseen) <= Score(M)
Number of sorted access V1 = NumTuples(AB1R) V2 = NumTuples(AB2R)
View Selection – Two Dimension(same side) Conclusion
• V2 is slower compared to V1• If several views in two dimension are available & all their vectors are to one side of query vector, then it is optimal for LPTA to use the vector that is
closet to the query vector.
Estimating the Number of Tuples
• Estimating and Comparing the Number of Tuples by simply comparing the areas of respective triangles.
• Such approach: Need to have an uniform distribution within the triangles, which is often quite unrealistic.
• In our approach for view selection, utilize the conceptual conclusions + borrow
knowledge of actual data distribution.
View Selection – Two Dimension(either side of query)
A
B
Min top-k tupleQV1
V2
O (0,0)
T (0,1)
P (1,0)
R (1,1)
X
YA1
B1
M
Can use only V1 or only V2 for execution
If uses only v1 to answer the query the stopping condition will be reached once the sweepline perpendicular to v1 crosses position A1B/ For V2 - AB1
View Selection – Two Dimension(either side of query)
A
B
Min top-k tupleQV1
V2
O (0,0)
T (0,1)
P (1,0)
R (1,1)
X
YA1
B1
M
Running LPTA on both V1 and V2, rather than just running on only one of V1 or V2? Two views are better than one
A11
B11
A21
B21
The intersection point of the sweep lines perpendicular to v1 and v2 is on the line AB
The stopping condition is reached when the sweeplines resp crosses A11B11 and A21B21 such that 1) intersection pt of A11B11 and A21B21 is on line AB
2) NumTuples(A11B11R) = NumTuples(A21B21PR) since algo sweeps each view in lock-step
LPTA on both Views versus One • For two views the position of each sweepline is before the
respective stopping positions if only one view has been used.• Total number of sorted accesses for two views: NumTuples (A11B11R) + NumTuples (A21B21R) = 2
NumTuples (A11B11R)• If Min (NumTuples (A1BR), NumTuples (AB1PR), 2 NumTuples
(A11B11R)) = NumTuples (A1BR) - Use V1• If Min (NumTuples (A1BR), NumTuples (AB1PR), 2 NumTuples
(A11B11R)) = NumTuples (AB1PR) - Use V2• Else use both V1, V2
Theorem for Two Dimensional Case
Theorem 1: Set of Views = {V1,…,Vr} Query = Q
Two Dimensional dataset Va = Closest to query in Anticlockwise
Vc = Closest to query in Clockwise
So they are on either side of the queryOptimal execution of LPTA requires the use of either
Va or Vc i.e., the use of subset from {Va , Vc }
View Selection – Higher Dimension
Extension of Theorem 1Theorem 2: Set of Views = {V1,…,Vr} Query = Q
m-dimensional datasetOptimal execution of LPTA requires the use of subset
of views U C V such that |U| <= m
Outline
• LPTA Algorithm
• View Selection Problem
Cost Estimation Framework
Cost Estimation Framework – Running LPTA
• Cost Estimation Framework: The cost of running LPTA when a specific set of views is used to answer a query.
• Cost = total number of sequential accesses in a view• Uses 2 views to answer a query
Cost = 6 sequential accesses
Min top-k tuple
Can we find that costwithout actually running LPTA?
A
B
QV1
V2
Cost Estimation Framework – without Running LPTA
• EstimateCost(Q, U): Returns an estimate of the cost of running LPTA on exactly this set of views: U
• Used within SelectViews(Q,V) to search the subset U that minimizes EstimateCost(Q,U)
• EstimateCost(Q,U) takes into account– Multi-attribute views– Non-uniform data distribution
Simulating LPTA on Histograms rather than on views U
• Equi-depth histograms: The number of tuples in each bucket is the same
• Base Table R : n tuples (10)Hi Equi-depth histogram
b buckets – 2buckets : represent the distribution of points along the Xi attribute
Each bucket will represent n/b data points 10/2 = 5 data points
Simulating LPTA on Histograms rather than on views U
• In our estimation procedure: HQ – represents the distribution of score of all tuples
of the database according to the scoring function Q
Cannot calculate the score of all tuples, so approximate HQ
Simulation of LPTA on Histograms• Simulate LPTA in a
bucket by bucket lock step to estimate the cost.
HQ HV1 HV2
topkmin
HQ: approximates the scoredistribution of the query Q b buckets histograms for
the score distribution of views
n/b tuples per bucket
Cost
We cannot afford to run LPTA on views U
Pre-estimate topkmin b’coz we do not have access to actual tuples or their tids. The value of topkmin is estimated from HQ by determining the bucket that contains the kth highest tuple. Since topkmin is very likely inside this bucket we use linear interpolation with in the bucket to estimate the topkmin
Cheap procedure because we have one iteration of the LPTA algorithm for every n/b tuples using the values from the bucket boundaries.
Approx the value of func
Calculating the Estimated cost• Number of buckets visited along each view’s = d(3) Number of views = r1 (2)
Number of tuples per bucket n/b (10)
• Compute the smallest number of tuples n1 need to be scanned from the last bucket before stopping
• Estimated number of sorted access ((d-1)n/b +n1) r1
((2)(10) + 2) 2 = 44 Therefore running time is O((d-1) + logn1) lock-step iteration
Outline
• LPTA Algorithm
• View Selection Problem
Cost Estimation FrameworkView Selection Algorithms
EstimateCost(Q, U) – Pseudo-code
• SelectViews(Q, V) : Select the subset of views U which minimizes the EstimateCost
• Exhaustive (E) Approach: Estimate the cost of all possible subsets of V and select the subset of views with the smallest cost.
• Feasible for database with few attributes• Greedy Approach: Keep expanding the set of views
to use until the estimated cost stops reducing.• SelectViews(Q,V) – Pseudo code
Requires the solution of a single linear program. Fix the score sUniform Data distribution & very cheapMaximize the scoring function of the query Max(fq) using the
inequalities that scoring function of each view <= s /fv <= s
(0,1)
(1,0)
(0,0)
Q Selected Views whose hyperplanes intersect at the point which maximize the scoring function
s
s
s
s
s
SelectViewsSpherical (SVS)
T
Select Views By Angle (SVA)
Select Views By Angle (SVA): Sort the views by increasing angle with respect to Query vector.
(0,1)
(1,0)
(0,0)
Q
Selected Views: view closer to query will result in minimum running time for the algoV1
V2V3V4
1
2
3
4
1 2 3 4
Outline
• LPTA Algorithm
• View Selection Problem
Cost Estimation FrameworkView Selection Algorithms
• Experimental Evaluation
Experimental Evaluation
• Two types of dataset: Real and synthetic (uniform and zipf data with varying skew distribution)
• The real dataset contains 30K tuples from a website specialized on automobiles.
• Experiments Conducted:– Performance comparison of LPTA, PREFER and
TA– Performance of LPTA using each of the view
selection algorithms– Scalability of the LPTA algorithm
Performance comparison of LPTA, PREFER and TA
Uniform dataset, 3dReal dataset, 2d
Conclusions
• Using views for top-k query answering• LPTA: linear programming adaptation of TA• View selection problem, cost estimation framework,
view selection algorithms• Experimental evaluation
References
• Answering Top-k Queries Using Views: Gautam Das, Dimitrios Gunopulos, Nick Koudas
• Optimal Aggregation Algorithms for Middleware : Ronald Fagin, Amnon Lotem & Moni Naor
• aitrc.kaist.ac.kr/~vldb06/slides/R13-1.ppt