one-pass wavelet decompositions of data streams
DESCRIPTION
One-Pass Wavelet Decompositions of Data Streams. TKDE May 2002 Anna C. Gilbert,Yannis Kotidis, S. Muthukrishanan, Martin J. Strauss Presented by James Chan CSCI 599 Multidimensional Databases Fall 2003. Outline of Talk. Introduction Background Proposed Algorithm Experiments End Notes. - PowerPoint PPT PresentationTRANSCRIPT
One-Pass Wavelet Decompositions of Data Streams
TKDE May 2002
Anna C. Gilbert,Yannis Kotidis, S. Muthukrishanan, Martin J. Strauss
Presented by James ChanCSCI 599 Multidimensional Databases
Fall 2003
Outline of Talk
• Introduction
• Background
• Proposed Algorithm
• Experiments
• End Notes
Streaming Applications
• Telephone Call Duration
• Call Detail Record (CDR)
• IP Traffic Flow
• Bank ATM Transactions
• Mission Critical Task:– Fraud– Security– Performance Monitoring
Data Stream Model
Data Stream Problem• One Pass – no backtracking• Unbounded Data – Algorithms require small memory usage• Continuous – Need to run real time
Stream ProcessingEngine
(Approximate) Answer
Synopsis in Memory
Data Streams
Data Stream Strategies
• Many stream algorithms produce approximate answers and have:– Deterministic Bounds: answers are within ±– Probabilistic Bounds: answers have high
success probability (1-) within ±
Data Stream Strategies
• Windows: New elements expire after time t
• Samples: Approximate entire domain with a sample
• Histograms: Partitioning element domain values into buckets (Equi-depth, V-Opt)
• Wavelets: Haar, Construction and maintenance (difficult for large domain)
• Sketch Techniques: estimate of L2 norm of a signal
Proposed Stream Model
Background: Cash Register vs. Aggregate
• Cash Register: incoming stream represents domain (increment or decrement range of that domain)
• Aggregate: incoming stream represents range, (update range of that domain)
Note: Examples in this paper assume– each cash register element as +1 unit– no duplicate elements in aggregate models
Background: Cash Register vs. Aggregate
Cash Register
(domain)
Aggregate
(range)
Ordered Easiest
Eg. Time Series
Unordered General Challenging
Eg. Network volume
Contiguous Same as aggregate unordered n/a
Background: Wavelet Basics
• Wavelet transforms capture trends in a signal
• Typical transform involves log n passes
• Each pass creates two sets of n/2 averages and differences.
• Process repeated on averages
• Output: Wavelet Basis vectors – one average and n-1 coefficients
Background: Haar Wavelet Notation
• High pass filter
• Low pass filter
• Input: signal a
• Basis Coefficients
• Coefficients
• Scaling Factor
• Psi Vectors (un-normalized)
}2/1,2/1{
}2/1,2/1{
],...,[ 21 naaa
kjjkj asd ,, ,
}{}{],...,[ ,0,0110 kjn dcwww
jj Ns 2/
kj ,
]0,0,0,0,1,1,1,1[
]0,0,0,0,1,1,0,0[
]0,0,0,0,0,0,1,1[
0,2
2,3
0,3
kj
kj
kj
Background: Haar Wavelet Example
Background: Small B Representation
• Most signals in nature have small B representation
• Only keep largest B wavelet coefficients to estimate energy of signal
• Additional coefficients do not help reduce squared sum error
Energy:
SSE:
2
2R
2
2Ra
Background: Storage
• Highest B wavelet coefficients
• Log N Straddling coefficients, one per level of the wavelet tree
2 2 0 2 3 5 4 4
-1.25
2.75
0.5 0
0 -1 0 -1
+
-+
+
+ + +
+
+
- -
- - - -
Original Signal
Background: Bounding Theorems
Theorem 1• Given O(B+logN) storage (B is number of dimensions)• time to compute new data item is O(B+logN) in ordered
aggregate model
Theorem 2• Any algorithm that calculates the 2nd largest wavelet coefficient of
the signal in unordered CR / unordered agg uses at least N/polylog(N)
• This holds if:– You only care about existence, not the coefficients value– Only calculating up to a factor of 2
Proposed Algorithm: Overview
• Avoid keeping anything domain size N in memory
• Estimate wavelet coefficients using sketches which are size log(N)
• Sketch is maintained in memory and is updated as data entries stream in
What’s a Sketch?
• Distortion Parameter (epsilon)• Failure Probability (delta)• Failure Threshold (eta)• Original Signal a• Random vector of {-1,+1}s r• Seed for r s• Atomic Sketch <a,r> dot product of a and r• Sketch O(log(N/ )/ ^2) atomic sketches
• We use the same j to index the atomic sketch, seed, and random vector, so there are j atomic sketches in a sketch
Updating a Sketch
• Cash Register– Add corresponding to the j atomic sketches
• Aggregate– Add corresponding to the j atomic sketches
Use generator that takes in seed which is log(N) to compute
jir
jiria )(
jis
),( isGr jji
jir
Reed Muller Generator
• Pseudo random generator meeting these requirements:– Variables are 4 wise
independent• Expected value of product of any
4 distinct r is 0
– Requires O(log N) space for seeding
– Performs computation in polylog(N) time
{0} {d} {c} …. {d,c,b,a}
X = median ( )
Estimation of Inner Product
…
O(log(1/))
O(log(1/^2))
= mean ( )
… … …
…
Boosting Accuracy and Confidence
• Improve accuracy to by averaging over more independent copies of for each average
• Improve Confidence by increasing number of averages to take median over
j
iijii rbra
O(log(1/^2)) copies of …
X = median of ( )
= means ( )…
…
O(log(1/)) copies of …
Using the sketches
j
iijii rbraba ~,
• We can approximate <a,> to maintain Bs• Note a point query is <a,ei> where e is a vector
with a 1 at index i and 0s everywhere else
jj rbra ,,Atomic Sketches in memory
Maintaining Top B Coefficients
• At most Log N +1 coefficient updates
• May need to approximate straddling coefficients to aggregate with already existing or near variables
• Compare updates with top B and update top B if necessary
ia
updated
unaffected
Algorithm Space and Time
Their algorithm uses polylog(N) space and per item time to maintain B terms (by approximation)
Experiments• Data: one week of AT&T call detail (unordered cash
register model)
• Modes– Batch: Query only between intervals– Online: Query anytime
• Direct Point: calc sketch of <ei,a> (ei is zero vector except with 1 at i)
• Direct Wavelets: estimate all supporting coefficients and use wavelet reconstruction to calculate point a(i)
• Top B: Reconstruction of point is done with Top B (maintained by sketch)
Top B – Day 0
Top B - 1 Week
(fixed-set) Value updates only. no replacement
Sketch Size on Accuracy
Heavy Hitters
• Points that contribute significantly to the energy of the signal
• Direct point estimates are very accurate for heavy hitters but gross estimates for non heavy hitters
• Adaptive Greedy pursuit: by removing the first heavy hitter from the signal, you improve the accuracy of calculating the next biggest heavy hitter
• However an error is introduced with each subtraction of a heavy hitter
Processing Heavy Hitters
Adaptive Greedy Pursuit
End Notes
• First Provable Guarantees for haar wavelet over data streams
• Can estimate Haar coefficients ci=<a,>• Top B is updated in:
• This paper is superseded by "Fast, Small-space algorithms for approximatehistogram maintenance" STOC 2002– Discusses how to select top B and find heavy hitters
))()log((log 3 BNNO