online piece-wise linear approximation of numerical streams with precision guarantees
DESCRIPTION
Online Piece-wise Linear Approximation of Numerical Streams with Precision Guarantees. Hazem Elmeleegy Purdue University Ahmed Elmagarmid (Purdue) Emmanuel Cecchet (UMass) Walid Aref (Purdue) Willy Zwaenepoel (EPFL). Outline. Introduction Swing & Slide Filters Experiments Conclusion. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Online Piece-wise Linear Approximation of Numerical Streams with Precision Guarantees](https://reader036.vdocuments.mx/reader036/viewer/2022062721/56813866550346895da01403/html5/thumbnails/1.jpg)
Online Piece-wise Linear Approximation of Numerical Streams with Precision Guarantees
Hazem ElmeleegyPurdue University
Ahmed Elmagarmid (Purdue) Emmanuel Cecchet (UMass) Walid Aref (Purdue)
Willy Zwaenepoel (EPFL)
![Page 2: Online Piece-wise Linear Approximation of Numerical Streams with Precision Guarantees](https://reader036.vdocuments.mx/reader036/viewer/2022062721/56813866550346895da01403/html5/thumbnails/2.jpg)
Outline
Introduction
Swing & Slide Filters
Experiments
Conclusion
![Page 3: Online Piece-wise Linear Approximation of Numerical Streams with Precision Guarantees](https://reader036.vdocuments.mx/reader036/viewer/2022062721/56813866550346895da01403/html5/thumbnails/3.jpg)
Application Scenario
Transmitter Receiver
Some Common Applications: Cluster Monitoring Sensor Networks Stock Market
![Page 4: Online Piece-wise Linear Approximation of Numerical Streams with Precision Guarantees](https://reader036.vdocuments.mx/reader036/viewer/2022062721/56813866550346895da01403/html5/thumbnails/4.jpg)
The Problem
Goal Minimize amount of transmitted data
Saves bandwidth Saves storage (at the receiver side) Saves battery life (esp. for sensor networks)
Using piece-wise linear approximation
Assumptions Receiver can tolerate:
A bounded error for each data point received (max error = ) A maximum lag behind the transmitter
Terminology We refer to any algorithm to solve this problem as a filtering
technique, or simply a filter
![Page 5: Online Piece-wise Linear Approximation of Numerical Streams with Precision Guarantees](https://reader036.vdocuments.mx/reader036/viewer/2022062721/56813866550346895da01403/html5/thumbnails/5.jpg)
Existing Techniques
t1 t2 t3 t4 t5
Time
Val
ue
x1
x2x3
x4
x5
Cache Filter The transmitter caches
the last transmitted value.
A new value is transmitted only if it is more than away from the cached value.
Piece-wise constant approximation
![Page 6: Online Piece-wise Linear Approximation of Numerical Streams with Precision Guarantees](https://reader036.vdocuments.mx/reader036/viewer/2022062721/56813866550346895da01403/html5/thumbnails/6.jpg)
Existing Techniques
t1 t2 t3 t4 t5
Time
Val
ue
x1
x2x3
x4
x5
Cache Filter The transmitter caches
the last transmitted value.
A new value is transmitted only if it is more than away from the cached value.
Piece-wise constant approximation
![Page 7: Online Piece-wise Linear Approximation of Numerical Streams with Precision Guarantees](https://reader036.vdocuments.mx/reader036/viewer/2022062721/56813866550346895da01403/html5/thumbnails/7.jpg)
Existing Techniques
t1 t2 t3 t4 t5
Time
Val
ue
x1
x2x3
x4
x5
Linear Filter The transmitter maintains
a line segment that can approximate the last observed data points.
The line segment is updated only when a new data point falls more than away from the maintained line.
![Page 8: Online Piece-wise Linear Approximation of Numerical Streams with Precision Guarantees](https://reader036.vdocuments.mx/reader036/viewer/2022062721/56813866550346895da01403/html5/thumbnails/8.jpg)
Outline
IntroductionIntroduction
Swing & Slide Filters
Experiments
Conclusion
![Page 9: Online Piece-wise Linear Approximation of Numerical Streams with Precision Guarantees](https://reader036.vdocuments.mx/reader036/viewer/2022062721/56813866550346895da01403/html5/thumbnails/9.jpg)
Swing and Slide Filters
Key Idea
Maintain a set of candidate line segments at any given time
Postpone the selection decision as late as possible to accommodate more points
![Page 10: Online Piece-wise Linear Approximation of Numerical Streams with Precision Guarantees](https://reader036.vdocuments.mx/reader036/viewer/2022062721/56813866550346895da01403/html5/thumbnails/10.jpg)
Swing Filter
t1 t2 t3 t4 t5
Time
Val
ue
x1
x2x3
x4
x5
Connected line segments
Complexity Maintains upper and lower
segments only O(1) space and time
complexity
Lag If max lag is exceeded,
switch to linear filter
Correctness Proof of correctness in the
paper
![Page 11: Online Piece-wise Linear Approximation of Numerical Streams with Precision Guarantees](https://reader036.vdocuments.mx/reader036/viewer/2022062721/56813866550346895da01403/html5/thumbnails/11.jpg)
Slide Filter
t1 t2 t3 t4 t5
Time
Val
ue
x1
x2x3
![Page 12: Online Piece-wise Linear Approximation of Numerical Streams with Precision Guarantees](https://reader036.vdocuments.mx/reader036/viewer/2022062721/56813866550346895da01403/html5/thumbnails/12.jpg)
Slide Filter
t1 t2 t3 t4 t5
Time
Val
ue
x1
x2x3
x4
![Page 13: Online Piece-wise Linear Approximation of Numerical Streams with Precision Guarantees](https://reader036.vdocuments.mx/reader036/viewer/2022062721/56813866550346895da01403/html5/thumbnails/13.jpg)
Slide Filter
t1 t2 t3 t4 t5
Time
Val
ue
x1
x2x3
x4
x5
Optimization #1 Connect line segments whenever
possible
Optimization #2 Do not maintain all the data points
currently being approximated Maintain their convex hull only
Complexity O(h) space and time complexity h is the number of data points on the
convex hull --- very small in practice
Lag If max lag is exceeded, switch to linear
filter
Correctness Proof of correctness in the paper
![Page 14: Online Piece-wise Linear Approximation of Numerical Streams with Precision Guarantees](https://reader036.vdocuments.mx/reader036/viewer/2022062721/56813866550346895da01403/html5/thumbnails/14.jpg)
Outline
IntroductionIntroduction
Swing & Slide FiltersSwing & Slide Filters
Experiments
Conclusion
![Page 15: Online Piece-wise Linear Approximation of Numerical Streams with Precision Guarantees](https://reader036.vdocuments.mx/reader036/viewer/2022062721/56813866550346895da01403/html5/thumbnails/15.jpg)
Compression Ratios for the Sea Temperature Signal
20.5
21
21.5
22
22.5
23
23.5
24
24.5
0 2000 4000 6000 8000 10000 12000Time (minutes)
Tem
per
atu
re (
C)
0
10
20
30
40
50
60
70
80
90
0 0.1 0.316 1 3.16 10
Precision width (% of range)
Co
mp
ress
ion
rat
io
cache linear
sw ing slide
Sea Surface Temperature
![Page 16: Online Piece-wise Linear Approximation of Numerical Streams with Precision Guarantees](https://reader036.vdocuments.mx/reader036/viewer/2022062721/56813866550346895da01403/html5/thumbnails/16.jpg)
Effect of Signal Behavior (Degree of Monotonicity)
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
0 0.1 0.2 0.3 0.4 0.5
Probability of decrease in value per data point
Co
mp
ress
ion
rat
io
cache linear
sw ing slide
Synthetic Signal:
Random walk with probability p to increase and (1-p) to decrease.
![Page 17: Online Piece-wise Linear Approximation of Numerical Streams with Precision Guarantees](https://reader036.vdocuments.mx/reader036/viewer/2022062721/56813866550346895da01403/html5/thumbnails/17.jpg)
Overhead for the Sea Temperature Signal
0
10
20
30
40
50
60
0 0.1 0.316 1 3.16 10 31.6 100
Precision width (% of range)
Pro
cess
ing
tim
e (
ms
/ dat
a p
oin
t)cache linearsw ing non-optimized slideoptimized slide
![Page 18: Online Piece-wise Linear Approximation of Numerical Streams with Precision Guarantees](https://reader036.vdocuments.mx/reader036/viewer/2022062721/56813866550346895da01403/html5/thumbnails/18.jpg)
Outline
IntroductionIntroduction
Swing & Slide FiltersSwing & Slide Filters
ExperimentsExperiments
Conclusion
![Page 19: Online Piece-wise Linear Approximation of Numerical Streams with Precision Guarantees](https://reader036.vdocuments.mx/reader036/viewer/2022062721/56813866550346895da01403/html5/thumbnails/19.jpg)
Conclusion
We introduced two new filtering techniques: the swing and slide filters
They have significantly higher compression ratios compared to earlier techniques, especially the slide filter
They have a small overhead, and hence are suitable for overhead-sensitive applications
![Page 20: Online Piece-wise Linear Approximation of Numerical Streams with Precision Guarantees](https://reader036.vdocuments.mx/reader036/viewer/2022062721/56813866550346895da01403/html5/thumbnails/20.jpg)
Thank you
Questions?