dynamic time warping improves sewer flow...
TRANSCRIPT
![Page 1: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/1.jpg)
Dynamic Time Warping improves sewer flow monitoring
David Dürrenmatt Dario Del Giudice
Jörg Rieckermann
![Page 2: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/2.jpg)
What’s the matter?
![Page 3: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/3.jpg)
Quality control of discharge measurements
Problem • Flow meters show considerable errors under normal operating
conditions in sewers.
• Discharge − Ultrasonic flow meters: 10% − Tracer dilution methods: 6% to 16% − Venturi: 12% to 20%
• Ultrasonic velocity sensors
− Single-point: 14 - 18% − Multi-point: 4 - 5%
Hoppe (2009), Smits (2008)
![Page 4: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/4.jpg)
Quality control of discharge measurements
• Manual
calibration in the lab or field is expensive.
• Usually only point calibration once per year or 3 months.
• During dry weather conditions!
Hoppe (2009), Smits (2008)
![Page 5: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/5.jpg)
Quality control of discharge measurements Create independent information on flow velocities
Idea • Use “natural” tracers in wastewater to obtain independent
information on average flow velocities ⇒ Time shift of characteristic patterns between 2 measuring
locations A and B contains information on travel time θ.
A
![Page 6: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/6.jpg)
Quality control of discharge measurements Create independent information on flow velocities
Idea • Use “natural” tracers in wastewater to obtain independent
information on average flow velocities ⇒ Time shift of characteristic patterns between 2 measuring
locations A and B contains information on travel time θ.
⇒ Length of the sewer section is obtained from map or field measurements.
θLtv ≈)(
![Page 7: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/7.jpg)
Methods
![Page 8: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/8.jpg)
Ideal plug-flow reactor Concept
t t
Δt
V 𝜃 𝑡 =𝑉
𝑄(𝑡)
A B
C
D
Equations: Packet A: 2𝛥𝑡 → Δ𝑡 𝑄1 + 𝑄2 = 𝑉 Packet B: 2𝛥𝑡 → Δ𝑡 𝑄2 + 𝑄3 = 𝑉 Packet C: 3𝛥𝑡 → Δ𝑡 𝑄3 + 𝑄4 + 𝑄5 = 𝑉 Packet D: 4𝛥𝑡 → Δ𝑡 𝑄4 + 𝑄5 + 𝑄6 + 𝑄7 = 𝑉 …
𝑄1 𝑄2 𝑄3 𝑄4 𝑄5 𝑄6 𝑄7
![Page 9: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/9.jpg)
Ideal plug-flow reactor
Δ𝑡 𝑄1 + 𝑄2 = 𝑉 Δ𝑡 𝑄2 + 𝑄3 = 𝑉
Δ𝑡 𝑄3 + 𝑄4 + 𝑄5 = 𝑉 Δ𝑡 𝑄4 + 𝑄5 + 𝑄6 + 𝑄7 = 𝑉
Matrix notation:
𝑨 =
1 1 0 0 0 0 00 1 1 0 0 0 00 0 1 1 1 0 00 0 0 1 1 1 1
,
Q = 𝑄1,𝑄2, … ,𝑄7 𝑇 and b = 𝑉
Δ𝑡1, 1, 1, 1 𝑇
4 Equations:
𝑨𝑄 = 𝑏 with
How do we get the residence times of the water packets?
![Page 10: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/10.jpg)
Dynamic Time Warping
• “Warps” two sequences non-linearly in the time domain so that the dissimilarity is minimized
• Was originally developed for speech recognition • Is a standard technique for non-linear pattern matching
• Has been used to successfully estimate flow distribution in hydraulic flow dividers at WWTPs.
Dürrenmatt (2011)
![Page 11: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/11.jpg)
Dynamic Time Warping
(Keogh und Pazzani, 2000)
„Warping-Path“
Time series B
Time series A p
q
(p,q)=(8/10)
→ 𝜃𝐷𝑇𝐷 = 10 − 8 Δt = 2Δ𝑡 Δ𝑡
∑=kw
mnDBAd ,),(
![Page 12: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/12.jpg)
Dynamic Time Warping
Conditions for the warping path: • Starts and ends in
opposite corners • continuous • Steps are restricted • Pattern appears first in
A, then in B.
B
A
![Page 13: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/13.jpg)
Dynamic Time Warping
Added stochasticity to avoid local optima: 1. Add noise to
observations 2. Iterate computation of
warping paths
B
A
![Page 14: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/14.jpg)
Dynamic Time Warping
Added stochasticity to avoid local optima: 1. Add noise to
observations 2. Iterate computation of
warping paths 3. Compute average
warping
B
A 𝜃𝐷𝑇𝐷 = 𝜃
?
![Page 15: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/15.jpg)
… 𝜽𝑫𝑫𝑫 = 𝜽 ?
Illustration: Tracer experiment
Error: 𝐸 = 𝑉𝑡𝜇− 𝑉
𝑡𝑚≠ 0 if 𝑡𝜇 ≠ 𝑡𝑚
• The estimated travel time does not equal the true travel time in real systems (Dispersion, Reaction).
![Page 16: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/16.jpg)
Numerical experiments
![Page 17: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/17.jpg)
Numerical experiments
• Testing the method on virtual data to determine the field of application
• “Benchmark Simulation Environment” − Inflow generator (Discharge, Temperature) − Hydrodynamic heat transport model (Aquasim) − Sensor model (BSM 1, Class “A”)
A
B
T
t T
t
![Page 18: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/18.jpg)
Results (1) Example
Time [s]
![Page 19: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/19.jpg)
Results (1) Example
![Page 20: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/20.jpg)
Results (1) Accuracy of DTW velocity estimates
Confirms the theorectical considerations.
![Page 21: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/21.jpg)
Results (2) Dispersion and heat exchange
![Page 22: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/22.jpg)
Results (3) Sensor response time and error
![Page 23: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/23.jpg)
Results (4) Sampling frequency
![Page 24: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/24.jpg)
Application
![Page 25: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/25.jpg)
Real-world case study
• Testing the performance of 2 flow meters
• Measurement campaign: 2 weeks
• 2x Onset TMC6-HD thermistor w. HOBO logger • Accuracy: 0.25 °C • Resolution: 0.03 °C • T10/90: 30s (90%)
![Page 26: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/26.jpg)
Results (5) Online analysis
![Page 27: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/27.jpg)
Results (6) Offline analysis
![Page 28: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/28.jpg)
This looks nice. But...
![Page 29: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/29.jpg)
Discussion
• Pre-processing is important! High-pass filtering is better than normalization of the Temperature signals.
• Results chould be improved by using other or multiple tracers with near-conservative behaviour (e.g., Conductivity).
• Using a physically-based model for data analysis could also be promising.
Dürrenmatt, D.J., D. Del Giudice, J. Rieckermann et al., Dynamic time warping improves sewer flow monitoring (submitted to Water Research)
![Page 30: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/30.jpg)
Discussion Comparison to Cross-correlation with sliding windows
![Page 31: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/31.jpg)
Conclusions
![Page 32: Dynamic Time Warping improves sewer flow monitoringhikom.grf.bg.ac.rs/stari-sajt/9UDM/Presentations/178_PPT.pdf · Dynamic Time Warping improves sewer flow monitoring David Dürrenmatt](https://reader031.vdocuments.mx/reader031/viewer/2022041009/5eb5502ccd7a73169e1f3185/html5/thumbnails/32.jpg)
Conclusions
• Dynamic time warping (DTW) can retrieve sewer flow velocities from online measurements of wastewater quality.
• DTW extracts travel times from the temporal shift between upstream and downstream patterns by computing a non-linear warping path which maximizes the similarity between both patterns.
• The method is very well suited for the conditions found in typical sewer systems. Errors are estimated to less than 7.5%.
• The simple set-up and low experimental costs for sensors make it a practicable approach to diagnose sewer flow monitoring devices.