GeoMAN: Multi-level Attention Networks for Geo-sensory Time Series Prediction
Yuxuan Liang1,2, Songyu Ke3,2, Junbo Zhang2,4, Xiuwen Yi4,2, Yu Zheng2,1,3,4
1 Xidian University, Xi an, China2 Urban Computing Business Unit, JD Finance, Beijing, China3 Shanghai Jiao Tong University, Shanghai, China4 Southwest Jiaotong University, Chengdu, China
Codes & Data
Temporal Attn
Conca
t
External Factor Fusion Multi-level Attention Network
LSTM
LSTM
LSTMTime Features Embed
Weather Forecasts Embed
SensorID Embed
POIs & Sensor NetworksDecoder
LSTM
Spatial Attn
LSTM
Spatial Attn
LSTM
Spatial Attn
POIs Model Input
h0
Spatial AttentionEncoder
Local Global
Concat
tc
1tc
c
1ˆ i
ty
ˆ i
ty
ˆ iy
Sensor Networks
Meteorology
Geo-sensoryTime series
Time
Methodology
Spatial Attention Capture dynamic inter-sensor correlation
Local: adaptively captures the correlation between target series and local features (other series)
Global: adaptively select the relevant sensors to make predictions
Temporal Attention
Select relevant historical time slots to make predictions
Model Training
Encoder-decoder + Multi-level attention
GeoMAN is smooth and differentiable
Loss function: MSE
Optimizer: Adam
tanh tanh tanh
softmax
... ...
... ...
... ...
concat
tanh tanh tanh
softmax
... ...
... ...
... ...
concat
similarity matrix Local Attn Global Attn
Concat
(a) Air quality stations in Beijing
0
3
6
9
S6 S11 S16 S23S1
(c) Plot of global spatial attention weights
(b) Plot of local spatial attention weights
Remote sensors
0
3
6
9
Wind speed towards different directionsAir pollutants
Southeast
wind
Hu
mid
ity
NO
2
Tem
per
atur
e
En
cod
er
Ste
p
En
cod
er
Ste
p
S13
S0
S1
S11
S6
S13
S17 S32
S23
S27 S26
S3
S4S16
Target sensor Discussed sensor
Results Visualization
Introduction
Geo-sensory time series Properties
Examples
GoalS4 S2
S3
Time
S1
Spatial
correlation
Temporal
correlation
t1
t3
t4
t2
t1
t2
t3
t4
t1
t2
t1
t2
t3
t5
Sudden change
Each sensor has a unique geospatial location
Reporting time series readings about different measurements
With geospatial correlation between their readings
Challenges Affected by many factors
Dynamic inter-sensor correlation
Dynamic temporal correlation
Readings of previous time interval
Readings of nearby sensors
External factors
Day 1 Day 2 Predict target series of a sensor over several future hours
Framework
External factors fusion module
Multi-level attention network Spatial attention
Temporal attention
Datasets: water quality dataset & air quality dataset
Insight
Local
Information
Sensor
Correlation
Unobserved factors Temporal factors Spatial factors
Sensor
Networks
Land
Function
POIs Time Weather
External
Factors
Target
Series
Global
Readings
Local
Readings
Local ViewGlobal View
Encoder Decoder
Sensors
Roads
Volume: 32
Speed: 50km/h
SensorsPipelines
RC: 0.84
pH: 7.1
Turbidity: 0.54
TimeWeather
POIs Sensor Network
