advanced tutorial on : global offset and residual covariance
DESCRIPTION
Advanced Tutorial on : Global offset and residual covariance. ENVR 468 Prahlad Jat and Marc Serre. Agenda. Why use a global offset? How is the global offset calculated ? Remove the global offset from data Effect of global offset on covariance. Why use a Global Offset? . - PowerPoint PPT PresentationTRANSCRIPT
Advanced Tutorial on :Global offset and residual
covariance
ENVR 468
Prahlad Jat and Marc Serre
Agenda Why use a global offset? How is the global offset calculated ? Remove the global offset from data Effect of global offset on covariance
Why use a Global Offset? We may be interested in
mapping a global trend (global warming).
To model short range variability more accurately.
The Trend Analysis can help to identify a global trend in the user dataset if it exists.
Variability =f (short range, long range variability)
Short range variability can in some cases be modeled in the global offset in the data.
However, there is a real danger of over fitting the data when using the global offset and leaving too little variation in the residuals to properly account for the uncertainty in the prediction.
What is our dilemma ?
Desirable: I. Low residual variability (for global offset with small range
variability) II. Long autocorrelation range in covariance model (very flat global
offset)
A global offset with small range variability is very informative and therefore leaves little autocorrelation in the residuals .
A flat global offset leaves too much variability in the residuals.
A tradeoff between residual variability and autocorrelation range is needed: One should choose a mean trend which captures some variability and leaves reasonable autocorrelation in the residuals
What we want to achieve ?
Model the Global Offset Temporal plot of Z versus time t for Monitoring Station 1 and 2
There is a temporal trend of increasing values with time
Model the Global Offset Spatial plot of Z versus monitoring event 1 and 2
There is a spatial trend of increasing values from left to right
Model the Global Offset Residual data plots
There is no trend in residual
Model the Global Offset
We model the S/TRF Z(s,t) as the sum of a global offset mz(s,t) and residual S/TRF X(s,t)
Z(s,t) = mz(s,t) + x(s,t)
Model the Global Offset BMEGUI assumes that the global offset is a space/time additive separable function i.e. space/time mean trend
Where : ms(s) is the spatial component and mt(t) is the temporal component
mz(s,t) = ms(s) + mt(t)
Model the Global Offset Temporal plot of log PM2.5 (ug/m3) versus time (days)
0 100 200 300 400 500 600 700 8001
1.5
2
2.5
3
3.5
4
Days
log(
PM
25) u
g/m3
0 100 200 300 400 500 600 700 8000.5
1
1.5
2
2.5
3
3.5
4
Days
log(
PM
25) u
g/m3
0 100 200 300 400 500 600 700 8000.5
1
1.5
2
2.5
3
3.5
4
4.5
Days
log(
PM
25) u
g/m3
0 100 200 300 400 500 600 700 800-0.5
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
Days
log(
PM
25) u
g/m3
0 100 200 300 400 500 600 700 800-0.5
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
Days
log(
PM
25) u
g/m3
0 100 200 300 400 500 600 700 800-1
0
1
2
3
4
5
Days
log(
PM
25) u
g/m3
0 100 200 300 400 500 600 700 800-1
0
1
2
3
4
5
Days
log(
PM
25) u
g/m3
0 100 200 300 400 500 600 700 800 900-8
-6
-4
-2
0
2
4
6
Days
log(
PM
25) u
g/m3
Model the Global Offset
Global Offset
Model the Global Offset Temporal plot of log PM2.5 (ug/m3) versus time (days)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 190
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Days
log(
PM
25) u
g/m
3
MS
iji
j n
tsPMtmt
),(25)(
Take the sum of all observations at time and divide it by number of observations; apply exponential filter for smoothness in the trend
jt
Tradius
Sradius
i range
i rangei
Sd
Sdms
SSM
exp
exp
j range
j rangej
Td
Tdmt
STM
exp
exp
Smoothen the Global Offset
Model the Global Offset
Global Offset for MS=4
Model the Global Offset
0 100 200 300 400 500 600 700 800-1
0
1
2
3
4
5
Days
log(
PM
25) u
g/m
3
Observed PM25
4),(25 itsPM ji
Time series of observed log(PM2.5) at MS 4.
We want to model the global offset at this MS
Apply exponential filter to smoothen the global offset
Remove the global offset and obtain residuals for covariance modeling
Model the Mean Trend
Time series of observed log(PM2.5) at MS 4.
Plot mt, the temporal component of the global offset
Shift the temporal global offset to zero (i.e. calculate mt–mean(mt))
Add the spatial component of the global offset, i.e. add ms to mt–mean(mt)
0 100 200 300 400 500 600 700 800 900-1
0
1
2
3
4
5
Days
log(
PM
25) u
g/m
3
Observed PM25Global temporal trend
MS
iji
j n
tsPMtmt
),(25)(
0 100 200 300 400 500 600 700 800 900-1
0
1
2
3
4
5
Days
log(
PM
25) u
g/m3
Observed PM25Global temporal trendDiff of Global-Observed
Model the Global Trend
0 100 200 300 400 500 600 700 800 900-1
0
1
2
3
4
5
Days
log(
PM
25) u
g/m
3
Observed PM25Global temporal trendDiff of Global-ObservedDiff + Spatial
)]()([)( mtmeantmtsms ji
ME
ji
i n
tsPMsms
),(25)(
Add spatial trend to this final temporal trendspatial trend + [temporal Global trend – mean of temporal global trend]
mz(s,t) = ms(s) + mt(t)
Removing the mean trend from data
Remove mean trend (i.e. global offset) and obtain residuals
Use residual data for covariance modeling
0 100 200 300 400 500 600 700 800-4
-3
-2
-1
0
1
2
3
4
5
Days
log(
PM
25) u
g/m3
Residual plot MS =4
x(s,t) = Z(s,t) - mz(s,t)
mz(s,t) = ms(s) + mt(t)
Mean Trend in BMEGUICase1 : Flat mean trend Case 2: Informative mean trend
Mean Trend in BMEGUI Case1 : Flat mean trend Case 2: Informative mean trend
Covariance Models in BMEGUI
Case1 : Flat mean trend Case 2: Informative mean trend
Flat Mean trend
Structure 1 Structure 2
Spatial Temporal Spatial TemporalSill 0.2 0.19 Model exp exp exp expRange 4 7 100 75
Very smoothened Mean trend
Structure 1
Structure 2
Spatial Temporal Spatial TemporalSill 0.05 0.0619 Model exp exp exp expRange 1.5 5 3 25
Covariance Models in BMEGUI
Flat Mean trend
Structure 1 Structure 2
Spatial Temporal Spatial TemporalSill 0.2 0.19 Model exp exp exp expRange 4 7 100 75
Very smoothened Mean trend
Structure 1
Structure 2
Spatial Temporal Spatial TemporalSill 0.05 0.0619 Model exp exp exp expRange 1.5 5 3 25
Case1 : Flat mean trend Case 2: Informative mean trend
Fitted Covariance Models
Spatial Component
Temporal Component
case
Search radius (deg.)
Smoothing range (deg.)
Search radius (days)
Smoothing range (days)
1 15 15 1000 1000
3 1 1 60 60
4 0.2 0.2 10 10
5 0.1 0.1 5 5
2 0.001 0.001 0.1 0.1
Changes in the smoothness in the mean trend we observe changes in the experimental covariance.
An extremely smoothed (i.e. flat) mean trend results in higher residual variance and larger spatial and temporal autocorrelation ranges.
On the other hand, very informative mean trend results in smaller residual variance but shorter spatial and temporal autocorrelation ranges.
Temporal Dist. Est. in BMEGUI
Case1 : Flat mean trend Case 2: Informative mean trend
Mean Trend ConclusionsEach mean trend model represents a tradeoff between residual variance and autocorrelation range.
Very flat mean trend: the highest residual variance but longer autocorrelation
Very informative mean trend: low residual variance but short autocorrelation range
The optimal level is the breakpoint where further decrease in smoothness results a drastic decreases in autocorrelation range. (green circle)
