examination of observation impacts derived from oses and adjoint models
DESCRIPTION
Examination of Observation Impacts Derived from OSEs and Adjoint Models. Ron Gelaro and Yanqiu Zhu. NASA Global Modeling and Assimilation Office. Ricardo Todling, Ron Errico. Upper-air virtual temperature. Brightness temperature. Surface (2m) pressure. Data Types. Rain rate. - PowerPoint PPT PresentationTRANSCRIPT
Examination of Observation Impacts Derived from OSEs and Adjoint Models
Ron Gelaro and Yanqiu Zhu
NASA Global Modeling and Assimilation Office
Ricardo Todling, Ron Errico
(x 1,000,000)2.21.1
Data Types
Brightness temperatureUpper-air virtual temperature
Surface (2m) pressure
Upper-air specific humidityUpper-air meridional windUpper-air zonal wind
Rain rate
The observing system today…a 6-hr ‘snapshot’ GEOS-5 GSI 09-Aug-2007 12UTC All Data: 2,794,770 observations
The data volumes entailed by future observing systems will massively increase over the next 10 years … new approaches to ingest, process, monitor, quality control, assimilate and archive the data will have to be developed. ECMWF 10-year plan, 2006-2015
Observing System Experiments (OSEs)
The traditional method of assessing the impact of observations on forecast skill…
Subsets of observations are removed from the assimilation system and forecasts are compared against a ‘control’ system that includes all observations
Performed intermittently at operational centers but, because of their expense, usually involve a relatively small number of independent experiments, each considering relatively large subsets of observations
But what if one wants to investigate, for example, the impact of all individual channels on a given satellite…and over arbitrary periods of time, or even routinely…?
Outline of Talk
Estimation of observation impact – adjoint (ADJ) method
Observation impact results in NASA’s GEOS-5 DAS
Comparison of ADJ and OSE results
Combined use of ADJ and OSEs
Concluding remarks
Atmospheric forecast model:
)( 0xmx =f
Note that may be viewed as a transformation between a perturbation in state space and a perturbation in observation space
ba0 xxx −=δ
Atmospheric analysis (best estimate of ) :0x
)( bxhyy −=δwhere: (increment, correction vector)
(innovation vector ~106)
Data Assimilation-Forecast System
yKx δδ =0
yKx δδ =0
K determines the scalar weight (gain) given to each observation
Forecast error measure (dry energy, sfc–140 hPa):
Estimating Observation Impact
)()( v0T
v0 xxCxx −−= ffe
gxxx
xxx
x T0
203
0
3
020
2
00 )(...)
6
1
2
1( δδδδδ =+
∂∂+
∂∂+
∂∂= eee
e
Taylor expansion of change in due to change in :e 0x
3rd order approximation of in observation space:
)]()([)( vaTavb
Tb
TT xxCMxxCMKy −+−≈ ffe δδ
eδ
model adjointanalysis adjoint
3T ~)( gyδ=
…summed observation
impact
Transformation to observation-space:
yKxxx δδ =−= ba0
3T ~)( gyδδ ≈e
Properties of the Impact Estimate
0<eδ0>eδ
…the observation improves the forecast
…the observation degrades the forecast
…see Langland and Baker (2004), Errico (2007), Gelaro et al. (2007)
The “weight” vector is computed only once, and involves the entire set of observations; removing or changing the properties of one observation changes the scalar measure of all other observations.
3~g
Valid forecast range limited by tangent linear assumption for TM
The impact of arbitrary subsets of observations (e.g. instrument type, channel, location) can be easily quantified by summing only the terms involving the desired elements of .yδ
GEOS-5 Observation Impact Experiments
Analysis System
3DVAR Gridpoint Statistical Interpolation (GSI) 0.5o resolution, 72 levels
Forecast Model
GEOS-5: FV-core + full physics 0.5o resolution, 72 levels
Adjoint: Exact line-by-line (Zhu and Gelaro 2007)
Adjoint: FV-core 1o resolution + simple dry physics
Experimentation
6h data assimilation cycle, July 2005 and January 2006 24h forecasts from 00UTC to assess observation impact
Separate error response functions for the globe, NH, SH and tropics
January 2006 Global Error Norm
-30
-25
-20
-15
-10
-5
0
5
amsuaamsubairs hirs goes msu raobs
satwindsspssmiaircraftsurfaceqkswnd
January 2006 NH Error Norm
-20
-15
-10
-5
0
5
amsuaamsubairs hirs goes msu raobs
satwindsspssmiaircraftsurfaceqkswnd
January 2006 SH Error Norm
-20
-15
-10
-5
0
5
amsuaamsubairs hirs goes msu raobs
satwindsspssmiaircraftsurfaceqkswnd
January 2006 Tropical Error Norm
-6
-4
-2
0
2
amsuaamsubairs hirs goes msu raobs
satwindsspssmiaircraftsurfaceqkswnd
N. Hemisphere (20o-80o)
S. Hemisphere (20o-80o) Tropics (20o-20o)
Global
Total 24hr Forecast Error Reduction due to Observations
GEOS-5 Adjoint Data Assimilation System
(J/kg)
eδ
(J/kg)
eδ
January 2006 00UTC
July 2005 Global Error Norm
-30
-25
-20
-15
-10
-5
0
5
amsuaamsubairs hirs goes msu raobs
satwindsspssmiaircraftsurfaceqkswnd
July 2005 NH Error Norm
-20
-15
-10
-5
0
5
amsuaamsubairs hirs goes msu raobs
satwindsspssmiaircraftsurfaceqkswnd
July 2005 SH Error Norm
-20
-15
-10
-5
0
5
amsuaamsubairs hirs goes msu raobs
satwindsspssmiaircraftsurfaceqkswnd
July 2005 Tropical Error Norm
-6
-4
-2
0
2
amsuaamsubairs hirs goes msu raobs
satwindsspssmiaircraftsurfaceqkswnd
N. Hemisphere (20o-80o)
S. Hemisphere (20o-80o) Tropics (20o-20o)
Global
GEOS-5 Adjoint Data Assimilation System
(J/kg)
eδ
(J/kg)
eδ
Total 24hr Forecast Error Reduction due to Observations
July 2005 00UTC
Global impact of most channels is beneficial on average…with some exceptions
Cha
nnel
sC
hann
els
Forecast Error Reduction (J/kg)
Forecast Error Reduction (J/kg)
Impact of satellite observations by channel
AMSU-A
AIRS
July 2005 00UTC
improve degrade
0.0
0.0-7.0
-0.7
GEOS-5 Adjoint Data Assimilation System
Localized examination of AIRS impactsJuly 2005 00UTC
AIRS impact map
H20
degrade
AIRS impact by channel
Removal of AIRS water vapor
channels improved the
forecast scores
(All Channels)
(20-50N, 0-80E)
degrade
improve
Comparison of ADJ results with OSEs
How do observation impact results based on the ADJ method compare with traditional observing system experiments…OSEs?
Can the two approaches be meaningfully compared?
GEOS-5 OSEs were conducted for July 2005 and January 2006 00UTC forecasts at 1o horizontal resolution
OSE Growth of S.H. Energy Norm
0
5
10
15
20
25
30
35
40
1 2 3 4 5
Forecast Day
Energy (J/kg)
control
no raobs
no amsua
no satwnd
no aircraft
OSE Growth of N.H. Energy Norm January 2006
0
5
10
15
20
25
30
35
40
1 2 3 4 5
Forecast Day
Energy (J/kg)
control
no raobs
no amsua
no satwnd
no aircraft
ControlNo raobNo amsuaNo satwindNo aircraft
Forecast Day Forecast Day
Skill (impact) measured using the same energy-based error metric for the globe, NH, SH and tropics as in the ADJ experiments
eJ/kg
July 2005 S. Hemisphere January 2006 N. Hemisphere
Comparison and Interpretation of ADJ and OSE Results
The ADJ measures the impact of observations in each analysis cycle separately and against the control background, while the OSE measures the impact of removing observational information from both the background and analysis in a cumulative manner
The ADJ measures the impacts of observations in the context of all other observations present in the assimilation system, while the OSE changes/degrades the system (i.e., differs for each OSE member)
The ADJ measures the response of a single forecast metric to all perturbations of the observing system, while the OSE measures the effect of a single perturbation on all forecast metrics
The ADJ is restricted by the tangent linear assumption (valid ~1-3 days), while the OSE is not
…a few things to keep in mind…
K
)()( v0T
v0 xxCxx −−= ffe
)]()([)( vaTavb
Tb
TT xxCMxxCMKy −+−= ffe δδ
‘Direct’ quantitative comparison of ADJ and OSEs
Define the fractional impact of observing system for each approach:jF j
eeF jj δδ / ADJ)( =
ctlctlno /)( OSE)( eeeFjj −=
Measures the % decrease in error due to the presence of observing system with respect to the background forecastj
1 ADJ)( =∑ j jF
Measures the % increase in error due to the removal of observing system with respect to the control forecastj
1 OSE)( ≠∑ j jF
% Impact Jan06 SH
0
5
10
15
20
25
30
35
40
amsua1amsua2amsua3
airsraobs
satwindsaircraftqkswnd
ADJ
OSE
% Contributions to 24hr Forecast Error Reduction January 2006
OSE ADJ% Impact Jan06 Global
0
5
10
15
20
25
30
35
amsua1amsua2amsua3
airsraobs
satwindsaircraftqkswnd
ADJ
OSE1
% Impact Jan06 NH
-5
0
5
10
15
20
25
30
35
40
45
50
amsua1amsua2amsua3
airsraobs
satwindsaircraftqkswnd
ADJ
OSE
% Impact Jan06 Tropics
05
101520253035404550556065
amsua1amsua2amsua3
airsraobs
satwindsaircraftqkswnd
ADJ
OSE1
Tropics (20o-20o)S. Hemisphere (20o-80o)
N. Hemisphere (20o-80o)Global
%
%
% Impact July05 SH
0
5
10
15
20
25
30
35
40
45
50
55
60
amsua1amsua2amsua3
airsraobs
satwindsaircraftqkswnd
ADJ
OSE
% Impact July05 Tropics
05
10152025303540455055606570
amsua1amsua2amsua3
airsraobs
satwindsaircraftqkswnd
ADJ
OSE1
OSE ADJ% Impact July05 Global
0
5
10
15
20
25
30
35
40
45
50
amsua1amsua2amsua3
airsraobs
satwindsaircraftqkswnd
ADJ
OSE1
% Impact July05 NH
0
5
10
15
20
25
30
35
40
45
50
amsua1amsua2amsua3
airsraobs
satwindsaircraftqkswnd
ADJ
OSE
% Contributions to 24hr Forecast Error Reduction July 2005
%
%
Tropics (20o-20o)S. Hemisphere (20o-80o)
N. Hemisphere (20o-80o)Global
Normalized % Impact Jan06 SH
0
0.05
0.1
0.15
0.2
0.25
0.3
amsua1amsua2amsua3
airsraobs
satwindsaircraftqkswnd
ADJ
OSE
Normalized % Impact Jan06 Global
0
0.05
0.1
0.15
0.2
0.25
0.3
amsua1amsua2amsua3
airsraobs
satwindsaircraftqkswnd
ADJ
OSE1
Normalized % Contributions to 24hr Fcst Error Reduction January 2006
OSE ADJ
%
%
Normalized % Impact Jan06 Tropics
0
0.05
0.1
0.15
0.2
0.25
amsua1amsua2amsua3
airsraobs
satwindsaircraftqkswnd
ADJ
OSE1
Normalized % Impact Jan06 NH
-0.05
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
amsua1amsua2amsua3
airsraobs
satwindsaircraftqkswnd
ADJ
OSE
Tropics (20o-20o)S. Hemisphere (20o-80o)
N. Hemisphere (20o-80o)Global
Normalized % Impact July05 SH
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
amsua1amsua2amsua3
airsraobs
satwindsaircraftqkswnd
ADJ
OSE
Normalized % Contributions to 24hr Fcst Error Reduction July 2005
%
%
Normalized % Impact July05 Global
0
0.05
0.1
0.15
0.2
0.25
0.3
amsua1amsua2amsua3
airsraobs
satwindsaircraftqkswnd
ADJ
OSE1
Normalized % Impact July05 NH
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
amsua1amsua2amsua3
airsraobs
satwindsaircraftqkswnd
ADJ
OSE
Normalized % Impact July05 Tropics
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
amsua1amsua2amsua3
airsraobs
satwindsaircraftqkswnd
ADJ
OSE1
OSE ADJ
Tropics (20o-20o)S. Hemisphere (20o-80o)
N. Hemisphere (20o-80o)Global
0
5
10
15
20
25
30
35
amsuaamsubairs hirs goes
eos_amsua
msuraobssatwinds
spssmiaircraftsurfaceqkswnd
Control
No AMSUA
No Raob
No AIRS
% Contribution to 24h forecast error reduction
January 2006 Global Observations
Removal of AMSUA results in large increase in AIRS (and other) impacts
Removal of AIRS results in significant increase in AMSUA impact
Removal of Raobs results in significant increase in impact of several obs types, with AIRS and Satwinds being a notable exceptions
Combined Use of ADJ and OSEs
ADJ tool applied to each OSE member
Control
No AMSUA
No Raob
No Satwind
-1
0
1
2
3
4
5
6
7
amsuaamsubairs hirs goes
eos_amsua
msuraobssatwinds
spssmiaircraftsurfaceqkswnd
% Contribution to 24h forecast error reduction
July 2005 Tropical Observations
Removal of AMSUA results in large increase in AIRS impact in tropics
Removal of wind observations results in significant decrease in AIRS impact in tropics (in fact, AIRS degrades forecast without Satwinds!)
Combined Use of ADJ and OSEs
ADJ tool applied to each OSE member
Adjoint data assimilation system provides an accurate and efficient tool for estimating observation impact on analyses/short-term forecasts
computed with respect to all observations simultaneously permits arbitrary aggregation of results by data type, channel, location, etc.
Complement and extend, but not replace, traditional OSEs as tools for assessing observation impact…metrics, interpretations differ
Applicable to data quality assessment and selection, understanding DAS behavior, identifying redundancies in the observing system
Conclusions - 1
Excellently suited for real-time monitoring of observation impact: …see NRL page: http://www.nrlmry.navy.mil/ob_sens/
Despite fundamental differences in how impact is measured, ADJ and OSE methods provide comparable estimates of the overall ‘importance’ of most observing systems tested
Conclusions - 2
Used together, ADJ and OSEs illuminate the complex, complementary nature of how observations are used by the assimilation system
Comparisons of impacts in different forecast systems should help clarify deficiencies in data quality vs. assimilation methodology, and hopefully provide useful feedback to data producers.