robert w. pinder, alice b. gilliland, robert c. gilliam, k. wyat appel atmospheric modeling...
DESCRIPTION
Sources of Uncertainty Structural Uncertainty: VOC species lumping Physical processes Approach: vary representation Parameter Uncertainty: Emissions Meteorology Chemical rate constants Approach: Monte Carlo methods Challenge: Monte Carlo methods are not feasible given CMAQ’s computational requirements.TRANSCRIPT
Robert W. Pinder, Alice B. Gilliland, Robert C. Gilliam, K. Wyat AppelAtmospheric Modeling Division, NOAA Air Resources Laboratory,
in partnership with USEPA National Exposure Research Laboratory
2007 CMAS ConferenceOctober 2, 2007
Evaluating uncertainty predictions using an ensemble of CMAQ model
configurations
Objective• Uncertainty: what is the
likelihood that the observed value is within a given range?
• Applications: Exposure studies Diagnostic evaluation tool
• One measure of uncertainty is the error when the model is compared with observations.
• Can we use an ensemble approach to make a better estimate of this range?
Histogram of CMAQ O3 Model Error 8-hour max O3, 228 AQS Sites from the SE US
Sources of UncertaintyStructural Uncertainty:
VOC species lumpingPhysical processesApproach: vary representation
Parameter Uncertainty:EmissionsMeteorologyChemical rate constantsApproach: Monte Carlo methods
Challenge: Monte Carlo methods are not feasible given CMAQ’s computational requirements.
Method
• Variety of CMAQ / MM5 model configurations
• Direct sensitivity calculations• Use observations to remove
spurious ensemble members
Generate Ensemble Members for Structural Uncertainty using Multiple Model Configurations
Planetary Boundary Layer / Land Surface Model Pleim-Xiu Land Surface Model; ACM: Asymmetric
Convective Model (Pleim and Chang, 1992) Miller-Yamada-Janjic (Janjic, 1994), NOAH Land
Surface Model Medium Range Forecast (Hong and Pan, 1996), NOAH
Land Surface Model
Chemical Mechanism Carbon Bond IV SAPRC-99
Six structural uncertainty cases
Generate Ensemble Members for Parametric Uncertainty using Direct Sensitivity Calculation
VOCNO
OVOCNO
VOCO
2VOC
NOO
2NO
VOCOVOC
NOONOO
x
32
x
23
22
2x
322
x
3
x
3x3
Use the Direct Decoupled Method (DDM) to calculate sensitivity to:• NOx Emissions• VOC Emissions• Second-order sensitivity• O3 Boundary conditions
Compared to brute-force calculation, errors are 5-10% (Cohan et al., 2005)
At each grid cell, calculate ozone response to emissions and boundary concentrations
Direct Calculation of Ozone SensitivityJuly 16, 2002, 2 PM
>30 -50510152025
)NO(EmissionO
x
3
)OCEmission(VO3
)O(BoundaryO
3
3
O3 (ppb)
Use Observations to Constrain Ensemble
Used to evaluate boundary conditions
Used to evaluate ensemble quality
AQS O3 Monitoring Sites
Repeat 200 times
Structural uncertainty simulations (6)
Use DDM to calculate O3 sensitivity to NOx, VOC, and boundary conditions.
Randomly sample from range of uncertain NOx emissions, VOC emissions, boundary concentrations, and structural uncertainty cases
Generate an ensemble member by calculating the O3 field acrossSE US domain
Use observations to remove spurious ensemble members
Example: Atlanta, GeorgiaJuly 1-28, 2002
Structural Uncertainty
Structural + Parametric Uncertainty
Spread is large – can we use the observations to narrow this range?
Prune ensemble members not consistent with observations
Remove ensemble members that do not constrain the range
Pruned ensemble has narrow range while still including observations
200 member ensemble10 member ensemble
Compare with +/-30%
Range is 40% lower± 30% of base case CMAQ10 member ensemble
Analysis at all sitesDataset:• 38 locations, 28 days• 1064 observations
Evaluation:• Randomly reserve 50% of dataset• Derive ensemble, prune using half
of observations• Evaluate using the reserve dataset
Ensemble range includes 85% of observations
Range is 40% smaller than ±30% of base case
AQS O3 Monitoring Sites
Trade-off between coverage of observations and spread in range
Ensemble Size
Observations within Range
Average Ensemble Range
10 70% ± 12%
20 72% ± 12%
50 75% ± 15%
100 83% ± 18%
200 85% ± 18%
± 30% CMAQ
91% ± 30%
Conclusions• This ensemble generation and pruning technique
provides a more robust uncertainty range: Observed value is within the range 85% of the time 40% reduction in spread compared to +/- 30% rule
• Simultaneously narrowing these bounds and improving the performance depends on reducing structural errors in CMAQ
• Locations and times that fall outside of the ensemble range should be targeted for uncovering structural errors in the model
DISCLAIMER: The research presented here was performed under the Memorandum of Understanding between the U.S. Environmental Protection Agency (EPA) and the U.S. Department of Commerce's National Oceanic and Atmospheric Administration (NOAA) and under agreement number DW13921548. This work constitutes a contribution to the NOAA Air Quality Program. Although it has been reviewed by EPA and NOAA and approved for publication, it does not necessarily reflect their policies or views.
Acknowledgements:
Sergey Napelenok, Jenise Swall, Kristen Foley