sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · school of earth...
TRANSCRIPT
![Page 1: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/1.jpg)
School of Earth and Environment
Sensitivity analysis and calibration of a global aerosol model
Lindsay Lee [email protected]
Ken Carslaw1, Carly Reddington1, Kirsty Pringle1, Jeff Pierce3, Graham Mann1, Jill Johnson1,
Dominick Spracklen1, Philip Stier2
1. University of Leeds 2. University of Oxford 3. Colorado State University
![Page 2: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/2.jpg)
1. Motivation
http://earthobservatory.nasa.gov/Features/CALIPSO/CALIPSO2.php
![Page 3: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/3.jpg)
2. GLOMAP
• We use the global aerosol model GLOMAP (Mann et al. 2010)
• A microphysical modal model simulating the evolution of global aerosol including sulphate, sea-salt, dust and black carbon
http://www.pnl.gov/atmospheric/research/aci/aci_aerosol_indeffects.stm
![Page 4: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/4.jpg)
3. Uncertainty in modelling
• Structural uncertainty
• Compare multiple models of the same type
• Compare model to observations
• AEROCOM • Aerosol Comparisons between Observations and
Models
• >14 models with the same emissions for 2000 and pre-industrial
![Page 5: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/5.jpg)
5
4. GLOMAP in AEROCOM
![Page 6: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/6.jpg)
6
4. GLOMAP in AEROCOM
![Page 7: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/7.jpg)
7
4. GLOMAP in AEROCOM
![Page 8: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/8.jpg)
8
4. GLOMAP in AEROCOM
![Page 9: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/9.jpg)
• More complex models = more uncertain parameters
• How do these uncertain parameters affect model predictions?
• Which processes in the model are dominating the prediction?
• Together with AEROCOM find the most important uncertainties
9
5. Parametric uncertainty
![Page 10: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/10.jpg)
• Ideally want to carry out sensitivity analysis
• Need a good estimation of the output distribution given uncertain inputs
• Consider GLOMAP output Y = η(X)
• X is uncertain so Y is uncertain
• Want to know f(Y) given G(X)
• Monte Carlo sampling impossible
• Have to use
10
6. Parametric uncertainty in a complex computer model
)(ˆ Yf
![Page 11: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/11.jpg)
• found using Bayesian statistics
• Use a limited number of model runs – training data
• Make some assumptions about the model behaviour – prior distribution
• Find the posterior distribution of the model – and use the mean as
11
7. Bayesian statistics
)(ˆ Yf
)(ˆ Yf
![Page 12: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/12.jpg)
8. The procedure
![Page 13: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/13.jpg)
9. The model output
• Focus on Cloud condensation nuclei (CCN) concentration
• Soluble aerosol (>50nm) form cloud droplets at a given supersaturation
• Uncertainty in CCN concentration due to: • uncertainty in the emission,
• nucleation of new aerosol,
• growth to 50nm,
• solubility,
• loss of aerosol.
![Page 14: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/14.jpg)
10. Expert elicitation
Elicitation: Ask the experts
‘We think these are the uncertain parameters and their values are very unlikely to fall outside of these ranges’
![Page 15: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/15.jpg)
11. Statistical design
– Need maximum information in fewest runs
– Space-filling in 28-dimensions
– Maximin Latin Hypercube used
–good marginal coverage
–good space-filling properties
– Number of runs 168 + 84 validation
– Emulator validation to highlight design issues
![Page 16: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/16.jpg)
• Interpolate ‘well-spaced’ model runs to estimate at untried points
• Gaussian Process - conditional probability • Non-parametric
– Key assumption is that parameter settings give information about model behaviour close by in parameter space
12. Filling in the gaps - emulation
GP mean True Function
X1
X2
Linear Model
![Page 17: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/17.jpg)
13. Filling in the gaps - emulation
Emulator mean
X2
X1
Emulated output distribution
Marginal functions X1 X2
Ou
tpu
t
Input
![Page 18: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/18.jpg)
14. Emulator validation
Is the emulator output a good approximate of the model
output?
YES – use the emulator mean instead
![Page 19: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/19.jpg)
15. Emulator validation I
![Page 20: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/20.jpg)
16. Emulator validation II
![Page 21: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/21.jpg)
17. Estimated CCN and its uncertainty concentration in every surface grid box
January
July
![Page 22: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/22.jpg)
18. Variance-based sensitivity analysis
ji
pij
i
i VVVYVar 12
1
)(
)),|(( iXi XYEVarVi
))|(( ijXij XYEVarVij
)(YVar
VS i
i
112
1
ji
pij
i
i SSS
Main effect sensitivity:
Variance decomposition:
Main effects +
Interactions:
Variance due to each parameter:
![Page 23: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/23.jpg)
Marginal functions
X1 X2
Input
Ou
tpu
t
Emulated output distribution
19. Variance-based sensitivity analysis
)),|(( iXi XYEVarVi
)(YVar
![Page 24: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/24.jpg)
24
20. Contributions to uncertainty in every grid box – January CCN
![Page 25: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/25.jpg)
25
20. Contributions to uncertainty in every grid box – January CCN
ACT_DIAM
AIT_WIDTH
ANTH_SOA
SO2O3_CLEAN
DRYDEP_ACC
BB_EMS
PRIM_SO4_DIAM
BB_DIAM
DMS_FLUX
![Page 26: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/26.jpg)
Surface parameter sensitivities 21. Contributions to uncertainty in every grid box
January January July July
σCCN/µCCN σCCN
![Page 27: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/27.jpg)
22. Summarising global maps
![Page 28: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/28.jpg)
23. Seasonal parameter sensitivities
![Page 29: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/29.jpg)
24. Multi-parameter analysis
![Page 30: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/30.jpg)
24. Multi-parameter analysis
BIO_SOA
BL_NUC
![Page 31: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/31.jpg)
25. Towards structural uncertainty and calibration
• Structural uncertainty
• Considering the model uncertainty is any model configuration near to observations
![Page 32: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/32.jpg)
26. AEROCOM versus AEROS
ACT_DIAM BIO_SOA ANTH_SOA
![Page 33: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/33.jpg)
27. Reduce global problem
Jan CCN
Jul CCN
• PCA and cluster analysis Similar patterns seen in both
![Page 34: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/34.jpg)
28. Cluster analysis
Fossil fuel emissions
Model parameters – Structure and cloud processing
ACT_DIAM
AIT_WIDTH
SO2O3_CLEAN
BB_EMS
BB_DIAM
![Page 35: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/35.jpg)
29. Summary
• Sensitivity analysis provides deeper understanding of model behaviour
• Quantifying parametric uncertainty can help identify structural errors/model errors/modeller errors
• Can help direct research areas/observation strategies
• Will help calibration
![Page 36: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/36.jpg)
30. Calibration discussion points
• Calibration of global means not satisfactory • How do we calibrate the global model with known
structural errors? • Do we expect one set of calibrated parameters given we
know model behaviour is variable? • How can we use the sensitivity information?
– to reduce dimensions • regional and temporal information • identify active parameters
• How do we specify discrepancy? – Use AEROCOM – Use grid box variability
![Page 37: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/37.jpg)
References
• Lee, L. A., Carslaw, K. S., Pringle, K. J., Mann, G. W., and Spracklen, D. V.: Emulation of a complex global aerosol model to quantify sensitivity to uncertain parameters, Atmos. Chem. Phys., 11, 12253-12273, doi:10.5194/acp-11-12253-2011, 2011.
• Lee, L. A., Carslaw, K. S., Pringle, K. J., and Mann, G. W.: Mapping the uncertainty in global CCN using emulation, Atmos. Chem. Phys., 12, 9739-9751, doi:10.5194/acp-12-9739-2012, 2012.
• Lee, L. A., Pringle, K. J., Reddington, C. L., Mann, G. W., Stier, P., Spracklen, D. V., Pierce, J. R., and Carslaw, K. S.: The magnitude and causes of uncertainty in global model simulations of cloud condensation nuclei, Atmos. Chem. Phys. Discuss., 13, 6295-6378, doi:10.5194/acpd-13-6295-2013, 2013.
• Mann, GW; Carslaw, KS; Spracklen, DV; Ridley, DA; Manktelow, PT; Chipperfield, MP; Pickering, SJ; Johnson, CE (2010) Description and evaluation of GLOMAP-mode: a modal global aerosol microphysics model for the UKCA composition-climate model, GEOSCI MODEL DEV, 3, pp.519-551. doi:10.5194/gmd-3-519-2010
• Partridge, D. G., Vrugt, J. A., Tunved, P., Ekman, A. M. L., Struthers, H., and Sorooshian, A.: Inverse modelling of cloud-aerosol interactions – Part 2: Sensitivity tests on liquid phase clouds using a Markov chain Monte Carlo based simulation approach, Atmos. Chem. Phys., 12, 2823-2847, doi:10.5194/acp-12-2823-2012, 2012.
• Roustant, O., Ginsbourger, D., and Deville, Y. (2010). DiceKriging: Kriging methods for computer experiments. R package version 1.1. http://CRAN.R-project.org/package=DiceKriging
• Bastos, L. S. and O'Hagan, A. (2009). Diagnostics for Gaussian process emulators. Technometrics 51, 425-438.
• Pujol, G. (2008). sensitivity: Sensitivity Analysis. R package version 1.4-0. • Saltelli, A., Chan, K., Scott, M. Editors, 2000, Sensitivity Analysis, John Wiley & Sons publishers, Probability
and Statistics series.
![Page 38: Sensitivity analysis and calibration of a global aerosol model · 2020. 2. 27. · School of Earth and Environment Sensitivity analysis and calibration of a global aerosol model Lindsay](https://reader035.vdocuments.mx/reader035/viewer/2022071408/610094c3754422635d4f266f/html5/thumbnails/38.jpg)
38