Regional Haze Modeling:Recent Modeling Results for

VISTAS and WRAP

October 27, 2003, CMAS Annual Meeting, RTP, NC

University of California, Riverside

Modeling Team Participants

• UC Riverside: Gail Tonnesen, Zion Wang, Chao-Jung Chien, Mohammad Omary, Bo Wang

• Ralph Morris et al., ENVIRON Corporation• Zac Adelman et al., Carolina Environmental

Program• Tom Tesche et al., Alpine Geophysics• Don Olerud, BAMS

Acknowledgments

• Western Regional Air Partnership: John Vimont, Mary Uhl, Kevin Briggs, Tom Moore,

• VISTAS: Pat Brewer, Jim Boylan, Shiela Holman

Topics

• Model Performance Evaluation

• WRAP 1996 Model Performance Evaluation

• VISTAS 2002 Sensitivity Results

• CMAQ Benchmarks

WRAP Modeling

• 1996 Annual Modeling

• 36 km grid for western US, 95x85x18 layers

• MM5 by Olerud et al.

WRAP Emissions Updates

• Corrections to point sources• MOBILE6 beta for WRAP states• Monthly corrections for NH3 based on EPA/ORD

inverse modeling.• Updated non-road model• Typical fires used for results shown here• 1996 NEI for non-WRAP states

WRAP - CMAQ revisions

• v0301, released in March 2001– Used as the base case and all sensitivity cases for

WRAP’s 309 simulations.

• v0602, released in June 2002• v4.2.2, released in March 2003• v4.3, released in Sept. 2003

Mean Normalized Bias (Yearly)

-200

0

200

400

600

800

1000

1200

1400

1600

1800

SO4 NO3 NH4 OC EC SOIL CM PM25 RCFM PM10 Bext

(%)

v0301 v422

v43

Comparisons based on IMPROVE evaluation

• How well does the model reproduces mean, modal, and variational characteristics ?– Using observations to normalize model error &

bias result in misleading conclusion:• if observation is very small large bias or error• if model under prediction bounded by -1• model over prediction is weighted more than under

prediction

• We used Mean Normalized Err & Bias in 309:– Poor metric for clean conditions

Model Performance Metrics

• Use fractional error and bias:– bias and error is bounded symmetrical limits of +2

• Normalized Mean Error & Bias:– Divide the sum of the errors by the sum of the

observations.

• Coefficient of determination (R2)– explains how much of the variability in the model

predictions can be explained by the fact that they are related to ambient observation, i.e. how close the points are to the observations.

Recommended Performance Metrics

Statistical measures used in model performance evaluation

Measure Mathematical Expression

Notation

Accuracy of unpaired peak (Au)Opeak = peak observation; Pu

peak= unpaired peak prediction within 2 grid cells of peak observation site

Accuracy of paired peak (Ap)P = paired in time and space peak prediction

Coefficient of determination

Pi = prediction at time and location i;

Oi =observation at time and location i;

=arithmetic average of Pi, i=1,2,…, N;

=arithmetic average of Oi, i=1,2,…,N

Normalized Mean Error (NME) Reported as %

Root Mean Square Error (RMSE)

Fractional Gross Error (FE)

peak

peakupeak

O

OP

peak

peak

O

OP

N

i

N

iii

N

iii

OOPP

OOPP

1 1

22

2

1

)()(

))((

N

ii

N

iii

O

OP

1

1

2

1

1

21

N

iii OP

N

N

i ii

ii

OP

OP

N 1

2

PO

Statistical measures used in model performance evaluation

Measure Mathematical Expression

Notation

Mean Absolute Gross Error (MAGE)

Mean Normalized Gross Error (MNGE); Mean Normalized Error (MNE)

Reported as %

Mean Bias (MB)

Mean Normalized Bias (MNB) Reported as %

Mean Fractionalized Bias (Fractional Bias, MFB)

Reported as %

Normalized Mean Bias (NMB) Reported as %

N

iii OP

N 1

1

N

i i

ii

O

OP

N 1

1

N

iii OP

N 1

1

N

i i

ii

O

OP

N 1

1

N

i ii

ii

OP

OP

N 1

2

N

ii

N

iii

O

OP

1

1

)(

Statistical measures used in model performance evaluation

• In addition…– Mean observation– Mean prediction– Standard deviation (SD) of observation– Standard deviation (SD) of prediction– Correlation variance

Expanded Model Evaluation Software to include…• Ambient data evaluation for air quality monitoring networks:

– IMPROVE (24-Hour average PM)– CASTNet (Weekly average PM & Gas)– STN (24-Hour average PM)– AQS (Hourly Gas)– NADP (weekly total deposition)– SEARCH

• 17 statistical measures in model performance evaluation• All performance metrics can be analyzed in an automated

process for model and data selected by:· allsite_daily · onesite_daily

· allsite_yearly · onesite_monthly

· allsite_monthly · onesite_yearly

• Facilitate model evaluation.

• Benefit from shared development of tool.

• Share monitoring data.

• UCR software available at website:

www.cert.ucr.edu/aqm

Community Model Evaluation Tool?

SO4, Monthly Statistical Measures

-50

0

50

100

150

200

250

1 2 3 4 5 6 7 8 9 10 11 12

Month

%

Normalized Mean Error (%)

Mean Normalized Gross Error (%)

Mean Normalized Bias (%)

Mean Fractionalized Bias (%)

Normalized Mean Bias (%)

WRAP 1996 Evaluation, CMAQ v4.3

NO3, Monthly Statistical Measures

-200

0

200

400

600

800

1000

1200

1 2 3 4 5 6 7 8 9 10 11 12

Month

%

Normalized Mean Error (%)

Mean Normalized Gross Error (%)

Mean Normalized Bias (%)

Mean Fractionalized Bias (%)

Normalized Mean Bias (%)

WRAP 1996 Evaluation, CMAQ v4.3

OC, Monthly Statistical Measures

-50

0

50

100

150

200

250

300

1 2 3 4 5 6 7 8 9 10 11 12

Month

%

Normalized Mean Error (%)

Mean Normalized Gross Error (%)

Mean Normalized Bias (%)

Mean Fractionalized Bias (%)

Normalized Mean Bias (%)

WRAP 1996 Evaluation, CMAQ v4.3

EC, Monthly Statistical Measures

-100

-50

0

50

100

150

200

250

300

1 2 3 4 5 6 7 8 9 10 11 12

Month

%

Normalized Mean Error (%)

Mean Normalized Gross Error (%)

Mean Normalized Bias (%)

Mean Fractionalized Bias (%)

Normalized Mean Bias (%)

WRAP 1996 Evaluation, CMAQ v4.3

WRAP 1996 cases in progress

• New fugitive dust emissions model

• New NH3 emissions model

• Actual Prescribed & Ag burning emissions

• 2002 annuals simulations being developed.

VISTAS Model 12 km Domain

• 34 L MM5 by Olerud

• 1999 NEI

• CMAQ v3

VISTAS Sensitivity Cases

• 3 Episodes: Jan 2002, July 1999, July 2001• Sensitivity Cases

– MM5 MRF and ETA-MY, – PBL height, Kz_min, Layer collapsing– CB4-2002– SAPRC99– CMAQ-AIM– GEO-CHEM for BC– NH3 emissions

VISTAS Key Findings

• NO3 over predictions in winter, under predictions in summer.– Thorton et al N2O5 had small benefit, July

MNB increased from –50% to –45%

• SO4 performance reasonably good• Problems with PBL height

– Kz_min = 1 improved performance– Investigating PBL height corrections

• Minor differences in 19 vs 34 layers

Benchmarks

• Athlon MP 2000 (1.66 GHz)

• Opteron 246 (2.0 GHz)– 32 bit code– 64 bit code

• Compare 1, 4 and 8 CPUs.

• Ported CMAQ to the 64 bit SuSE– Pointers & memory allocation for 64 bit

Test Case for benchmarks

• VISTAS 12 km domain– 168 x 177 x 19 layers

• Benchmarks for CMAQ 4.3• One day simulation, CB4, MEBI• Single CPU run time hour:minutes

– Athlon 2 GHz: 14:10– Opteron 32bit 2 GHz: 12:49– Opteron 64 bit 2 GHz: 10:57

Clock Time by CPU Type

0

200

400

600

800

1000

1200

1 2 3 4 5 6 7 8

number CPUs

Tim

e (m

inu

tes)

Athlon 1.66 GHzOpteron-32Opteron-64Athlon 2.13 GHz

Parallel Scaling by CPU type

0.125

0.150

0.175

0.200

0.225

0.250

0.275

0.300

0.325

2 4 6 8

number CPUs

Sc

ali

ng

Ra

tio

AthlonOpteron-32Opteron-64Perfect Scaling

Parallel Scaling by CPU Type

1.00

1.10

1.20

1.30

1.40

1.50

1.60

1.70

1.80

1 2 3 4 5 6 7 8

number CPUs

actual/perfect

Athlon MPOpteron-32Opteron-64

Optimal Cost Configuration

• Small cluster < 8 CPUs use Athlon

• Large cluster >16 CPUs use Opterons?

Conclusions

• Major Improvements in WRAP 1996 Model

• WRAP 2002 annual modeling underway

• VISTAS Sensitivity Studies– still have problems in NO3– Need better NH3 inventory– Need more attention to PBL heights in MM5

• Community model evaluation tool?