evaluation of 1-day models in california: status€¦ · status of 1-day experiment in california...
TRANSCRIPT
Evaluation of 1-day models in California: Status
Max Werner
With M. Herrmann, J. Zechar, M. Taroni, P. Maechling, W. Marzocchi, F. Silva, T. Jordan
Status of 1-day experiment in California
• Initiated in August 1, 2007 • 526 M3.95+ eqks since then including 2010 M7.2 El Mayor, 2014 M6
Napa and 2010 M6.5 Mendecino. • Currently 18 “models” under live test:
– Matlab-STEP retired in 2013 because of bugs & Matlab licensing – 1 Java-STEP from 2010 – 2 ETAS flavours by Zhuang – 4 ETAS flavours by Rhoades – 2 ETAS flavours by Gordon – 2 ETAS flavours by Helmstetter & Werner – 2 non-parametric (K3) model flavours by Helmstetter/Werner – 2 straight averages of (ETAS, K3) by Helmstetter/Werner – 2 hybrid models by Rhoades (JANUS, SE) – 1 smoothed seismicity model by Kagan/Jackson
• 2 dynamic Bayesian ensemble models (on demand)
Status of results
• August 2017 inventory revealed some issues: – some of the latest test results are missing. – some of the latest T/W test results are wrong. – The v16.10 release may be at fault. – Maintenance/reprocessing since October 2016 may be at fault.
• We are developing a strategy to verify, reprocess and complete the results database. – CSEP@SCEC’s workflow requires updating. – We need a strategy for verifying results.
• Are you working with 1-day forecasts? Let’s organise.
Results: California 1-day (v9.1)
419
Testing Region: California Target events: M ≥ 3.95 (419 total) Testing period: 09/2007 - 07/2015 Testing method: T-test
G = 3.5/eqk
Results: California 1-day (v10.10)
176
G = 1.5/eqk 205
G = 0.3/eqk
Testing Region: California Target events: M ≥ 3.95 (206 total) Testing period: 10/2010 - 07/2015 Testing method: T-test
Results: California 1-day (v12.10 Md3)
102
G = 17/eqk
Testing Region: California Target events: M ≥ 3.95 (102 total) Testing period: 10/2012 - 07/2015 Testing method: T-test
83
102
102
102
102
102
102
Results: California 1-day (v12.10 Md3)Testing Region: California Target events: M ≥ 3.95 (102 total) Testing period: 10/2012 - 07/2015 Testing method: T-test
102
83
102
102
102
102
102
102
Results: California 1-day (v12.10 Md 2)
102
102
102
8
6
102
Testing Region: California Target events: M ≥ 3.95 (102 total) Testing period: 10/2012 - 07/2015 Testing method: T-test
83
102
102
8
6
102
102
102
102
102
102
102
Bayesian ensemble
K3 Mean(K3,ETAS)
KJSS
ETASv1
K3
ETAS-PPE ETASv1.1
ETAS-DR
ETAS_HW Mean(K3,ETAS)
109
132
132
132
132
132
132
132
132
Results: California 1-day (v12.10)Testing Region: California Target events: M ≥ 3.95 (132 total) Testing period: 10/2012 - 08/2016 Testing method: T-test
Conclusions
• CSEP@SCEC needs a strategy for verifying, reprocessing and completing the results database
• Recent 1-day models are more informative than early models.
• Bayesian ensemble modelling is available on demand – Down-weighting correlated forecasts to calculate prior weights is
unnecessary overhead – performance is nearly identical. – “interesting” temporal fluctuations in posterior weights. – BMA is very sensitive and quickly dominated by single
component (see also Marzocchi et al. 2012)
