ci verification methodology & preliminary results [email protected]
TRANSCRIPT
![Page 2: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/2.jpg)
In short:
1. Find observed CI using radar echoes aloft
2. Compare to CI forecasts from UAH and UW
3. Find hits, misses, false alarms4. Preliminary results5. Discussion
![Page 3: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/3.jpg)
From radar data aloft
1. How observed CI was determined
![Page 4: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/4.jpg)
Observed CI
For verification purposes, need a “truth” field Independent of way in which CI is detected Not tied to “objects”
Based on multi-radar reflectivity at -10C isotherm Reflectivity aloft, associated with graupel
formation Good indication on convection Less contaminated by clutter, biological
echoes The multi-radar reflectivity is QC’ed, but QC is
not perfect
![Page 5: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/5.jpg)
Reflectivity at -10C on 4/4/2011 Approx. 1km resolution over CONUS
![Page 6: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/6.jpg)
Classifying CI
Define convection as: Reflectivity at -10C exceeds 35 dBZ
New convection: Was below 35 dBZ in previous image Images are 5 minutes apart
Done on a pixel-by-pixel basis But allow for growth of ongoing convection
![Page 7: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/7.jpg)
Model verification
The CI detection algorithm is now running realtime Being used to verify NSSL-WRF model
forecasts of CI
![Page 8: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/8.jpg)
Aside: model verification
Probability of CI in one hour very similar But time evolution different
![Page 9: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/9.jpg)
Real time: Image at t0
![Page 10: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/10.jpg)
Real time: Image at t1
![Page 11: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/11.jpg)
Real time: Observed CI
![Page 12: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/12.jpg)
Methodology
Take image at t0 and warp it to align it with the image at t1 Warping limited to a 5 pixel movement Determined by cross-correlation with a
smoothness constraint imposed on it 5 pixels in 5 min 60kmph maximum
movement Then, do a neighborhood search
Pixels above 35 dBZ with no pixel above 35 dBZ within 3km of aligned image is “New Convection”
![Page 13: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/13.jpg)
Example: Image at t0
![Page 14: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/14.jpg)
Example: Image at t1
![Page 15: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/15.jpg)
Example: Image at t0 aligned to t1
![Page 16: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/16.jpg)
Classification
![Page 17: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/17.jpg)
Definition of Observed CI
Computed CI using 4 different distance thresholds: 3 km (as described) 5 km 15 km 25 km
The 15 km threshold means that a new CI pixel would have to be at least 15 km from existing convection to considered new In the HWT, this is what forecasters tended to like What I will use for scoring
![Page 18: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/18.jpg)
Significant cells?
One possible problem is that even one pixel counts as CI So, also tried to look for at least 13 km^2 cells
This will be called ObservedCIv2 Tends to find only significant cells (or cells after
they have grown a little bit). Started doing this after some feedback on this
point Not available for all days Can go back and recompute, but doesn’t seem to
make much difference to final scores
![Page 19: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/19.jpg)
By finding distance between centroids
2. Comparing Observed to Forecast
![Page 20: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/20.jpg)
Computing distance
Take the ObservedCI, SatCast and UWCI grid points Find contiguous pixels and call it an object Find centroid of those objects
Use storm motion derived from radar echoes and model 500mb wind field
Compute distance between each ObservedCI centroid and each forecast CI centroid
![Page 21: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/21.jpg)
Distance computation
Distance is computed as follows: If observed CI is outside time window of
forecast CI (-15 to +45 min), then dist=MAXDIST
Project forecast CI to time of observed CI Using storm motion field
Compute Euclidean distance in lat-lon degrees
MAXDIST was set to be 100 km Pretty generous
![Page 22: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/22.jpg)
Two ways: Hungarian match and distance
3. Scoring
![Page 23: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/23.jpg)
Scoring: Hungarian Match
Create cost matrix of distance between each pair Observed CI to forecast CI
Find best association for each centroid to minimize global sum-of-distances
Any associated pair is a hit Any unassociated observed CI is a miss Any unassociated forecast CI is a false
alarm
![Page 24: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/24.jpg)
Scoring: Neighborhood Match Consider each observed CI
If there is any forecast CI within MAXDIST, then it is a hit
Otherwise, it is a miss Consider each forecast CI
If there is no observed CI within MAXDIST, then it is a miss
More generous than the Hungarian Match Since multiple forecasts can be verified by
a single observation
![Page 25: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/25.jpg)
Summary of numbers that matter Observed CI:
35 dBZ 5 pixel warp in 5 minutes 15 pixel isolation for new CI
Significant cells area threshold (ObservedCIv2) 13 km^2
Time Window: -15 min to +45 min
Distance threshold: Hits have to be within 100 km
![Page 26: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/26.jpg)
Real time images and daily scores
4. Preliminary results
![Page 27: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/27.jpg)
Real time
Can see ObservedCI, ObservedCIv2, UAH and UWCI algorithms at:
http://wdssii.nssl.noaa.gov/web/wdss2/products/radar/civer.shtml
![Page 28: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/28.jpg)
Example
![Page 29: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/29.jpg)
Verification dataset
Dataset of centroids over Spring experiment is available at:
ftp://ftp.nssl.noaa.gov/users/lakshman/civerification.tgz
Contains: All ObservedCI, SatCast and UWCI centroids ObservedCIv2 for when we started creating
them Results of matching and skill scores by day
![Page 30: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/30.jpg)
Example result for June 10, 2011 UAH
UWCI
These scores are typical
![Page 31: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/31.jpg)
Only significant cells (ObservedCIv2)
UAH
UWCI
![Page 32: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/32.jpg)
5. Discussion
![Page 33: CI VERIFICATION METHODOLOGY & PRELIMINARY RESULTS lakshman@ou.edu](https://reader036.vdocuments.mx/reader036/viewer/2022062309/5697c00e1a28abf838cc9fbe/html5/thumbnails/33.jpg)
Possible reason for low values Could be a factor of the cirrus mask
Computing scores without taking the mask into account is problematic Because mask is so widespread, most
radar-based CI happens under the mask