© 2016 The Climate Corporation All Rights Reserved
QPE at the Climate [email protected] www.vlakshman.com
Talk at U. Washington, Jan 28, 2016
1
© 2016 The Climate Corporation All Rights Reserved
The Climate Corporation (TCC) provides decision-making tools for farmers
Weather is both a feature that we provide to our users, and an input to our agronomic models
http://www.climate.com/
© 2016 The Climate Corporation All Rights Reserved
24h rainfall estimates are critical to many farming decisions and advisors
3
Crop health/damage Field workability
If and when to fertilize
© 2016 The Climate Corporation All Rights Reserved
Agronomic models are sensitive to the accuracy of the rainfall estimates
4
lbs/acre of fertilizer
MAE of QPE (mm)
© 2016 The Climate Corporation All Rights Reserved
Our team in 2015
5
Scientists, engineers, statisticians, and data specialists work together to curate comprehensive data sets and develop scalable, production-ready algorithms.
© 2016 The Climate Corporation All Rights Reserved
Our team in 2015
6
Scientists, engineers, statisticians, and data specialists work together to curate comprehensive data sets and develop scalable, production-ready algorithms.
RainingChamp
© 2016 The Climate Corporation All Rights Reserved
Our QPE pipeline has uncertainty estimation, but is otherwise similar to, say, AHPS
7
© 2016 The Climate Corporation All Rights Reserved
Drop Size Distribution (DSD) variability induces uncertainty in radar QPE.
8
R(A)
R(Z)
R(Kdp)
Fig 1 of Ryzhkov 2014 variance from 47k DSDs
Fall speed vs drop size
Bayesian sampling of DSD and rain rate from dual-pol radar observations.
Probabilistic QPE
Latent DSD
Source: TCC Patent GENERATING PROBABILISTIC ESTIMATES OF RAINFALL RATES FROM RADAR REFLECTIVITY MEASUREMENTS by Kleeman, Lakshmanan, Reid
© 2016 The Climate Corporation All Rights Reserved
We optimize every step of the pipeline, and often end up creating new/hybrid approaches
9
Quantified differences between:● QPE methods● mosaicing approaches● gauge fusion methods● other “minor” parts of pipeline
As we implemented and measured known approaches, we also ended up implementing new/hybrid methods that outperformed what’s known in the literature
© 2016 The Climate Corporation All Rights Reserved
This is an example of one such comparison, on the use cases that matter to our customers
10
© 2016 The Climate Corporation All Rights Reserved
There is of course a computational constraint to be able to do this quickly and routinely
http://goo.gl/NB5hDO
© 2016 The Climate Corporation All Rights Reserved
So, we do almost everything on the cloud
12
https://cloud.google.com/
https://aws.amazon.com/
Elastic Compute Cloud (EC2)
Simple Storage Service (S3)
© 2016 The Climate Corporation All Rights Reserved
The NOAA CRADA puts NEXRAD data on S3, thus making our science faster and better
Simple Storage Service (S3)
1. LaunchElastic Compute Cloud (EC2)
Amazon public bucket
2. Compute
© 2016 The Climate Corporation All Rights Reserved
There is a limit to what you can do with radar
About 20% of our users live more than 150km away from a NEXRAD, so the solution will also involve deploying weather stations in an optimal manner
14
© 2016 The Climate Corporation All Rights Reserved
We instrumented Climate Research Farms to help explain spatial variability of rainfall
15