quantitative precipitation estimation at the climate corporation

© 2016 The Climate Corporation All Rights Reserved

QPE at the Climate [email protected] www.vlakshman.com

Talk at U. Washington, Jan 28, 2016

1

mailto:[email protected]

http://www.vlakshman.com

mailto:[email protected]


The Climate Corporation (TCC) provides decision-making tools for farmers

Weather is both a feature that we provide to our users, and an input to our agronomic models

http://www.climate.com/


24h rainfall estimates are critical to many farming decisions and advisors

3

Crop health/damage Field workability

If and when to fertilize


Agronomic models are sensitive to the accuracy of the rainfall estimates

4

lbs/acre of fertilizer

MAE of QPE (mm)


Our team in 2015

5

Scientists, engineers, statisticians, and data specialists work together to curate comprehensive data sets and develop scalable, production-ready algorithms.


Our team in 2015

6

Scientists, engineers, statisticians, and data specialists work together to curate comprehensive data sets and develop scalable, production-ready algorithms.

RainingChamp


Our QPE pipeline has uncertainty estimation, but is otherwise similar to, say, AHPS

7


Drop Size Distribution (DSD) variability induces uncertainty in radar QPE.

8

R(A)

R(Z)

R(Kdp)

Fig 1 of Ryzhkov 2014 variance from 47k DSDs

Fall speed vs drop size

Bayesian sampling of DSD and rain rate from dual-pol radar observations.

Probabilistic QPE

Latent DSD

Source: TCC Patent GENERATING PROBABILISTIC ESTIMATES OF RAINFALL RATES FROM RADAR REFLECTIVITY MEASUREMENTS by Kleeman, Lakshmanan, Reid

http://journals.ametsoc.org/doi/abs/10.1175/JTECH-D-13-00038.1


We optimize every step of the pipeline, and often end up creating new/hybrid approaches

9

Quantified differences between:● QPE methods● mosaicing approaches● gauge fusion methods● other “minor” parts of pipeline

As we implemented and measured known approaches, we also ended up implementing new/hybrid methods that outperformed what’s known in the literature


This is an example of one such comparison, on the use cases that matter to our customers

10


There is of course a computational constraint to be able to do this quickly and routinely

http://goo.gl/NB5hDO


So, we do almost everything on the cloud

12

https://cloud.google.com/

https://aws.amazon.com/

Elastic Compute Cloud (EC2)

Simple Storage Service (S3)


The NOAA CRADA puts NEXRAD data on S3, thus making our science faster and better

Simple Storage Service (S3)

1. LaunchElastic Compute Cloud (EC2)

Amazon public bucket

2. Compute


There is a limit to what you can do with radar

About 20% of our users live more than 150km away from a NEXRAD, so the solution will also involve deploying weather stations in an optimal manner

14


We instrumented Climate Research Farms to help explain spatial variability of rainfall

15


These are truly independent gauges, so we also used them to evaluate QPE performance

16

small data!

quantitative precipitation estimation at the climate corporation

Science