hourly precipitation prediction

Hourly Precipitation Prediction

Nithya

Agenda• Model• Variable Selection

• Correlation• Time Lagged Correlations• Simulation Result• Variable selection Vs Mean Error

• Input -Neighbor Stations Selections• Stations which can be used as good predictors• Cross Correlation Between Stations for Hourly Precipitation• Correlation Plots

• Results • Error Plot• Hourly System output

Model

Data Collection Train Network Predict Data Web Interface

Train Network

Normalize Data..

∑❑

∑❑

∑❑

∑❑∑❑

Neural Network Training Section Cubic Spline Interpolation Multiple Sections

Online Prediction and Web Section Trained Neural Network Aggregate Data∑❑

Variable Selection-Correlation

• Output• 4 hours of Probability data for

Hourly Precipitationhttp://10.39.8.247/predict/correlation.php#div-1

All VariablesTemperatureHumidityDew pointPressureWind SpeedWind Speed GustWind DirectionWind Direction DegreeDaily RainHourly Precipitation

Selected VariablesHourly PrecipitationHumidityWind DirectionDaily RainSolar RadiationTemperature

Cross Correlation Between VariablesHP Vs Humidity HP Vs Pressure HP Vs Solar Radiation

HP Vs Temperature HP Vs Wind Direction Degree HP Vs Wind Speed

*HP- Hourly Precipitation

Variable Selection-Cross Correlation

HP Vs Humidity HP Vs Pressure HP Vs Solar Radiation

HP Vs Temperature HP Vs Wind Direction Degree HP Vs Wind Speed

Selected VariablesHourly PrecipitationHumidityWind DirectionDaily RainSolar RadiationTemperature

All VariablesTemperatureHumidityDew pointPressureWind SpeedWind Speed GustWind DirectionWind Direction DegreeDaily RainHourly Precipitation

Selected VariablesHourly PrecipitationHumidityDaily RainWind Direction

CorrelationCross Correlation

Simulation –Variable Combination

HP-WD-DR-T HP-P-WD-H-DR HP-H-T T-HP-SR-P-WS-H-DR HP-P-WD HP-P-H-DR HP-SR-P-H0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

Mean Absolute Error

Mea

n Ab

solu

te E

rror

1 2 3 4 5 6 7 80

0.002

0.004

0.006

0.008

0.01

0.012

0.014

Mean Absolute Error

Number of Variables usedM

ean

Abso

lute

Err

or

• List of Selected Variables Combinations• HP-WD-H-DR• HP-WD-DR-T• HP-WD-H-DR-T• HP-P-WD-WS-DR• HP-P-WD-H-T

• Legend• T- Temperature• HP- Hourly Precipitation• WD- Wind Direction• WS- Wind Speed

• P- Pressure• H- Humidity• DR- Daily Rain• SR- Solar Radiation

• HP-SR-WD-DR• HP-P-WD-DR• HP-P-WD-WS-H• HP-P-WD-T• HP-WD-WS-H-T

Variable Selection-Simulation

HP-WD-DR-T HP-WD-WS-T HP-P-WS-DR HP-SR-WD-WS HP-WS-DR-T HP-DR HP-SR-P-WD-H-T HP-SR-WD HP-P-T0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

Mean Absolute Error

Mea

n Ab

solu

te E

rror

1 2 3 4 5 6 7 80

0.0020.0040.0060.008

0.010.0120.014

Mean Absolute Error

Number of Variables used

Mea

n Ab

solu

te E

rror

Selected VariablesHourly PrecipitationHumidityWind DirectionDaily RainSolar RadiationTemperature

All VariablesTemperatureHumidityDew pointPressureWind SpeedWind Speed GustWind DirectionWind Direction DegreeDaily RainHourly Precipitation

Selected VariablesHourly PrecipitationHumidityDaily RainWind Direction

Selected VariablesHourly PrecipitationHumidityDaily RainWind Direction

CorrelationCross CorrelationSimulation

Number of Hours of input data

0 5 10 15 20 25 300

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

Number of Input hours of data required

Number of hours

MAE

0 5 10 15 20 25 30 351

1.1

1.2

1.3

1.4

1.5

1.6

1.7

1.8

1.9

Number of Neurons

Number of Neurons

MAE

• Number of Hours input data- 24• Number of hidden Neurons=7

Stations which can be used as good predictors

-18 -17 -16 -14 -13 -12 -11 -10 -9 -8 -7 -6 -5 -4 -3 -2 -10

5

10

15

20

25

Number of Times Rainfall occured at stations before IBSURRE21

Burna10CWWKIbccoqui5Ibcpittm3Surre10Surre6

Number Hours Ahead

Coun

t

Possible Predictor Burna 10, Coqui5, Surre6

-7 -6 -5 -4 -3 -2 -10

5

10

15

20

25

Burna10CWWKIbccoqui5Ibcpittm3Surre10Surre6

Cross Correlation Between Stations for Hourly Precipitation

SURRE21-COQUI5 SURRE21-SURRE6 SURRE21-PITTM3

SURRE21-BURNA 10 SURRE21-SURRE10

Cross Correlation Between Stations for Hourly Precipitation- Daily Rain

SURRE21-Surre10 SURRE21-Surre6

SURRE21-COQUI5 SURRE21-BURNA10

Cross Correlation Between Stations for Hourly Precipitation- Humidity

SURRE21-BURNA10 SURRE21-CWMM SURRE21-CWWK

SURRE21-COQUI5 SURRE21-SURRE6 SURRE21-SURRE10

Input Selection –Station Combination

1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 6.50

0.02

0.04

0.06

0.08

0.1

0.12

0.14

MEAN Absolute Error

Number of Input Variables

MAE

Best CombinationSurre21,Burna 10, Ibccoqui5Surre21,Burna 10, PITM3Surre21,PITM3, Surre6

Sur21-NA10-IBC5 Sur21-NA10-IBC5-PIT3-CWWK Sur21-PIT3-CWWK Sur21-IBC5-PIT3-Sur6 Sur21-IBC5-Sur10-CWMM0

0.05

0.1

0.15

0.2

0.25

0.3

Combination Plot for Stations

Mea

n Ab

solu

te E

rror

Input Selection –Station Combination

Sur21-NA10-IBC5 Sur21-PIT3-Sur10-CWMM-CWWK Sur21-PIT3-Sur6-CWWK Sur21-NA10-IBC5-PIT3-Sur10Sur21-IBC5-PIT3-Sur10-CWMM Sur21-NA10-CWWK Sur21-NA10-IBC5-PIT3-CWMM0

0.05

0.1

0.15

0.2

0.25

0.3

Combination Plot for Stations

Mea

n Ab

solu

te E

rror

Input/output Representation• Input is 24 hour Normalized data from three stations and 3 variables

from the stations.• Output is 4 variable. Each variable indicating probability of rain for

every hour ahead.

Date Time Hour1 Hour2 Hour3 Hour4

… 0 0 0 0

…. 0 1 0 0

… 1 0 0 0

…. 0 0 0 1

• Sample Actual OutputData Used for Training• Rainfall>1(mm) ? Output=1 : Output =0

We Have 2^4= 16 different Combinations of possible Output

Distribution of Actual Output Classes2011-2013

Error PlotPr

edic

ted

Error Computed for Random Sample data 2011-2013*This data was not used for Training

Value 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Output Class 0000 0001 0010 0011 0100 0101 0110 0111 1000 1001 1010 1011 1100 1101 1110 1111

Pred

icte

d

actual actual

counts percentage

Hourly Precipitation Results

12:00 AM

1:00 AM

2:00 AM

3:00 AM

4:00 AM

5:00 AM

6:00 AM

7:00 AM

8:00 AM

9:00 AM

10:00 AM

11:00 AM

12:00 PM

1:00 PM

2:00 PM

3:00 PM

4:00 PM

5:00 PM

6:00 PM0

102030405060708090

100

0

0.5

1

1.5

2

2.5

3

Low Rain 1/7/2014

Prob

abili

ty(%

)

Prec

ipita

tion

(mm

)

2:00 AM

4:00 AM

6:00 AM

8:00 AM

10:00 AM

12:00 PM

2:00 PM

4:00 PM

6:00 PM

8:00 PM0

102030405060708090

100

00.511.522.533.54

Average Rain 1/2/2014

Prob

abili

ty(%

)

Prec

ipita

tion

(mm

)

4:00 PM

5:00 PM

6:00 PM

7:00 PM

8:00 PM

9:00 PM

10:00 PM

11:00 PM

12:00 AM

1:00 AM

2:00 AM

3:00 AM

4:00 AM

5:00 AM

6:00 AM

7:00 AM

8:00 AM

9:00 AM

8:00 PM0

20

40

60

80

100

012345678

Heavy Rain 1/11/2014

Prob

abili

ty(%

)

Prec

ipita

tion

(mm

)

6:00 AM

7:00 AM

8:00 AM

9:00 AM

10:00 AM

11:00 AM

12:00 PM

1:00 PM

2:00 PM

3:00 PM

4:00 PM

5:00 PM

6:00 PM

7:00 PM

8:00 PM

9:00 PM

10:00 PM

11:00 PM0

102030405060708090

100

0

0.5

1

1.5

2

2.5

3

Snow Fall 2/22/2014

Prob

abili

ty(%

)

Prec

ipita

tion

(mm

)

Hourly Precipitation – Low rain

12:00 AM

1:00 AM 2:00 AM 3:00 AM 4:00 AM 5:00 AM 6:00 AM 7:00 AM 8:00 AM 9:00 AM 10:00 AM

11:00 AM

12:00 PM 1:00 PM 2:00 PM 3:00 PM 4:00 PM 5:00 PM 6:00 PM0

10

20

30

40

50

60

70

80

90

100

0

0.5

1

1.5

2

2.5

3

Rainfall Prediction 1/7/2014

Predict 1 Predict 2 Predict 3 Predict 4 Predict 5 Predict 6 Preict 7 Predict 8 Precipitation

Prob

abili

ty(%

)

Prec

ipita

tion

(mm

)

Hourly Precipitation – Average rain

2:00 AM

3:00 AM

4:00 AM

5:00 AM

6:00 AM

7:00 AM

8:00 AM

9:00 AM

10:00 AM

11:00 AM

12:00 PM

1:00 PM

2:00 PM

3:00 PM

4:00 PM

5:00 PM

6:00 PM

7:00 PM

8:00 PM

0

10

20

30

40

50

60

70

80

90

100

0

0.5

1

1.5

2

2.5

3

3.5

4

Rainfall Prediction 1/2/2014Predict 1 Predict 2 Predict 3 Predict 4 Predict 5Predict 6 Preict 7 Predict 8 Precipitation

Prob

abili

ty(%

)

Prec

ipita

tion

(mm

)

Hourly Precipitation – Heavy rain

4:00 PM 5:00 PM 6:00 PM 7:00 PM 8:00 PM 9:00 PM 10:00 PM 11:00 PM 12:00 AM 1:00 AM 2:00 AM 3:00 AM 4:00 AM 5:00 AM 6:00 AM 7:00 AM 8:00 AM 9:00 AM 8:00 PM0

20

40

60

80

100

0

1

2

3

4

5

6

7

8

Rainfall Prediction 1/11/2014

Predict 1 Predict 2 Predict 3 Predict 4 Predict 5 Predict 6 Preict 7 Predict 8 Precipitation

Prob

abili

ty(%

)

Prec

ipita

tion

(mm

)

Hourly Precipitation – Snow Fall

6:00 AM

7:00 AM

8:00 AM

9:00 AM

10:00 AM

11:00 AM

12:00 PM

1:00 PM

2:00 PM

3:00 PM

4:00 PM

5:00 PM

6:00 PM

7:00 PM

8:00 PM

9:00 PM

10:00 PM

11:00 PM

0

10

20

30

40

50

60

70

80

90

100

0

0.5

1

1.5

2

2.5

3

Rainfall Prediction 2/22/2014Predict 1 Predict 2 Predict 3 Predict 4 Predict 5Predict 6 Preict 7 Predict 8 Precipitation

Prob

abili

ty(%

)

Prec

ipita

tion

(mm

)

Cross Correlation Plots

Hourly Precipitation Vs Humidity6-23-13

5-11-131-3-13

Hourly Precipitation Vs Pressure6-23-13

5-11-131-3-13

Hourly Precipitation Vs Solar Radiation

6-23-13

5-25-131-3-13

Hourly Precipitation Vs Temperature6-23-13

5-25-131-3-13

Hourly Precipitation Vs Wind Direction

6-23-13

5-25-131-3-13

Back time lagged correlation

Hourly Precipitation Vs Wind Direction

Back time lagged correlationNorth NNE NE

ENEEast ESE SE

SSESouth

SSW SWWSW

WestWNW NW

NNW0

50

100

150

200

250

300

350

Direction Vs Degress

Degr

ee

2013- 6 Hours Prior to Rainfall- Wind Direction Distribution

Correlation Matrix1. SURRE 21

2. CWWK

*Everything Variable with a number suffix belongs to corresponding station listed Back To Station Correlation>>

Correlation Matrix1. SURRE 21

2. SURRE6

*Everything Variable with a number suffix belongs to corresponding station listed Back To Station Correlation>>

Correlation Matrix1. SURRE 21

2. CWMM

*Everything Variable with a number suffix belongs to corresponding station listed Back To Station Correlation>>

Correlation Matrix1. SURRE 21

2. SURRE10

*Everything Variable with a number suffix belongs to corresponding station listed Back To Station Correlation>>

Correlation Matrix1. SURRE 21

2. PITM3

*Daily Rain and Wind Direction Deg are dropped, because more 80% of the data is NA Back To Station Correlation>>

Correlation Matrix1. SURRE 21

2. IBCCOQUI5

*Daily Rain and Wind Direction Deg are dropped, because more 80% of the data is NA Back To Station Correlation>>

Correlation Matrix1. SURRE 21

2. BURNA10

*Daily Rain and Wind Direction Deg are dropped, because more 80% of the data is NA Back To Station Correlation>>