verification of quantitative precipitation forecast (qpf

221

15

Verification of Quantitative Precipitation Forecast

(QPF) 2019 during SW Monsoon over

River Sub-basins of India

B. P. Yadav, Ashok Kumar Das,Charu Nanda and Jyotsana Dhingra

Hydromet Division, India Meteorological Department, New Delhi

Performance of the Quantitative precipitation forecasts issued by India

Meteorological Department during the southwest Monsoon season to Central Water

Commission for generation and issuance of Flood warnings is discussed in this chapter, in

terms of various skill scores.

15.1 Introduction

Flood is a regular feature in India. Every year floods occurs in one or another part

of the country which causes huge losses to both life and property. In India, India

Meteorological Department (IMD) and Central Water commission (CWC) carry out the

work of Flood Forecasting jointly. IMD is the nodal agency for issuing Quantitative

Precipitation Forecast (QPF) for river Basins/ sub-Basins where as CWC is the nodal

agency for issuing Flood Forecast. The QPF is the main input in the Flood Forecasting

models for issuing flood forecast by CWC. IMD through its field offices called “Flood

Meteorological Offices (FMOs)” issues QPF on operational basis during flood season.

There are 13 FMOs located at different parts of flood prone areas of the country, which

are located at Agra, Ahmedabad, Asansol, Bhubaneswar, Guwahati, Hyderabad,

Jalpaiguri, Lucknow, New Delhi, Patna, Srinagar, Chennai and Bengaluru that caters to

the river basins mentioned in Table 15.1. IMD also provides similar support to Damodar

Valley Corporation (DVC) for the river Basins of Barakar and Damodar. The location and

area of jurisdiction of 13 FMOs and DVC Kolkata are shown inFig.15.1.

222

Figure 15.1: Flood Meteorological offices

The main River basin under 13 FMOs and DVC Kolkata along with the area under

their jurisdictions and the number of sub-basins for which the QPFs were issued on

operational basis are given in Table –15.1.

Table –15.1: Main River Basins/Sub-Basins under FMOs/DVC with Jurisdiction area

S. No.

FMOs

Area (Km

2)

No of Sub- Basins

Main Basins/Sub-Basins

1 Agra 2,92,492 8 Chambal, Betwa, Ken, Yamuna

2 Ahmedabad 2,20,946 19

Narmada, Tapi, Daman Ganga, Sabarmati, Banas, Mahi

3 Asansol 23,669 3 Ajoy, Mayurakshi, Kangsabati

4

Bhubaneswar

2,44,670

9

Subarnarekha,Baitarni,Burhabalang, Vamsadhara,Brahmani,Mahanadi,Rushikulya

5 DVC, Kolkata 21,013 3 Damodar

6

Guwahati

1,82,195

20

Brahmaputra, Barak, Dehung, Lohit, Buridihing, Subansiri, N. Dhansiri, S. Dhansiri, Jiabharali, Kapili, Manas/ Beki,Sankosh

7 Hyderabad 6,11,056 16 Godavari, Manjira, Wainganga,Penganga, Wardha, Indravati, Sabari

8 Jalpaiguri 16,151 5 Teesta, Jaldhaka, Raidak

9

Lucknow

2,20,465

14

Ghaghra, Rapti, Ramganga, Gomti, Sai, Sahibi, Chhatang, Bhagirathi, Alaknanda, Ganga, Sharda

10 New Delhi 36,670 3 Yamuna upto Mathura, Sahibi

223

11 Patna 1,71,698 8 Kosi, Mahananda, Adhwara, Bagmati, Gandak, Punpun, Sone, Kanhar, North Koel

12 Srinagar 4,788 8 Jhelum

13

Bengaluru

3,05,049

26

Upper Cauvery, Middle Cauvery, Lower Cauvery, Hemavathi, Kabini, Harangi,Periyar, Upper Vaigai, Lower Vaigai, Upper Bhima, Upper Krishna, Middle Krishna, Lower Bhima, Upper Tungabhadra, Ghataprabha, Bennehalla, Hagari or Vedavati,MiddleTungabhadra, LowerTungabhadra, Achankoil, Meenachil, Pamba, Bharathapuzha, Chalakudi, Upper Periyar, Lower Periyar

14

Chennai

6,05,708

11

Gummanur, Upper South Pennar, Korttalaiyar, Vellar, Lower South Pennar, Kunderu, Sagileru, Upper Pennar,Lower Pennar, PapagniCheyyeru

Total 29,56,570 153

Flood Meteorological Service of IMD provides through their FMOs/DVC, daily QPF

bulletin and Hydromet Bulletin to Central Water Commission (CWC) for operational flood

forecasting. QPF bulletin is issued at 0930hrs IST (0400 UTC) and Hydromet Bulletin at

1230 hrs IST (0700 UTC). QPF is issued for a lead-time of 7-days (forecast for 3 days

and outlook for subsequent 4 days) daily during flood season. If situation demands, QPF

bulletins are further updated in the evening.

In addition to flood season, QPF Bulletins consists of sub-basin wise QPFs and

heavy rainfall warning is issued by concerned FMOs/DVC during cyclone period or when

there is a chance of heavy rainfall that may lead toflood.

Hydromet Bulletin contains the following information.

Prevailing Synopticsituations,

Spatial and temporal distribution ofrainfall,

Categorical Sub-basin wise QPF for Day-1, Day-2, andDay-3.

Heavy rainfall warnings for Day-1, Day-2, andDay-3.

Sub-basin wise past 24 hour realizedrainfall.

Station wise observed significant rainfall (≥ 5cm).

The technical controls of FMOs are lying with Hydromet Division at HQ whereas the

administrative controls are lying with their respective RMCs. The performance of QPF is

verified for the flood season annually. In this chapter we will discuss the verification

results of operational sub-basin wise QPF for the SW monsoon season 2019.

15.2 Data Used

QPF is issued as an areal precipitation forecast sub-basin wise by the FMOs/DVC daily

in the flood season in the followingcategories.

i. 0 (norain)

224

ii. 0.1 – 10mm

iii. 11 –25mm

iv. 26 – 50 mm

v. 51 – 100mm

vi. > 100mm

The sub-basin wise QPF are verified with the observed areal rainfall. The sub-basin

observed areal rainfall has been computed by averaging the observed rainfall of stations

in thearea.The all India no. of QPF issued for Day-1, Day-2 and Day-3 by FMOs during

the SW monsoon season 2019 is 18541for each day for total 153 flood prone river sub-

basins.

15.3 Methodology

For all the precipitation categories mentioned in section 15.2 above, 6 X 6 contingency

table for observed and forecast precipitation is prepared as follows:

Table- 15.2: 6 X 6 Contingency

Observed category (mm)

Forecast Precipitation category (mm)

0 0.1-10 11-25 26-50 51-100 >100 Total

0 a B C d e f A

0.1-10 G H I j k l B

11-25 M N O p q r C

26-50 S T U v w x D

51-100 Y Z Aa ab ac ad E

>100 Ae Af Ag ah ai aj F

Total G H I J K L T

The performance of categorical QPF issued for different river sub-basins is verified

from 6X6 contingency table. The QPF issued for different river basins be verified by

computing Percentage Correct, Heidke Skill Score(HSS) and Critical Success Index (CSI),

from 6X6 Contingency table which are asfollows;

PC=(a+h+o+v+ac+aj)/TX100

CSI=a/(A+G-a), h/(B+H-h), o/(C+I-o), v/(D+J-v), ac/(E+K-ac), aj/(F+L-aj)

HSS=((T(a+h+o+v+ac+aj)-(AG+BH+CI+DJ+EK+FL))/T)/((T*T-

(AG+BH+CI+DJ+EK+FL))/T)

225

The POD, FAR, MR, CSI, BIAS, PC, TSS and HSS for each category be computed by

reducing the above 6X6 contingency table into 2X2 contingency table for YES/NO

forecast.

Table 15.3: 2 X 2 Contingency table

Observed Forecast

Yes No

Yes A B

No C D

Probability of detection(POD)=(A/(A+B))

False Alarm Rate (FAR)=C/(C+A)

Missing Rate(MR)=B/(B+A)

Correct Non-Occurrence (C-NON)=D/(C+D)

Critical Success Index (CSI)==Threat Score=A/(A+B+C)

Bias for occurrence (BIAS)=(A+C)/(A+B)

Percentage Correct(PC)=(A+D)/(A+B+C+D) X100=Hit Rate X 100

True Skill Score (TSS)=A/(A+B)+D/(C+D)-1

Heidke skill score (HSS)=2{(AD-

BC)/(B*B+C*C+2AD+(B+C)(A+D))}

FOR BEST/PERFECT FORECAST, POD=1, FAR=0, MR=0

The final skill score is the average of this verification over all forecasting offices. In

addition to percentage of correct forecast within the same category, the percentages of

QPFs that are within ±1 category and out by two or more categories are computed.

During SW monsoon season 2019, the skill scores for operational sub-basin wise QPFs

are computed for all FMOs/DVC for day-1, day-2 andday-3.

15.4 Results andDiscussions

The QPF verification statistics for different FMOs/DVC for Day-1, Day-2 and Day-3

forecast are computed and given in the subsequent sections.

15.4.1 Verification of Day-1QPF

The QPF verification statistics for different FMOs, and DVC for Day-1 are given in

Table –15.4. The all India percentage correct QPF within the same category is 58.1%

and within ±1 category is 94.6%. The highest percentage correct QPF was for FMO DVC

which was 80.9% and lowest was for FMO Bengaluru at 42.1% as seen from Fig.-.15.2.

The percentage correct forecast within ±1 category for all the forecasting offices is more

than 85% and it is 100% for DVC.

226

Figure – 15.2 : Percentage correct forecast by different FMOs.

Table –15.4: Performance of Day-1 QPF for the Flood Season 2019

FMO/MC

Total

No. of

QPF

issue

d

Correct

Foreca

st

Out by one

Stage

Correct and

±1

Out by two

Stage

Correc

t (%)

Usable

Foreca

st

Correct

& ±1

Stage

Ove

r

fct.

Unde

r fct.

Ove

r

fct.

Unde

r fct.

Agra 976 620 237 90 947 22 7 63.5% 97.0%

Ahmedabad 2318 1429 615 149 2193 98 16 61.6% 94.6%

Asansol 366 284 60 22 366 0 0 77.6% 100.0%

Bengaluru 3169 1333 117

7 272 2782 308 32 42.1% 87.8%

Bhubanesw

ar 1098 630 298 118 1046 28 20 57.4% 95.3%

Chennai 1342 622 551 94 1267 60 8 46.3% 94.4%

DVC 732 592 104 36 732 0 0 80.9% 100.0%

Guwahati 2440 1767 465 118 2350 51 29 72.4% 96.3%

Hyderabad 1952 1141 513 230 1884 33 32 58.5% 96.5%

Jalpaiguri 610 394 115 64 573 26 8 64.6% 93.9%

Lucknow 1708 911 493 222 1626 51 28 53.3% 95.2%

New Delhi 366 241 90 31 362 2 2 65.8% 98.9%

Patna 976 530 349 60 939 30 6 54.3% 96.2%

Srinagar 488 279 133 68 480 3 5 57.2% 98.4%

All Indiafct. 18541 10773 520

0 1574 17547 712 193 58.1% 94.6%

227

The all India skill scores viz, POD, FAR, MR, CSI, BIAS, HR, TSS and HSS have

been computed from 2X2 contingency Table and are given in Table -15.5. The false alarm

rate is lowest as 0.27 (no rain category). It increases for higher rainfall categories. The

false alarm rate and missing ratewas for highest>100 mm rainfall, which was 0.74 and

0.83respectively.

Table –15.5: All India Skill Scores of Day-1 QPF (2019)

SKILL SCORE 0 0.1-10 11-25 26-50 51-100 >100

Probability of Detection (POD): 0.38 0.73 0.54 0.39 0.32 0.17

False Alarm Rate (FAR): 0.27 0.30 0.63 0.65 0.59 0.74

Missing Rate (MR): 0.62 0.27 0.46 0.61 0.68 0.83

Correct Non-Occurrence (C-NON): 0.96 0.62 0.83 0.96 0.99 1.00

Critical Success Index (CSI): 0.35 0.56 0.29 0.22 0.21 0.10

Bias for Occurrence (BIAS): 0.47 1.05 1.56 1.17 0.82 0.59

Hit Rate: 0.84 0.68 0.79 0.93 0.98 1.00

Percentage of Correct (PC): 0.84 0.68 0.79 0.93 0.98 1.00

True Skill Score (TSS): 0.34 0.34 0.37 0.34 0.31 0.17

Heidke Skill Score (HSS): 0.40 0.34 0.31 0.31 0.31 0.15

It can also be seen that CSI is more for 0.1-10 category which is 0.56 and for

higher ranges, it decreases. Therefore, it reveals that the success rate for higher

categories of rainfall forecast is low.

Figure - 3: % Correct Forecast within Category of Day-1 QPF

228

15.5 Verification of Day-2QPF

The QPF verification statistics for different FMOs and DVC for Day-2 are computed

and presentedinTable–15.6. The all India percentage correct QPF with in category is 55%

and within±1 category is 93.7%. The highest percentage correct QPF issued by FMO

DVC is 69.7% and lowest is 42.1% by FMO Bengaluru. The percentage correct forecast

within ±1 category for all the forecasting offices is more than 85%.

Table–15.6: Performance of QPF (Day-2) updated for the Monsoon Season 2019

FMO/MC

Total

No. of

QPF

issue

d

Correct

Forecast

Out by one

Stage

Correct

and ±1

Out by two

Stage

Corre

ct (%)

Usable

Foreca

st

Correc

t & ±1

Stage

Over

fct.

Under

fct.

Over

fct.

Under

fct.

Agra 976 579 246 108 933 25 15 59.3% 95.6%

Ahmedabad 2318 1315 479 365 2159 56 82 56.7% 93.1%

Asansol 366 232 90 40 362 3 1 63.4% 98.9%

Bengaluru 3169 1334 1104 307 2745 311 57 42.1% 86.6%

Bhubanesw

ar 1098 627 270 148 1045 17 32 57.1% 95.2%

Chennai 1342 650 447 176 1273 50 14 48.4% 94.9%

DVC 732 510 142 77 729 1 2 69.7% 99.6%

Guwahati 2440 1551 453 324 2328 36 63 63.6% 95.4%

Hyderabad 1952 1105 445 291 1841 47 53 56.6% 94.3%

Jalpaiguri 610 353 136 86 575 20 13 57.9% 94.3%

Lucknow 1708 887 472 243 1602 64 38 51.9% 93.8%

New Delhi 366 211 116 33 360 5 1 57.7% 98.4%

Patna 976 559 318 64 941 26 5 57.3% 96.4%

Srinagar 488 285 124 69 478 3 7 58.4% 98.0%

All Indiafct. 18541 10198 4842 2331 17371 664 383 55.0% 93.7%

The all India skill scores viz, POD, FAR, MR, CSI, BIAS, PC, TSS and HSS have

been computed from 2X2 contingency and are given in Table–15.7. The all India false

alarm rate is lowestas0.34 (for 0.1-10 mm category) and increases for higher rainfall

categories. The highest false alarm rate is for 51-100 rainfall category and highest

missing rate is for the category >100mm.


SKILL SCORE 0 0.1-10 11-25 26-50 51-100 >100



Missing Rate (MR): 0.71 0.28 0.59 0.78 0.86 0.96


229



Hit Rate: 0.83 0.64 0.77 0.93 0.98 1.00




The category wise CSI is plotted in Fig.–15.4. It can also be seen that CSI is more

for 0.1-10 category which is 0.53 and higher ranges, it decreases. Therefore, it reveals

that the success rate for higher categories of rainfall forecast islow.

Figure–15.4: Critical Success Index for Day-2

15.6 Verification of Day-3 QPF

The QPF verification statistics for different FMOs and DVC for Day-3 are given in

Table -15.8. It can be seen from this Table that the all India percentage correct QPF within

category is 55.1% and within ±1 category is 92.8%. The highest percentage correct QPF

issued by FMO DVC is 70.6% and lowest is 43.2% by FMO Bengaluru. The all India

percentage correct forecast within±1 category for all the forecasting offices is 92.8%.

Table–15.8: Performance of Day-3 QPF for the Flood Season 2019

FMO/MC

Total

No. of

QPF

issue

d

Correct

Forecas

t

Out by one

Stage Correc

t

and ±1

Out by two

Stage

Correc

t (%)

Usable

Forecas

t

Correct

& ±1

Stage

Ove

r fct.

Unde

r fct.

Ove

r fct.

Unde

r fct.

Agra 976 542 251 126 919 34 20 55.5% 94.2%

230

Ahmedabad 2318 1293 353 450 2096 53 135 55.8% 90.4%

Asansol 366 238 82 44 364 1 1 65.0% 99.5%

Bengaluru 3169 1368 994 366 2728 311 90 43.2% 86.1%

Bhubaneswa

r 1098 625 261 139 1025 20 46 56.9% 93.4%

Chennai 1342 649 460 161 1270 40 24 48.4% 94.6%

DVC 732 517 134 76 727 1 4 70.6% 99.3%

Guwahati 2440 1542 419 356 2317 51 58 63.2% 95.0%

Hyderabad 1952 1130 352 335 1817 31 89 57.9% 93.1%

Jalpaiguri 610 332 132 107 571 20 16 54.4% 93.6%

Lucknow 1708 896 437 259 1592 68 43 52.5% 93.2%

New Delhi 366 214 110 28 352 11 3 58.5% 96.2%

Patna 976 565 317 62 944 21 8 57.9% 96.7%

Srinagar 488 309 103 65 477 4 7 63.3% 97.7%

All Indiafct. 18541 10220 4405 2574 17199 666 544 55.1% 92.8%

The all India skill scores viz, POD, FAR, MR, CSI, BIAS, PC, TSS and HSS have

been computed from 2X2 contingency Table and are given in Table-9. The all India false

alarm rate is lowest as0.35 (for 0.1-10 mm category) and it is higher at higher rainfall

categories. The highest false alarm rate is for 26-50mm rainfall categories and highest

missing rate is for >100 category. However, it may be mentioned that number of forecasts

issued for higher rainfall categories of QPF are rare because there is less no. of heavy

rainfall cases.


SKILL SCORE 0 0.1-10 11-25 26-50 51-100 >100



Missing Rate (MR): 0.71 0.26 0.62 0.85 0.89 0.99




Hit Rate: 0.83 0.64 0.78 0.93 0.98 1.00




The category wise CSI is plotted in Fig–15.5. It can also be seen that CSI is more

231

for 0.1-10 mm category which is 0.53 and higher ranges, it decreases. Therefore, it

reveals that the success rate for higher categories of rainfall forecast is given blow.

Figure–15.5: Critical Success Index for Day-3 QPF

15.7 Analysis of Heavy rainfall events

Generally, the flood occurs when the basin receives heavy rainfall and it also

depends on antecedent conditions of the soil. When the soil is fully saturated, then even

the small amount of rainfall may also lead to floods. While giving QPF forecast,

forecasters try to minimize both false alarm and missing cases; false alarm results into

unnecessary displacement and missing rate of such events results into unexpected

inundation. The accurate prediction of heavy rainfall cases (rainfall 26-50 mm and above

categories) are very important for the flood events. The statistics for total number of

under forecast cases in respect of rainfall categories 26-50, 51-100 and >100mm for

FMOs and DVC Kolkata are prepared. The analysis of QPF for all the above three

categories indicate that the total numbers of under/over forecast cases are 856 out of

1471for Day 1; 1090 out of 1474 for Day 2 and 1222 out of 1475 for Day 3 (Fig.-15.6).

Figure –15.6: All India observed and under/over forecast cases( >11-25 mm of

rainfall)

15.8 Performance of QPF in the recent years

232

The category-wise all India performance of Percentage Correct and CSIfor all India

sub-basins under different FMOs/DVC for Day-1, Day-2 and Day-3 are given in Fig.- 15.7

and Fig. –15.8. It is observed that there is an improvement in all Day-1, Day-2 and Day-

3forecast.

Figure –15.7: All India Day-1, Day-2 and Day-3 % correct forecast

Figure–15.8: All India Day-1, Day-2 and Day-3 % CSI

The category-wise all India performance of FAR and MR for all India sub-basins

under different FMOs/DVC for Day-1, Day-2 and Day-3 are given in Fig.–15. 9 and Fig. –

15.10. It is observed that there is slight decrease in both FAR and MR which indicates the

slight improvement in all Day-1, Day-2 and Day-3 forecast.

0.2

3

0.1

8

0.1

5

0.2

3

0.1

7

0.1

5

0.2

4

0.1

9

0.1

80.2

9

0.2

0

0.1

9

0.00

0.20

0.40

0.60

0.80

1.00

Day1 Day2 Day3

Cri

tica

l Su

cce

ss In

de

x

Rainfall Category (mm)

CSI : 2016-2019

2016 2017 2018 2019

233

Figure–15.9: All India Day-1, Day-2 and Day-3 % correct forecast

Figure–15.10: All India Day-1, Day-2 and Day-3 % correct

forecast

234

Fig.–15.11: The FMO-wise Percentage correct QPF for Day-1 are plotted for the year 2013 to 2019

235

The FMO-wise Percentage correct QPF for Day-1 are plotted for the year 2013 to

2019 (Fig.- 15.11). The accuracy of Day-1 QPF of FMOs namely, Agra, DVC, Guwahati,

Jalpaiguri, Lucknow, New Delhi and Patna have improved (5% or more) during the last years

while it has deteriorated in respect of FMOs namely, Ahmedabad, Asansol, Bengaluru,

Chennai, Hyderabad and Srinagar.

15.9 Conclusion :

i). The all India percent correct QPF within the same category is 58.1% for Day-1, 55%

for Day-2, and 55.1% for Day-3.

ii). However, accuracy of correct QPF within ±1 category is 94.6% for Day-1, 93.7% for

Day-2, and 92.8% for Day-3

iii). The accuracy of QPF is highest for FMO DVC (80.9%, 69.7%, 70.6%) and lowest by

FMO Bengaluru (42.1%, 42.1%, 43.2%) for Day-1, Day-2 and Day-3 forecasts,

respectively.

iv). All India Critical Success Index (CSI) for the rainfall categories 0.1-10, 11-25, 26-50,

51-100 and >100 mm varies from 0.56 to 0.1 for Day-1, 0.53 to 0.03 for Day-2, 0.53 to

0.01 for Day-3. CSI decreases as we move to the higher rainfall categories. Probability

of Detection (POD) and HSS also vary in similar manner.

v). All India False Alarm Rate (FAR) for the rainfall categories 0,0.1-10, 11-25, 26-50, 51-

100 and >100 mm varies from 0.27 to 0.74 for Day-1, 0.34 to 0.81 for Day-2, 0.35 to

0.77 for Day-3. FAR are generally increases as we move to the higher rainfall

categories.

vi). The accuracy of Day-1 QPF of FMOs namely, Agra, DVC, Guwahati, Jalpaiguri,

Lucknow, New Delhi and Patna have improved (5% or more) during the last years

while it has deteriorated in respect of FMOs namely, Ahmedabad, Asansol, Bengaluru,

Chennai, Hyderabad and Srinagar.

236

References:

1. „Verification of Quantitative Precipitation Forecast (QPF) 2015 over river basins of

India‟ by S. Kaur, S. K. Diwakar and A. K. Das, Report No. ESSO/IMD/HS/Basin

Hydrology/12/2017(1).

2. „Verification of Quantitative Precipitation Forecast (QPF) 2017 during SW Monsoon

over River Sub-basins of India‟ by B. P. Yadav and Ashok Kumar Das, SW Monsoon

Report 2017, Chapter -21.

3. „Verification of Quantitative Precipitation Forecast (QPF) 2018 during SW Monsoon

over River Sub-basins of India‟ by B. P. Yadav and Ashok Kumar Das, SW Monsoon

Report 2018, Chapter -17.

4. „Skill of operational forecast of heavy rainfall events during southwest monsoon

season over India‟ by B. P. Yadav, Naresh Kumar and L. S. Rathore, 66, 579- 584

5. „Verification of Quantitative Precipitation Forecast (QPF) 2014 over river basins of

India‟, IMD MET. Monograph: Synoptic Meteorology No.: ESSO/IMD/SYNOPTIC

MET/01-2014/15.

6. „Performance of IMD Multi-model Ensemble (MME) and WRF ARW based sub-basin

wise rainfall forecast for Mahanadi basin during Flood Season 2009, 2010‟ by Ashok

Kumar Das and Surinder Kaur, Mausam, 64, 4(Oct 2013), 625-644.

7. „Performance of IMD Multi-Model Ensemble and WRF (ARW) model for sub-basin

wise rainfall forecast during Monsoon 2012‟ by Ashok Kumar Das and Surinder Kaur

in Mausam 67, 2 (April 2016), 323-332.

--

verification of quantitative precipitation forecast (qpf

Documents