A Sideways Lookat ASSESSING THE ACCURACY ANDVALUEOF WEATHER FORECASTS

The greatest disservice inflicted on appliedmeteorology, at least since 1911 (see Weather,August 1989 p.358) has been the successiveclaims by spokesmen from the Met. Office thatforecasts are 85 per cent, or thereabouts, accu­rate. Add to this the compulsion of Civil Servicemeteorology to avoid at all cost admitting that aforecast was wrong, in case it somehow under­mines the authority of the State, and you end upwith a disturbingly large section of the public nolonger believing weather forecasters.

No wonder weather forecasters are so oftenthe subject of ridicule. It all wouldn't matter ex­cept that this collective reaction against forecast­ers' claims of accuracy results in hundreds ofmillionsof pounds worth of needless losses to thenational economy each year.

These losses occur wherever adverseweather conditions cause damage or make oper­ations less efficient in every kind of business, gov­ernment, energy or transport operation as well asin people's day to day lives, All of us frequentlymake weather sensitive decisions. We get the vastmajority of them right - and it is this fact that iscrucial to the understanding of the value of fore­cast accuracy as opposed to its scientific validity,The user is not interested in scientific validity,only in whether an assessment of future weatherprospects enables a better decision to be made,Onee this starting point is accepted a useful,rather than a purely scientific, evaluation canbegin,

85 per cent of the weather forecaster's pre­dictions may be correct but if 75 per cent of theuser's critical decisions relate to those 15per centof occasions when the forecaster gets it wrong theforecasts are perceived to be disastrous, What theforecaster forgets is that users themselves havequite a high score level in terms of 'getting itright', They know their local weather and theyknow how it impacts on their operations.

We come back to the continuing debate be­tween geographers and physicists. The physicistsees the problem as one of objective verifiablespot check accuracy. The geographer sees it as ahuman problem of decision-making in a situationwhere there are a high number of varying levelsof uncertainty. These include both the degree ofchangeability and the amount of local variation ofthe weather elements as well as in terms of howthey will interact with the users' operations.

A good example of a high objective ac­curacy Met. Office service has been 'Open Road'provided to highways authorities for their wintermaintenance operations, The Birmingham Uni­versity based Thermal Mapping International hashad impressive successes in making this servicespecific to stretches of road. Yet in spite of all

428

this, highwaysauthorities are stilla long way fromoptimising savings that could be made if appliedmeteorologists with what I call the 'geographer'sapproach' could work closely with small groupsof neighbouring authorities,

The purpose of such a liaison should be tomonitor data from ice sensors, combine this withall other available local road-weather data, andthen add to this a clear understanding of all thefactors involved in the user's decision-makingprocess - lead times, vehicle deployment, routelengths, overtime payments, residual salt levels,local run-off problems, etc,

Once some approximate cost: benefit ratioshave been established then the role of the appliedmeteorologist shifts from being a producer offorecasts to being an assessor of probabilities ofcritical conditions, These can then be relatedback to these cost: benefit ratios to achieve thehighest possible overall savings, The user's per­ception changes from worrying about forecast ac­curacy to looking at how the appliedmeteorologist can be used to optimum advan­tage, i.e. the problem becomes one of deter­mining at what point of diminishing returns itbecomes too costly to be confident of savingsgreater than the cost of using applied mete­orological skills.

People expect to receive weather forecastsfree, but they expect to pay a lot for expert pro­fessional advice. Customers are not interested inthe accuracy of the solution of an atmosphericphysics equation, Clear decisions emerge fromrisk assessments provided by people who knownot only how to access and interpret weatherforecast products, but who can also understandand take into account the customers operationalparameters.

The meteorological profession now needs toask itself whether it would not by now be a lotlarger and a lot wealthier if it had spent more ofits time concentrating on value rather than ac­curacy, But what is 'value'? How are the skillsofthe philosopher, physicist, the geographer andthe economist to be brought together to createthis very valuable and rare entity of the appliedmeteorologist?

This could become an even greater chal­lenge to our profession than that of the increasingrefinement to the computer models which haveso greatly improved the accuracy of many routineforecasts. What is now being called for are newways of assessing weather services in relation totheir utility, both in quantitative economic terms,and in qualitative terms of user confidence andsatisfaction.

ROLAND CHAPLAIN

Weather Watchers