Tuesday, December 31, 2013

Meteorological Snake Oil Salesman: Ultra-Long Daily Forecasts

The overwhelming majority of meteorologists are upfront about their capabilities, not promising or suggesting that they can achieve more than the state-of-the-science.    But there is a growing group of forecasting entities that are promising daily or weekly forecasts for weeks, months or a year ahead of time, even though such forecasting is not possible for both theoretical and practical reasons.

They are embarrassments to my field and are the snake-oil salesmen of weather prediction.

Before I describe the various long-term forecasting services, let me note that groups that do rigorous verification find that weather prediction skill for day to day weather fades out around 7-9 days.   Theoretical studies suggest weather prediction (that is describing what is going to happen at some specific time) may not be possible past about 2 weeks.  The reason for this has to do with the chaotic nature of highly complex systems like the atmosphere.  We can never describe the atmosphere or its processes exactly and as a result forecast errors grow over time.   Eventually the errors swamp the forecast.  Although forecasting the exact weather at some point in time may not be possible more than a few weeks ahead, it may well be possible to produce useful forecasts of long-term averages over a region. The National Weather Service is trying to do this with their Climate Forecast System (CFS).

The first group to provide detailed forecasts up to a year ahead of time is the most harmless:  the Old Farmer's Almanac and the Farmer's Almanac (these are two distinct products).  The Old Farmer's Almanac will give you sub-weekly forecasts out about a year into the future.  Here is a sample:

Their forecast for the first week in January is clearly going to be completely in error (heavy rain is not going to happen to the southern NW coastal region).  This doesn't give you much confidence for their extended range predictions! My colleague Nick Bond, Washington State climatologist did a study of the accuracy of the Old Farmer's temperature and precipitation forecasts. The bottom line:  they are no more more accurate than flipping a fair coin.  Other researchers have found the same thing.

I don't think many people take almanac forecasts seriously, so I am not too concerned about it.

But what really bothers me is that several weather firms are seriously pushing long-term daily forecasts as real products.

For example, Accuweather, the well known weather forecasting firm, started to provide 45-day forecasts this summer (see graphic). 

You can view these 45-day forecasts for free on their web site or mobile app. Here are the daily  forecasts for the first two weeks of February for Seattle!  Not very daring forecasts:  high temperatures range from 43 to 45!   Their icons for clouds also look like fish...but that is another matter.


On their web site you will not find verifications, but it turns out some folks have done it for them. Jon Nese, an instructor at the well-known meteorology department at Penn. State, had his students verify the Accuweather forecasts over an extended period.  The results for temperature are summarized below.

A good way to tell whether you have any forecast skill is to compare your forecast errors to that of climatology...the average value for that day.  Accuweather does better than climatology for the first 9 days, but after ten days THEY ARE WORSE THAN CLIMATOLOGY!  In other words, their customers would be better off just using the climatological temperatures rather than trust the Accuweather forecast.   Any clams that Accuweather has long-term skill is uber snake oil salesmanship.   And a profound disservice to my profession.  Accuweather has a proud history as a national forecast firm:  why throw away their reputation in this way?


But as bad as Accuweather and the Old Farmer's Almanac are, they are not in the same league as a company that provides one-year DAILY forecasts: WeatherTrends360.com (see below)


Their products are inconsistent with our knowledge of how forecast errors grown in the atmosphere, but that does not stop them from pushing it on consumers and businesses.  To show you how problematic this forecast service is, let's check their predictions for the next month (see below).  Wow...on January 9th the high will be 22 and the low about 15F, with the cold wave starting next Tuesday.  I have checked reliable forecasting systems; nothing suggests this kind of extreme cold for Seattle. There will be cold air to the east of the coastal NW:  I suspect they are using low-resolution model output even for the short-term.


How about for the warm time of the year, late next July and August?  See below.  Only ONE DAY will get to 70F, making it perhaps the coldest August in Seattle history!  Pretty sad.


Folks such as WeatherTrends360.com and the rogue group at Accuweather are not only undermining their reputations but are diminishing the credibility of the weather forecasting profession.

Is providing forecasts you know to be inaccurate any different than selling magical elixirs that you know can't provide the promised cures?  I will let you decide.

No comments:

Post a Comment