Just how accurate is your favourite weather forecast model?
- Thread starter GHA
- Start date
franksingleton
Well-known member
I repeat; accuracy is meaningless unless you define the term and measure it. Our memories are selective.
Operational numerical weather models, primarily the GFS, UK, ECMWF, JMA (global) UK4, UKV, AROME, WRF and HIRLAM (limited area) are built by a fairly small number of groups who are in close contact with each other and exchange information and ideas. There is little difference between models.
The first 4 are the leading global models. On occasion, two have been known to have a similar outcome and the other a different but mutually similar outcome. Similar outcomes between different models is not a useful guide to accuracy of a forecast.
The major modelling centres run ensembles of the same model with slightly varying data analyses. Every outcome is a possible solution. The distribution (between 20 anf 60) gives an idea of confidence. The Predictwind two model ensemble is meaningless. You have no idea as to whether both outcomes are in the wings of a large ensemble distribution, both near the middle or on opposite wings. The same is true for comparing forecasts from several different models. I suspect also that when sailors say different models the sometimes mean different presentations of the GFS.
There is no doubt that the WRF is as good a LAM as any. Run by the various “unofficial” bodies it will (or should) improve on the GFS for topographical forcing – funnelling through straits etc.
The weakness in all these forecasts is that I know of none that inputs weather data at high resolution. All start with global model data on a 25 km resolution at best. Therefore, none know about weather detail less than on a 100-km resolution. You cannot make a silk purse out of a sow’s ear even if it is a cultivated and refined sow.
Even when using high resolution data as national Met services do, fines scale forecasts are really only valid in a deterministic sense for limited times; a few hours in a fast-moving situation, longer in a slow moving one.
Operational numerical weather models, primarily the GFS, UK, ECMWF, JMA (global) UK4, UKV, AROME, WRF and HIRLAM (limited area) are built by a fairly small number of groups who are in close contact with each other and exchange information and ideas. There is little difference between models.
The first 4 are the leading global models. On occasion, two have been known to have a similar outcome and the other a different but mutually similar outcome. Similar outcomes between different models is not a useful guide to accuracy of a forecast.
The major modelling centres run ensembles of the same model with slightly varying data analyses. Every outcome is a possible solution. The distribution (between 20 anf 60) gives an idea of confidence. The Predictwind two model ensemble is meaningless. You have no idea as to whether both outcomes are in the wings of a large ensemble distribution, both near the middle or on opposite wings. The same is true for comparing forecasts from several different models. I suspect also that when sailors say different models the sometimes mean different presentations of the GFS.
There is no doubt that the WRF is as good a LAM as any. Run by the various “unofficial” bodies it will (or should) improve on the GFS for topographical forcing – funnelling through straits etc.
The weakness in all these forecasts is that I know of none that inputs weather data at high resolution. All start with global model data on a 25 km resolution at best. Therefore, none know about weather detail less than on a 100-km resolution. You cannot make a silk purse out of a sow’s ear even if it is a cultivated and refined sow.
Even when using high resolution data as national Met services do, fines scale forecasts are really only valid in a deterministic sense for limited times; a few hours in a fast-moving situation, longer in a slow moving one.
capnsensible
Well-known member
I can remember back as far as two to ten weeks ago. My memory is that up to about three or four days ahead, the wind and cloud forecast each period was almost spot on and the rain bit only failed a little on squalls. I'd give that a rating of 'accurate' .
:encouragement:
:encouragement:
franksingleton
Well-known member
One swallow does not make a summer.
capnsensible
Well-known member
Yeah, it was only a sixty day trip from Antigua through the Panama Canal to Tahiti via the Galapagos and Marqesas Islands. Guess they got lucky.
franksingleton
Well-known member
I have worked as a forecaster in the tropics. Now do a test in mid-latitudes where there are well and not so well defined frontal systems and convection is not the main driver. PredictWind once showed me how good was their prediction for Hawaii. I would have been surpraised had they not predicted wind well and the possibility of showers. Mid-latitude frontal systems are a very different matter.
capnsensible
Well-known member
Hope you didn't drip too much sweat on the chart or they may have got worried about a tsunami.
franksingleton
Well-known member
By #27 on the 17th ( a couple of days prior, in fact) we had identified a one day window of 21st or 22nd of E'lies. By early on the 18th it was still uncertain which but 21st seemed more likely. By evening, 21st was pretty certain. Confirmed today, 19th. After that W'lies are ex-ected but going back to E'lies around the 25th.
To be able to plan ahead in this way was impossible 20 or 30 years ago. Today, most sources of objective forecasts would probably have given the same result because most start with the GFS and use the GFS for boundary conditions. Whether the individual forecasts turn out to be "accurate" is not very relevant and difficult to define. To me, the important criterion is that they are (or will be shown to be) useful. I am not sure how to quantify that.
On the latest statistics that I have seen, the GFS outperforms all others. Whether more detailed forecasts (Windguru, Windfinder, Meteoconsult, PredictWind, They're, etc) add value is uncertain, topographic effects apart, I doubt that they do. What would be useful would be a carefully set up trial suitable for a college of FE or a 6th form college. If there are any takers, I would be more than happy to advise.
To be able to plan ahead in this way was impossible 20 or 30 years ago. Today, most sources of objective forecasts would probably have given the same result because most start with the GFS and use the GFS for boundary conditions. Whether the individual forecasts turn out to be "accurate" is not very relevant and difficult to define. To me, the important criterion is that they are (or will be shown to be) useful. I am not sure how to quantify that.
On the latest statistics that I have seen, the GFS outperforms all others. Whether more detailed forecasts (Windguru, Windfinder, Meteoconsult, PredictWind, They're, etc) add value is uncertain, topographic effects apart, I doubt that they do. What would be useful would be a carefully set up trial suitable for a college of FE or a 6th form college. If there are any takers, I would be more than happy to advise.
Last edited:
