Just how accurate is your favourite weather forecast model?
- Thread starter GHA
- Start date
franksingleton
Well-known member
Those with really long memories may remember Professor Joad of the BBC Brains T-rust in the 1940s. He would have started a reply by saying “It all depends on what you mean by accuracy.” And, of course, how you measure it. Strange though it may seem, it is difficult to find a measure that is meaningful to us as users and useful to the scientists developing numerical weather prediction models. Memories are selective so I always take subjective assessments with a large heap of salt.
RMSE of MSLP is a fundamental measure that tells the modellers whether or not their latest update has been beneficial or not. It tells us nothing. Using that measure, forecasts at 5 days are as good as at one day ahead 30 years ago.
Some subjective views of mine:-
1. The short period forecast of rainfall on the Met Office app is amazingly good. The app lets you compare forecast with outcome. That is one reason why I have been trying to get support for the output to include vector winds over UK waters – similar to http://www.met.ie/forecasts/short-range.asp
2. The Met Office forecasts as seen/heard on the BBC are remarkably good up to a week ahead. Particularly useful is when they say that the situation is uncertain.
3. I doubt that, on average, there is much difference between the major modelling groups – Met Office, ECMWF, US GFS, JMA. Differences on specific occasions are most likely to be due to differences in the analysis.
A few comments on some points made above:-
1. Computer models do not use past history or pattern matching as some still think. In the simplest terms, they use Newton’s third law – Force = mass x acceleration applied to a fluid.
2. YR.NO uses a LAM (limited area/meso-scale) model known as HIRLAM. As do Iceland, Sweden, Denmark, Finland, Estonia, Ireland, Netherlands, Spain.
3. The UK and France use their own LAMs.
4. WRF is a LAM used for development of LAMs in the USA. The software is available to anyone free. It is as good as any other LAM.
5. National Met services start their LAMs using the maximum amount of data that their computers can process . They start with an analysis on the same resolution as the models. For the UK and France that is under 2 km. HIRLAM countries may do the same or may (used to) use slightly larger resolution – 2+ km. To do this, they use satellite data and, in some cases, eg the UK, radar rainfall data.
6. “Non-official” modellers usually use the WRF.
7. None of these “Non-official” modellers, as far as I know, can use these high resolution data. They start with much coarser data, usually the ¼ degree (25 km) output from the GFS. Weather detail that they predict will be compromised by
lack of detailed weather input. I would not waste my money in paying for such services.
8. Small weather details have short lifetimes. Hence, LAMs run by National Met services are really only useful for short times ahead.
9. Available computer power means that LAM areas are small. The UK uses an outer area with a 4 km grid and a British Isles inner area with a 1.5 km grid. Any detail in the analysis soon gets lost. This is another limitation easily forgotten by providers and users of forecasts.
Too much I know but, as ever, there is so much in such a thread to comment on.
RMSE of MSLP is a fundamental measure that tells the modellers whether or not their latest update has been beneficial or not. It tells us nothing. Using that measure, forecasts at 5 days are as good as at one day ahead 30 years ago.
Some subjective views of mine:-
1. The short period forecast of rainfall on the Met Office app is amazingly good. The app lets you compare forecast with outcome. That is one reason why I have been trying to get support for the output to include vector winds over UK waters – similar to http://www.met.ie/forecasts/short-range.asp
2. The Met Office forecasts as seen/heard on the BBC are remarkably good up to a week ahead. Particularly useful is when they say that the situation is uncertain.
3. I doubt that, on average, there is much difference between the major modelling groups – Met Office, ECMWF, US GFS, JMA. Differences on specific occasions are most likely to be due to differences in the analysis.
A few comments on some points made above:-
1. Computer models do not use past history or pattern matching as some still think. In the simplest terms, they use Newton’s third law – Force = mass x acceleration applied to a fluid.
2. YR.NO uses a LAM (limited area/meso-scale) model known as HIRLAM. As do Iceland, Sweden, Denmark, Finland, Estonia, Ireland, Netherlands, Spain.
3. The UK and France use their own LAMs.
4. WRF is a LAM used for development of LAMs in the USA. The software is available to anyone free. It is as good as any other LAM.
5. National Met services start their LAMs using the maximum amount of data that their computers can process . They start with an analysis on the same resolution as the models. For the UK and France that is under 2 km. HIRLAM countries may do the same or may (used to) use slightly larger resolution – 2+ km. To do this, they use satellite data and, in some cases, eg the UK, radar rainfall data.
6. “Non-official” modellers usually use the WRF.
7. None of these “Non-official” modellers, as far as I know, can use these high resolution data. They start with much coarser data, usually the ¼ degree (25 km) output from the GFS. Weather detail that they predict will be compromised by
lack of detailed weather input. I would not waste my money in paying for such services.
8. Small weather details have short lifetimes. Hence, LAMs run by National Met services are really only useful for short times ahead.
9. Available computer power means that LAM areas are small. The UK uses an outer area with a 4 km grid and a British Isles inner area with a 1.5 km grid. Any detail in the analysis soon gets lost. This is another limitation easily forgotten by providers and users of forecasts.
Too much I know but, as ever, there is so much in such a thread to comment on.
Last edited:
Oliver Perkins
New member
I think that Windguru is good, but the gusts are always much stronger than they actually are.
KellysEye
Active member
>What I have found that every different forecast, such as the Met office, Windfinder and Windguru are diffferent. Accuracy of any them I estimate to be 60%.
>>Unless you pay then Windguru and Windfinder default at looking exactly at the same gfs model data. Good example of how bad we are at any kind of objective analysis when 2 different ways of presenting the same numbers seem different.
They do use the same model but their forecasts are not all the same. We found GFS the best for long distnce sailing but as said forecasts are only 60% accurate. Emsworth is a micro and the Met Office forecast can be totally wrong for days running, particularly about rain, we are sheltered by the Isle of Wight and the South Downs.
>>Unless you pay then Windguru and Windfinder default at looking exactly at the same gfs model data. Good example of how bad we are at any kind of objective analysis when 2 different ways of presenting the same numbers seem different.
They do use the same model but their forecasts are not all the same. We found GFS the best for long distnce sailing but as said forecasts are only 60% accurate. Emsworth is a micro and the Met Office forecast can be totally wrong for days running, particularly about rain, we are sheltered by the Isle of Wight and the South Downs.
franksingleton
Well-known member
As kellyaeye says, the GFS is as good as anything for sailing purposes. I never pay for any of the various automated forecasts. I have said many times why. You may get a nicer presentation or one delivered in a way that you want. You will not get a forecast consistently better than the GFS. Claims about greater accuracy are simply untenable.
My advice is always tu use the GFSfor planning purposes. Use it for day to day sailing but with an eye on GMDSS texts. No objective forecast should be relied upon for forecasts of strongest winds likely nor of poor visibility.
My advice is always tu use the GFSfor planning purposes. Use it for day to day sailing but with an eye on GMDSS texts. No objective forecast should be relied upon for forecasts of strongest winds likely nor of poor visibility.
GHA
Well-known member
Would agree with that, I pay for predict wind but mostly just because the interface is so good, don't really rate their 2 models. Gfs does consistently seem best within 24h for timing and force based on looking out the hatch and nearest actual data, anything more than that it's impossible for a mere mortal like I to quantify
I would suggest that anyone who has a preference a few days out is guessing, the data logging must be massive and beyond most of us.
But gfs and synoptics it is, often the synoptics and sat photos will show the fronts with the wind getting less vicious quicker as the front passes away than gfs (or the other models) might suggest.
franksingleton
Well-known member
Sometimes the weather, the general pattern at least, is predictable several days ahead. Last September we knew a week ahead with some certainty when we would cross the Channel. This month, we were not sure until the day before we crossed. I continually watch forecasts out to 10 days ahead. I look for consistency between successive forecasts. At present we are sitting in Lezardrieux wanting to head westwards. In our old age, we are rather cautious and do not want to bear or motor sail into strong winds. Currently, I am looking at 22nd as a one day window. Time will tell.
I do despair when I see Kellyseye, who I respect as a sailor, stating that forecasts are 60% accurate. Without defining accuracy and saying how it is measured, such statements are meaningless. The Met Office can tell you how well they predict surface pressure and give you RMSE values. They could probably tell you how well they predict average wind over a sea area. I doubt that they can tell you how well they predict wind at a specific location. You have only to sail a short distance over even the open sea, never mind inshore to see just how much wind varies in space and time.
I do despair when I see Kellyseye, who I respect as a sailor, stating that forecasts are 60% accurate. Without defining accuracy and saying how it is measured, such statements are meaningless. The Met Office can tell you how well they predict surface pressure and give you RMSE values. They could probably tell you how well they predict average wind over a sea area. I doubt that they can tell you how well they predict wind at a specific location. You have only to sail a short distance over even the open sea, never mind inshore to see just how much wind varies in space and time.
duncan99210
Well-known member
I use PocketGrib as my 'go to' forecast source. It's a simple to use grib file viewer which does not overlay any local interpretation on the basic forecast model. I use the GFS dataset, mainly because it's the default setting and I've never seen enough variation between it and the other models available in the app to be worth switching.
I've been using it in the Med and UK for the past few years and have found it to be good enough for my purposes with only one occasion when it failed to predict a significant blow. I suspect that this failure was more down to local geography than an overall failure of the model to predict high winds from a particular direction: that's the price you pay for using a global model with no local overlay.
I've been using it in the Med and UK for the past few years and have found it to be good enough for my purposes with only one occasion when it failed to predict a significant blow. I suspect that this failure was more down to local geography than an overall failure of the model to predict high winds from a particular direction: that's the price you pay for using a global model with no local overlay.
franksingleton
Well-known member
I agree about the minimal, if any , benefit in selecting other than the GFS. If I could get the UK global model then, maybe. Or the UK high resolution model. Even then, I doubt it.
With the various GRIB apps - http://weather.mailasail.com/Franks-Weather/Grib-Files-Apps, the only difference is primarily in the presentation. A minor drawback with the ones that I have used is that you cannot display more than 7 days ahead. I like to look on a 10 day rolling basis. I can do that on my laptop using zyGrib - another GFSproduct.
On the iPad I send an email to query@saildocs.com. The message is
send gfs:N45.8,N50.5,W6.5,E1.5|0.25,0.25|0,3..240|
The reply email will open in any of the GRIB apps but the only one that I have found that will show all 10 days is the iNavX chart program.
That message will give a forecast at 3 hourly intervals, to 230 hours at 0.25 degree spacing, of wind and isobars.
As you might guess, we are Weather bound into Lezardrieux with steady rain just now.
With the various GRIB apps - http://weather.mailasail.com/Franks-Weather/Grib-Files-Apps, the only difference is primarily in the presentation. A minor drawback with the ones that I have used is that you cannot display more than 7 days ahead. I like to look on a 10 day rolling basis. I can do that on my laptop using zyGrib - another GFSproduct.
On the iPad I send an email to query@saildocs.com. The message is
send gfs:N45.8,N50.5,W6.5,E1.5|0.25,0.25|0,3..240|
The reply email will open in any of the GRIB apps but the only one that I have found that will show all 10 days is the iNavX chart program.
That message will give a forecast at 3 hourly intervals, to 230 hours at 0.25 degree spacing, of wind and isobars.
As you might guess, we are Weather bound into Lezardrieux with steady rain just now.
GHA
Well-known member
Another one to add is opencpn with the weatherfax & grib plugins. You can overlay grib, satellite and synoptic images all on the same screen..
bats
Member
Ref message: send gfs:N45.8,N50.5,W6.5,E1.5|0.25,0.25|0,3..240|
With the output from this message OpenCPN shows 3 hourly intervals out to 8 days and then 6 hourly. You can create/send a message through OpenCPN using the Grib plugin.
Mike
With the output from this message OpenCPN shows 3 hourly intervals out to 8 days and then 6 hourly. You can create/send a message through OpenCPN using the Grib plugin.
Mike
franksingleton
Well-known member
Thank you both for reminding me about CPN. I do have it bot not used it for several years. I was, however, talking about tablets. Does anyone know of tablet apps other than iNavX that will show GRIBs beyond 7 days?
PS. Answered my own question. iGrib lets you select up to 10 days. It will also read the Saildocs message. It only provides for data at 0.5 degree intervals, good enough for planning purposes. It was all we had in the early GRIB era.
PS. Answered my own question. iGrib lets you select up to 10 days. It will also read the Saildocs message. It only provides for data at 0.5 degree intervals, good enough for planning purposes. It was all we had in the early GRIB era.
Last edited:
GHA
Well-known member
Opencpn on android has the grib plugin which can download, 0.5deg resolution by the looks of it.
franksingleton
Well-known member
An update on #27. It looks like 21st or 22nd for a one day window. The GFS and Meteo France both say 21st. ECMWF says 22nd'.
GHA
Well-known member
What do you use to log forecasts? Paper pad or some very clever forecasters method?
I've used evernote in the past to record screen shots of forecasts then compare with actuals later, it's a load of work though to get into any kind of detail.
Nowadays it's easier not to have a planning calendar which goes more than a day ahead
franksingleton
Well-known member
What do you use to log forecasts? Paper pad or some very clever forecasters method?
I've used evernote in the past to record screen shots of forecasts then compare with actuals later, it's a load of work though to get into any kind of detail.
Nowadays it's easier not to have a planning calendar which goes more than a day ahead
On the laptop, zyGrib or Saildocs files are automatically saved. Using Pocketgrib you can define for how long a file is retained. Using Saildocs on the iPad, I can save the emails with the Saildocs files. Those that I open with iNavX are saved automatically. Post hoc, I was able to produce these from zyGrib
This shows 9 forecasts all verifying at the same time as an example of consistency between successive forecasts. This level of consistency is a good indicator of the forecast being reliable. Inconsistency between successive forecasts is a good indicator of uncertainty in the forecast. I regard this constant reviewing of forecasts as essential to our cruising.
I describe this in Reeds Weather Handbook and I practice what I preach.
M33
New member
Personally I use Windguru pro and have found it extremely accurate over the last couple of years. It gives results for several weather models and you can make a judgement on how strong the prediction is based on how they agree. If they don't agree then its a fair assumption the weather is going to be changeable and noone is really sure.
I have a Manson Supreme a Fortress and a Bruce by the way, oh and a small Delta in the garage...
How about radios.....
I have a Manson Supreme a Fortress and a Bruce by the way, oh and a small Delta in the garage...
How about radios.....
ex-Gladys
Well-known member
Personally I use Windguru pro and have found it extremely accurate over the last couple of years. It gives results for several weather models and you can make a judgement on how strong the prediction is based on how they agree. If they don't agree then its a fair assumption the weather is going to be changeable and noone is really sure.
I have a Manson Supreme a Fortress and a Bruce by the way, oh and a small Delta in the garage...
How about radios.....
+1. The accuracy of the WRF 9km model on the East Coast is very good.
Ludd
New member
What do you use to log forecasts? Paper pad or some very clever forecasters method?
I've used evernote in the past to record screen shots of forecasts then compare with actuals later, it's a load of work though to get into any kind of detail.
Nowadays it's easier not to have a planning calendar which goes more than a day ahead
Cast your mind back to March! The very comprehensive wind forecast and trip time projection you found (don't know what you were using!) said I'd have to motor to maintain 4kt average, Portimao to Las Palmas..First day out ,I motored for twelve hours. After that---- suffice to say , my average speed was over six knots! 'Twere a tad breezy!
capnsensible
Well-known member
Recently used Predict Wind Offshore. Got it via an Iridium modem. Linked up on laptop with Adrena first router. Dogs doodads.
Forecast very accurate, updated twice a day.
Recommend for distance sailors. :encouragement:
Forecast very accurate, updated twice a day.
Recommend for distance sailors. :encouragement:
