Has somebody broken the Met Office weather model?

[franksingleton]

A note on gust forecasts.
As far as I am aware, forecast gust speeds are derived from the predicted gradient winds using algorithms that take into account the stability of the atmosphere and the depth of the atmosphere through which there can be convection. With shallow convection mixing of the air may only be through a depth of about 1 km. with very vigorous0 convection mixing might be through a depth of 4 km. In such cases gusts might well be double the gradient wind.

 

[dunedin]

A note on gust forecasts.
As far as I am aware, forecast gust speeds are derived from the predicted gradient winds using algorithms that take into account the stability of the atmosphere and the depth of the atmosphere through which there can be convection. With shallow convection mixing of the air may only be through a depth of about 1 km. with very vigorous0 convection mixing might be through a depth of 4 km. In such cases gusts might well be double the gradient wind.

I understand the general point on gusts (and have been using sailing forecasts for the past 30 years or so, and intensively most days throughout the summer for the past 10 years).
But that doesn’t explain the three points raised in my post #77 - particularly why there is always such a big difference in the gust speeds quoted for UKHO models in the web page vs the Predict Wind version of the two UKMO models, both ostensibly from the same source but the web page consistently much higher.
And why the UKMO gust forecast has been consistently higher than all the other models - and generally simply less accurate than the other models.
It has “cried wolf” so much that now tends to get ignored by me.

 

[wwarw00]

I understand the general point on gusts (and have been using sailing forecasts for the past 30 years or so, and intensively most days throughout the summer for the past 10 years).
But that doesn’t explain the three points raised in my post #77 - particularly why there is always such a big difference in the gust speeds quoted for UKHO models in the web page vs the Predict Wind version of the two UKMO models, both ostensibly from the same source but the web page consistently much higher.
And why the UKMO gust forecast has been consistently higher than all the other models - and generally simply less accurate than the other models.
It has “cried wolf” so much that now tends to get ignored by me.

Yep - fully agree. Would never go sailing if I took the current Met Office gust forecasts seriously. They need to explain what they have changed recently. If we have to get used to a new "forecast versus what actually happens" that's fine - but they should be warning us that some sort of reset has happened as their baseline has changed really significantly.
Aidan

 

[Snowgoose-1]

I understand the general point on gusts (and have been using sailing forecasts for the past 30 years or so, and intensively most days throughout the summer for the past 10 years).
But that doesn’t explain the three points raised in my post #77 - particularly why there is always such a big difference in the gust speeds quoted for UKHO models in the web page vs the Predict Wind version of the two UKMO models, both ostensibly from the same source but the web page consistently much higher.
And why the UKMO gust forecast has been consistently higher than all the other models - and generally simply less accurate than the other models.
It has “cried wolf” so much that now tends to get ignored by me

I agree about the gust forecasting.
I no longer follow the Met Office wind forecasts.
It would be interesting though, to plot the actual against the predicted forecasting , over a period of say three summer months.

 

[franksingleton]

I do not know what PW does with gust speeds. You would have to ask them. I have a deep distrust of PW who make a number of incorrect statements about models.
As regards the general question, our memories are selective. It would need a comprehensive statistical study to verify claims made. I offer one explanation. In recent months we have had some severe convective storms. Gust forecasts could have been stronger than usual. Many gusts will go unrecorded.
However, that is speculation. I will write to the Met Office enquiry office and ask them if there has been any change in gust forecasting. Of course, there is no reason why anybody should not write. I would encourage you to do so. Obviously, they verify forecasts as a matter of course but they might find some feed back to be useful.

 

[franksingleton]

I have sent this to the Met Office.

“I have been seeing comments from sailors, some in the Clyde area, that Met Office forecasts of gusts have seemed abnormally strong recently.

One comment is that the Met Office website shows stronger gust speeds than are apparent using output from UKV on, for example, the PredictWind site.

Has there been any modification to algorithms for predicting gust speeds? Are you aware of differences between what appears on the website and UKV output?”

I will await their reply with interest.

 

[Snowgoose-1]

I have sent this to the Met Office.

“I have been seeing comments from sailors, some in the Clyde area, that Met Office forecasts of gusts have seemed abnormally strong recently.

One comment is that the Met Office website shows stronger gust speeds than are apparent using output from UKV on, for example, the PredictWind site.

Has there been any modification to algorithms for predicting gust speeds? Are you aware of differences between what appears on the website and UKV output?”

I will await their reply with interest.

Excellent. Thanks.

 

[John_Silver]

Their admin HQ is in France but when I was trying to find what model(s) they use, I got a reply with a Czech or some othe East European address. They would not say which model they use on their web service. However, the only free service of forecasts out to 15 days is the GFS. Does anyone else think that the GFS is “spot on”.

There’s a ‘compare models’ tab, on the Meteo Consult app. Don’t know if the comparator models go into the ‘secret sauce?’
Screenshots below list the models. To my untrained eye, there’s no simple relationship between the MC prediction and any/all of the comparators:

 

[franksingleton]

There’s a ‘compare models’ tab, on the Meteo Consult app. Don’t know if the comparator models go into the ‘secret sauce?’
Screenshots below list the models. To my untrained eye, there’s no simple relationship between the MC prediction and any/all of the comparators:
View attachment 183622
View attachment 183623

So, when you say MeteoConsult is “spot on”, which MeteoConsult?

I know that I am not good at navigating complex web pages, but I cannot find the Caomparsyor pages.

However, I have to ask, why pay for information that is freely available?
If I want detailed forecasts for any country, my first stop is the national weather service. If I am sailing, my first immediate stop is the GMDSS forecasts produced by the natuional service. If I want to see NWP output, I have access to ECMWT and various national services. I can see detailed output from models run by national services.
Why should I pay for a tarted up version?

To the best of my knowledge, no private entity can access all the data used to run a NWP model. No private Met service can develop a NWP model. I really doubt the understanding of models by any private weather firm. Maybe, I really mean that what they say about models does not suggest much real knowledge.

 

[John_Silver]

So, when you say MeteoConsult is “spot on”, which MeteoConsult?

The top line, first screen shot, is the Meteo Consult prediction.

When not in ‘Compare Models’ screen, that’s the one shown.

As to ‘why use it?’ Because I’ve found (since it was first shown to me, back around 2012) that it matches reality better than the alternatives. (By ‘alternatives,’ I’m meaning served-up-on-a-plate, local, hourly forecasts. Requiring minimal user knowledge or understanding).

The one time I ignored MC, as an outlier amongst forecasts, and went with Met Office, I found myself at anchor, in Scilly, for Storm Evert. With the other forecasts modifying their predictions, once it was too late to run for shelter.

 

[franksingleton]

The top line, first screen shot, is the Meteo Consult prediction.

When not in ‘Compare Models’ screen, that’s the one shown.

As to ‘why use it?’ Because I’ve found (since it was first shown to me, back around 2012) that it matches reality better than the alternatives. (By ‘alternatives,’ I’m meaning served-up-on-a-plate, local, hourly forecasts. Requiring minimal user knowledge or understanding).

The one time I ignored MC, as an outlier amongst forecasts, and went with Met Office, I found myself at anchor, in Scilly, for Storm Evert. With the other forecasts modifying their predictions, once it was too late to run for shelter.

As I am not logged I, I do not see that display.

However, although the forecasts differ slightly, they all look to be within natural weather variability. Effectively, I see them as members of a large ensemble. ECMWF runs a 50-member ensemble. Output is on the Meteociel site. A copy of an ensemnle for wind speed is attached. Values are in km/hr. This gives an idea of the uncertainties inherent in NWP models.

Like PredictWind, Squid, Windy.com, Ventusky, LuckGrib etc, MeteoConsult can only regurgitate official models or run a model in a degraded form. The initial data handling is an immense task in its own right.

When MC had a free service on its app, it was, clearly, the GFS. As I cannot see their output. I do not know what they now do.

We all have selective memories none of which will be accurate. I am as sure as can be that, whichever model or combination of models they use, MC will be no better than other free services. They could well be not as good.

 

[Martin&Rene]

Frank

RE your message to the UKMO mentioned in post 86. As my experience in Scotland was the same as Dunedin, I sent a message a long time ago and the reply I received was in post 38

Quote from post 38, dated 26th August.

""Yesterday, I raised this query, mentioning this forum thread, with the Met Office and here is there reply.

Quote

Thank you for taking the time to contact The Met Office, with your feedback. After referring this information to our Meteorologist team, it has been advised the data you are referring to is automatically produced from our model data and as such we have raised a feedback, so our teams can review the reason for the discrepancy and work to adapting and improving the service for all users.

It is also noted that for the most consistent and confident forecast, we would advise checking Inshore waters and strong winds, which is produced by our highly trained Marine Meteorologists.

I hope this is helpful and please note, all feedback is reported across our senior management teams, so they can prioritise the improvements we make for our end users.

Thank you again for your time and for the information supplied.

Unquote

So we will see what happens.""


It will be interesting to see if your name carries more impact.

 

[RunAgroundHard]

I don’t see it as an issue. I have sailed with significantly less certainty in the past and planned passages.

Today, the level of forecasting is superior and very easy to gauge confidence compared to the past, even with the degree of uncertainty being expressed in this thread.

My last passage of uncertain weather was a beat in a F6 (7) between Gigha and MoK in August. Most Predict Wind models agreed, but all were around a F4>5. Direction was about right, but strength was significantly more, especially in the gusts as has been mentioned.

I just reefed main and genoa and dealt with it. After all that what we will all do if the forecast is out.

We are very fortunate sailors to have access to these models for not a lot of money, and in some cases free. I agree with the met office feedback, the inshore forecast and station reports (in my case via Navtext), are good sources for further gauging prediction accuracy, in the near term. This is an area where old school synoptic chart creation experience pays off.

 

[dunedin]

I have sent this to the Met Office.

“I have been seeing comments from sailors, some in the Clyde area, that Met Office forecasts of gusts have seemed abnormally strong recently.

One comment is that the Met Office website shows stronger gust speeds than are apparent using output from UKV on, for example, the PredictWind site.

Has there been any modification to algorithms for predicting gust speeds? Are you aware of differences between what appears on the website and UKV output?”

I will await their reply with interest.

Thanks. For info, I believe the Met Office web app forecast suddenly changed to create this issue in the high gust forecasts sometime around July this year.
Certainly wasn’t noticeable up to mid June, when was using forecast daily. Then I was away from the water from a bit so can’t pinpoint a precise date, but started noticing the trend by early August.

 

[dunedin]

Frank

RE your message to the UKMO mentioned in post 86. As my experience in Scotland was the same as Dunedin, I sent a message a long time ago and the reply I received was in post 38

Quote from post 38, dated 26th August.

""Yesterday, I raised this query, mentioning this forum thread, with the Met Office and here is there reply.

Quote

Thank you for taking the time to contact The Met Office, with your feedback. After referring this information to our Meteorologist team, it has been advised the data you are referring to is automatically produced from our model data and as such we have raised a feedback, so our teams can review the reason for the discrepancy and work to adapting and improving the service for all users.

It is also noted that for the most consistent and confident forecast, we would advise checking Inshore waters and strong winds, which is produced by our highly trained Marine Meteorologists.

I hope this is helpful and please note, all feedback is reported across our senior management teams, so they can prioritise the improvements we make for our end users.

Thank you again for your time and for the information supplied.

Unquote

So we will see what happens.""


It will be interesting to see if your name carries more impact.

Thanks. Interesting.
But also rather naive to suggest using the Inshore Waters forecast. I suspect you are like me and check a range of forecasts, including already looking at the Inshore Waters forecast every day. But they do different things.
The IW forecast covers a very wide area and is generally fairly vague. Definitively a useful source, and out of habit I note down the 0700 issue in my day log every day - but I generally find / found a combination of the Xc / GFS and Met Office online (pre-issue) to be more accurate than the IW one!

 

[dunedin]

I don’t see it as an issue. I have sailed with significantly less certainty in the past and planned passages.

Today, the level of forecasting is superior and very easy to gauge confidence compared to the past, even with the degree of uncertainty being expressed in this thread.

My last passage of uncertain weather was a beat in a F6 (7) between Gigha and MoK in August. Most Predict Wind models agreed, but all were around a F4>5. Direction was about right, but strength was significantly more, especially in the gusts as has been mentioned.

I just reefed main and genoa and dealt with it. After all that what we will all do if the forecast is out.

We are very fortunate sailors to have access to these models for not a lot of money, and in some cases free. I agree with the met office feedback, the inshore forecast and station reports (in my case via Navtext), are good sources for further gauging prediction accuracy, in the near term. This is an area where old school synoptic chart creation experience pays off.

Yes of course we expect stronger winds at times, particularly in places with headlands and or hills like Mull of Kintyre and N or Arran.
But forecast which regularly had base winds less than 10mph and gusts of nearly 30mph for weeks on end aren’t much use. May as well look at the seaweed. And for these many weeks the Met Office gust forecast was wrong (very pessimistic) at least 95 % of the time, whereas XC (and the Met Office in Predict Wind) was generally broadly correct at least 80% of the time - which is as good as one can expect.

 

[franksingleton]

I have had the same reply re GMDSS forecasts. I replied to the Met Office that I have always said that sailors should be aware of and heed these but as headline summaries and warning purposes, primarily. I reminded them that the Office issues UKV data that can be seen published by Windy, Ventusky, Predictwind etc. I pointed out that sailors would use these and that differences only served to create doubt about the forecast.

I will be interested to see what they say. Gust speeds in the app must, surely, be from UKV. So, why the differences?

 

[lustyd]

I think that that highlights a problem with this whole area. Consumers don't care about the detail, they want to look at a report, see it's consistent and use the information. The MET's answers are effectively experts who are blaming the consumer for not knowing enough, and ultimately that will be their downfall. It's not helpful and it doesn't lead to an effective solution since the forecast isn't aimed at experts. From a consumer perspective, I couldn't care less why the gust forecast doesn't line up with reality, I need someone to provide one that does. Even if that means subtracting a percentage and fudging it.

It's a common problem in many industries these days, and has always been a problem with experts in any field. Experts tend to double down on an explanation their consumers don't about rather than listen to what the consumer was trying to find. This is based on the mistaken assumption that other people are just as interested in the subject.
It's fundamentally the same as your conclusion on the AI stuff - if the answer is consistent and accurate enough, and the consumer gets what they need, then the black box might be a better approach.

 

[franksingleton]

Like weather services of most maritime nations, the Met Office has multiple functions. First, of direct interest to us, it is required to issue forecasts in a manner defined by the user, in the UK this is the MCA. Inshore and offshore forecasts are required to be provided in a well defined manner in a defined format with strict word limits. The formats are far from perfect but nobody has come up with anything better bearing in mind the distribution processes.

The Met Office also has to provide a service to the general public. Hence, in addition to broadcast media, it has its own website and products including the app. Maybe there is a problem with the app and feedback from the general public, ie us, will be looked at and questions asked.

We, as users, have to remember that weather is far from a perfect system. There are rarely any real certainties. With any weather forecast system, and I include AI, there will always be uncertainty on all scales from that of gusts lasting a few seconds right up to major circulations.


To some extent, meteorologists are victims of their own success. When it goes wrong for whatever reason, we feel let down. Cruising Brittany this year, there were spells of a week or more when the models all differed both from each other and from successive runs of the same model. These effects seemed more marked than in recent years. But, as I have said, or memories are selective. We are often swayed by others as is seen in the way fallacies picked up by social media all too easily become accepted as facts. It will be interesting to see whether we get any definitive response from the Met Office, let alone whether it will be believed.

 

[lustyd]

We, as users, have to remember that weather is far from a perfect system

My point wasn't that it should be perfect, and neither was I attacking weather pros specifically. The problem is that consumers have identified a problem that seems to be consistent and the response has effectively been to tell those consumers they don't understand how weather forecasting works. It's the expert problem I was describing. Sometimes it's good to have a less technical product owner between the experts and the consumers for this very reason.