Has somebody broken the Met Office weather model?

[lustyd]

Maybe another way of looking at it is that it took a generation's worth of data to irrefutably show that there was a trend and not just randomness. I do question the implications of this, and your previous post (to which I responded) that all experts are only interested in preserving the status quo until retirement. That being the case, which generation of experts do we believe and which status quo do we choose to accept?

It's not just about that one issue, this is a very well known phenomenon and it's always been the same. Take the Harrison clock - astronomers went out of their way to discredit him and his method despite significant evidence that it worked flawlessly. There's a difference between asking for more evidence and actively fighting against progress or listening to reason.
I have to say I was impressed with Frank recently after a lengthy thread on AI when he did go on to research and accept that times are changing. He stated he didn't like it, but did accept it's probably the future. Many others I've had that conversation with continued to fight and try to explain why the current system is the correct one.

 

[wwarw00]

Hi Frank,

a quick "thank you" for all your input and responses into this thread.

Unlike you, I know very little about weather forecasts other than that which the average sailor learns over 20 years of watching forecasts before going sailing. That said, I'm very much in the "met office website gust forecasts have changed" (e.g. Exe Estuary (Devon) weather) in the last few months camp.

You said that "I do not know if gust algorithms have changed. My instincts say NO but, I do not know". Like you I don't know if the algorithms have changed but I know, for sure, that the Met Office website changed mid summer when they changed the help text displayed by the gust values to "wind gust enhanced" (mentioned here earlier here).

I also know that the gust values shown the above Met Office web page no longer line up 1:1 with the values that I pull from their "Met Office Global hourly spot data" API when they did in the past. I'm afraid I don't know if their "hourly spot data" API values are the same thing as the UKV model mentioned in this thread.

I guess this only really matters because, like many of us, I'd built the Met Office forecasts (and especially their gust forecasts) into my "mental model" of when to go sailing / kayaking. That "mental model", for sure, placed greater weight on the Met Office forecasts compared to the various others I also use. Nowadays the Met Office input ranking to my "mental model" is way lower (I want to go out sailing occasionally!!!!). If there was some info from them on how they've changed their gust forecast, it would make their forecast more useful.

Cheers

Aidan

 

[franksingleton]

Hi Frank,

a quick "thank you" for all your input and responses into this thread.

Unlike you, I know very little about weather forecasts other than that which the average sailor learns over 20 years of watching forecasts before going sailing. That said, I'm very much in the "met office website gust forecasts have changed" (e.g. Exe Estuary (Devon) weather) in the last few months camp.

You said that "I do not know if gust algorithms have changed. My instincts say NO but, I do not know". Like you I don't know if the algorithms have changed but I know, for sure, that the Met Office website changed mid summer when they changed the help text displayed by the gust values to "wind gust enhanced" (mentioned here earlier here).

I also know that the gust values shown the above Met Office web page no longer line up 1:1 with the values that I pull from their "Met Office Global hourly spot data" API when they did in the past. I'm afraid I don't know if their "hourly spot data" API values are the same thing as the UKV model mentioned in this thread.

I guess this only really matters because, like many of us, I'd built the Met Office forecasts (and especially their gust forecasts) into my "mental model" of when to go sailing / kayaking. That "mental model", for sure, placed greater weight on the Met Office forecasts compared to the various others I also use. Nowadays the Met Office input ranking to my "mental model" is way lower (I want to go out sailing occasionally!!!!). If there was some info from them on how they've changed their gust forecast, it would make their forecast more useful.

Cheers

Aidan

Thank you. I had missed that. I have now seen Enhanced weather data. It does seem that they have made some changes to the app. However, as I said earlier, changes to output are only made after extensive resting. They will continue to monitor their product.

A quick check using Ventusky showed that the app gusts were close to the UKV values. Gust/mean wind ratios may have changed on the app following the changes. They are not mentioned. It will be difficult to assess whether the forecasts are better or not. By their very nature gusts have a random element. The Meteociel display will be useful but not conclusive.

 

[franksingleton]

It's not just about that one issue, this is a very well known phenomenon and it's always been the same. Take the Harrison clock - astronomers went out of their way to discredit him and his method despite significant evidence that it worked flawlessly. There's a difference between asking for more evidence and actively fighting against progress or listening to reason.

I do not think that you can apply that to meteorologists. By the early post WW2 years, we had a global radiosonde network. The UK Met O, in particular developed a manual technique to ensure consistency between charts drawn at levels from the surface up to 200 hPa. That was how I first had forecasting experience. Papers were written and much invested in the technique.

Come the mid 1960s, even with the first, crude operational NWP, that process was junked. By 1967 when I became a senior forecaster at Bracknell, I never had to go through that process.

Now, 60 years on, meteorologists have been studying AI for a few years. I could point to ways in which satellite data are used in data analysis despite having totally different characteristics to our long-standing in situ systems.

I have to say I was impressed with Frank recently after a lengthy thread on AI when he did go on to research and accept that times are changing. He stated he didn't like it, but did accept it's probably the future. Many others I've had that conversation with continued to fight and try to explain why the current system is the correct one.

I have only yet seen results of AI starting from NWP data analyses. I can well see that the AI learning using raw data, rather than a nicely ordered data set, will be a big jump. I can well see that forecasters in the future will find life difficult in that they could well lose the insight into forecasts by not having a physical model to look at. There will still be the turbulence on all scale, not just the gusts that have been exercising minds here. Maybe the days of human forecasters are nearly at an end. Algorithms based on (AI) model output will be the norm.

 

[Chiara’s slave]

It’s all gone a bit Pete Tong here. App says sunny intervals, less than 5% rain. Weather, not a sky in the clouds, persisting down. Hiding down below. Just come on to rain even harder. More a reflection of British weather than a gripe at the met office.

 

[franksingleton]

It’s all gone a bit Pete Tong here. App says sunny intervals, less than 5% rain. Weather, not a sky in the clouds, persisting down. Hiding down below. Just come on to rain even harder. More a reflection of British weather than a gripe at the met office.

We had to walk to our surgery last Saturday morning. On Friday, it looked 50:50 that we would be able to get there in the dry. Friday evening, the forecast showed rain just clearing by 1045 when we would have to leave for the 1/2 hour walk. On Saturday morning, 1030, the rain cleared. On several occasions over the past year or two we have rarely been misled by the app. We do not now have a car. We have to make critical decisions.

 

[Snowgoose-1]

It’s all gone a bit Pete Tong here. App says sunny intervals, less than 5% rain. Weather, not a sky in the clouds, persisting down. Hiding down below. Just come on to rain even harder. More a reflection of British weather than a gripe at the met office.

Met Office workers get £25million in bonuses despite weather predictions failing

 

[lustyd]

Met Office workers get £25million in bonuses despite weather predictions failing

Over 5 years and the biggest bonus they mention is £10k. Hardly excessive, and it’s unlikely accuracy is their only metric for success.

 

[Snowgoose-1]

Over 5 years and the biggest bonus they mention is £10k. Hardly excessive, and it’s unlikely accuracy is their only metric for success.

I don't think money is the issue. It's the gust speeds raised by the OP. Unfortunately, we are no wiser . I no longer use the Met Office forecasts .

 

[Chiara’s slave]

I don't think money is the issue. It's the gust speeds raised by the OP. Unfortunately, we are no wiser . I no longer use the Met Office forecasts .

We look at all available forecasts. We are sailing tomorrow. The met office are true to recent form, the inshore forecast is 25 gusting 28. It’s an outlier by 70%.

 

[franksingleton]

I don't think money is the issue. It's the gust speeds raised by the OP. Unfortunately, we are no wiser . I no longer use the Met Office forecasts .

Perhaps you missed #63.

 

[franksingleton]

We look at all available forecasts. We are sailing tomorrow. The met office are true to recent form, the inshore forecast is 25 gusting 28. It’s an outlier by 70%.

We often see this kind of post. What forecast are you using, what time, what area, what was the outcome etc. Then, anyone interested can make their own judgement,

Many years ago I drew the short straw and was assigned to meet a reporter with a Rottweiler reputation. His opening gambit was to produce several forecasts provided by his readers. All, he said, had “gone wring.” Playing for time, I called a clerk and gave him/her the list and asked them to bring copies (paper, of course) of what we had actually written. When we read through our texts, none of the examples provided bore much resemblance to the actual forecasts. When the article, about 3 pages in the then Observer magazine c1976, I, and the Met Office, came up smelling of roses Phew

 

[Snowgoose-1]

Generally, I find the Inshore Waters ok. Also produced by the Met Office . Perhaps there is more human editing/input involved.

It would be difficult to edit the 100's of other places and the models are left to run ?

 

[Ammonite]

I think its the other way around. The UKV/UKMO 2km forecasts in Windy and Predict Wind display the raw data from the models whereas the Metoffice website uses "enhanced" data.
High resolution Met Office forecast on Windy.com

Enhanced weather data

 

[franksingleton]

Generally, I find the Inshore Waters ok. Also produced by the Met Office . Perhaps there is more human editing/input involved.

It would be difficult to edit the 100's of other places and the models are left to run ?

Like the shipping forecast and the NAVTEX versions, the Inshore Waters forecast is a subjective précis of NWP output for broadcast over voice and text radio channels to meet the GMDSS requirement. All these have limits on number of words to ensure that they can be broadcast within defined time slots. The IW could probably be generated by AI. To do similarly for the offshore. Shipping and NAVTEX broadcasts would be more difficult.