Weather Models

[lustyd]

Nope, it’s the weather

No, it's really not. You're actually describing the problem quite well though. The industry has too many people with entrenched knowledge holding it back by insisting progress cannot be made and that we can't or don't need to do better because (I'm paraphrasing your text) "weather is essentially random".

Eventually the next generation will replace such people and progress will be made in both prediction ability and in quality of supplied information. For now, I guess we have what we have.

That said, I suspect some of the global cloud providers will refocus their compute resources on this problem some time soon and remove the need for weather professionals entirely. The progress with large AI models is astounding right now and the investment unimaginable. Machine learning removes the problem of humans being unable to deal with large numbers of variables.

 

[Stemar]

Weather is a chaotic system. That bloody butterfly flapped the wrong wing/

 

[laika]

Some friends had been planning a picnic for this evening. I spent some time last night clicking through the models which Windy lets you use. The ECMWF's model seemed to give the best picnicking weather. It's a shame you can't just pay a subscription and get the actual weather your chosen model predicts. ECMWF was wrong.

 

[johnalison]

I tend to choose whichever forecast is likely to be most acceptable to my crew. However, the suggestion that there seems to be something wrong with current forecasts might be a reasonable one, but not one I feel qualified to answer. I am not sure that enough evidence has been given to support the contention, but am happy to accept it as a posable question and will wait until someone such as Frank Singleton can come along and give an answer I can trust.

 

[KeelsonGraham]

No, it's really not. You're actually describing the problem quite well though. The industry has too many people with entrenched knowledge holding it back by insisting progress cannot be made and that we can't or don't need to do better because (I'm paraphrasing your text) "weather is essentially random".

Eventually the next generation will replace such people and progress will be made in both prediction ability and in quality of supplied information. For now, I guess we have what we have.

That said, I suspect some of the global cloud providers will refocus their compute resources on this problem some time soon and remove the need for weather professionals entirely. The progress with large AI models is astounding right now and the investment unimaginable. Machine learning removes the problem of humans being unable to deal with large numbers of variables.

Oh dear. SOOOO many misconceptions I don’t know where to begin.

 

[smert]

The models have been built with a specific set of parameters. Conditions recently have seen a number of those parameters with values outside of nominal e.g. The jet stream was further south than the models were designed for. This meant that forecasting has been a bit of a guessing game! The models will be getting updated with the new data, so will start to get back to the accuracy of previous, but it's going to take a bit of time for them to 'learn'.

 

[WoodyP]

This year it's been a case of look out the window. Our local forecasts are usually way out on all models.

 

[KeelsonGraham]

The models have been built with a specific set of parameters. Conditions recently have seen a number of those parameters with values outside of nominal e.g. The jet stream was further south than the models were designed for. This meant that forecasting has been a bit of a guessing game! The models will be getting updated with the new data, so will start to get back to the accuracy of previous, but it's going to take a bit of time for them to 'learn'.

Nope. They’re far more sophisticated than this. Read my earlier post.

 

[lustyd]

Literally demonstrating that you’re the problem. People who spend their careers telling people how things are impossible are often replaced by more open minded folk. Look at any project Elon Musk has done for examples in many industries, weather forecasting as we know it will be gone in a decade.

 

[KeelsonGraham]

Literally demonstrating that you’re the problem. People who spend their careers telling people how things are impossible are often replaced by more open minded folk. Look at any project Elon Musk has done for examples in many industries, weather forecasting as we know it will be gone in a decade.

Hey, I’m just giving you a brief precis of the problem. Perhaps you‘d like to read about the Cray supercomputer that Met Office uses: The Cray XC40 supercomputing system

Or learn about Chaos theory: Chaos theory - Wikipedia

Or read Chapter 18 of my 892-page book on Aviation Meteorology.

Just sayin

 

[lustyd]

Perhaps you’d like to read about the supercomputer I use azure.Microsoft.com. We have orders of magnitude more compute than the Met Office. Cray is quite outdated as a platform, but I guess it suits the way weather people work.
You’re so indoctrinated you can’t even consider the possibility there are solutions you aren’t aware of.

If you wish to achieve results that have not been seen before, you must try methods that have not been used before.
-Francis Bacon

 

[Close hauled]

Check out the Met Office 10 day weather summary on YouTube. It’s a very unpredictable week.

 

[franksingleton]

Discounting seasonal and longer periods, forecasting is in two broad categories. First, the large scale, ie where the highs, lows, fronts will be over the next few days. Secondly, what will happen on the local scale in the short term over the next few hours. Large scale prediction has definitely improved over the past 70 years - my lifetime connection with the subject. Since the 1960s, improvements have been made possible by technological developments. First, computers provide the only way to calculate the physical processes that drive the atmosphere. Secondly, remote sensing, mainly space based, is the only way to measure the atmosphere in its global entirety.

Both have problems. No computer yet imaginable can match the atmosphere for power and complexity. No observing system or systems can measure the atmosphere as completely and as accurately as the modelling scientists would like.

Undoubtedly, there are weather situations where different global models come up with different solutions. This is mainly because of small differences in models. Many of the calculations use guesstimates of the way heat is transferred in the atmosphere. There is no way that radiant heat, convective or conductive heat transfer can be calculated accurately. Terms are derived by a system known as paramaterisation - a long word meaning “intelligent guess.” In my experience, different models (GFS, ECMWF, ICON, UK UM, give similar but never identical answers. When they do not, my advice is to trust none. Be mor cautiou than usual.

Before commenting on suggestions that models have been unusually unreliable this year, I would have to see some statistical evidence. It is tempting to say that climate change is creating more variability but that is looking for an explanation without understanding the problem. To get an idea of the inherent variability in models it is necessary to look at grid point values from model ensembles. Meteociel has a useful service showing grid point vale’s for a selection of global and more detailed models.

Limited area or meso—scale models should, in principle, give better local detail. However, that depends on two factors. First, is that small detail, say less than 30 km have short lifetimes and, so, are virtually unpredictable for most sailing purposes. Most, probably all “unofficial” models do not have detailed weather data input.

A word of warning, no supplier of weather model data ever tells you that models have in-built smoothing so that effective resolution is about 5 grid lengths, regardless of model. Claims of high precision forecasts are advertising crap. ECMWF free data are only available on a 0.4 degree grid but the loss of information from the 0.1 degree data output is minimal.
I may or may not be the only person in the forum with a first hand knowledge of weather models, weather data and forecasting but I get the impression that few here have a realistic appreciation of the problem. It is always invidious to mention specific names but Little Sister is one who seems to be realistic.

 

[lustyd]

Both have problems. No computer yet imaginable can match the atmosphere for power and complexity. No observing system or systems can measure the atmosphere as completely and as accurately as the modelling scientists would like

You’re down in the using current methods trough again. We don’t need to model molecule for molecule we just need better ways to model. Everyone is so busy refining the riding crop that they can’t imagine a motor car.

 

[franksingleton]

You’re down in the using current methods trough again. We don’t need to model molecule for molecule we just need better ways to model. Everyone is so busy refining the riding crop that they can’t imagine a motor car.

You misunderstand what I am saying in the same way that you do not understand weather models. Tell us how you get better models given the basic laws of physics, eg Newton’s third law.

 

[lustyd]

Lol no I’ll leave it there since you guys keep trying to mansplain things you think I don’t know and ignoring that there might conceivably be a better way. I’ll leave you in your cosy little world of knowing everything there is to know about 20th century weather science.

I will take back one thing though, it’s not the models. It’s the people that are the problem.

 

[johnalison]

Lol no I’ll leave it there since you guys keep trying to mansplain things you think I don’t know and ignoring that there might conceivably be a better way. I’ll leave you in your cosy little world of knowing everything there is to know about 20th century weather science.

I will take back one thing though, it’s not the models. It’s the people that are the problem.

I would bet on Frank knowing 90%+ of what is currently known about weather science, which compares rather well with the <5% that applies to all the rest of us (put together possibly). I hope that Frank is in a cosy place because he deserves it.

 

[RobbieW]

Lol no I’ll leave it there since you guys keep trying to mansplain things you think I don’t know and ignoring that there might conceivably be a better way. I’ll leave you in your cosy little world of knowing everything there is to know about 20th century weather science.

I will take back one thing though, it’s not the models. It’s the people that are the problem.

There assuredly are better ways but to assume meteorogists are not exploring them is simply arrogance

 

[lustyd]

I would bet on Frank knowing 90%+ of what is currently known about weather science, which compares rather well with the <5% that applies to all the rest of us (put together possibly). I hope that Frank is in a cosy place because he deserves it.

Yes I’m sure he and his colleagues congratulate themselves every day on how clever they are. As I said though, in 10 years that knowledge will be worthless when someone comes along and shakes things up. That person probably won’t be within the profession. As demonstrated here any contrary thinking is quickly squashed and ridiculed, much like the medical community.

 

[lustyd]

There assuredly are better ways but to assume meteorogists are not exploring them is simply arrogance

They’re actively mansplaining why they don’t want or need to. At least the ones here.