Weather advancements

[lustyd]

As a curiosity (my first time ever to use AI tools) I asked both Chatgpt and "Claude" (a French AI tool)
"how to cross the ITCZ on a sailboat?", two friends had access to the professional versions.
"Claude" derived a lot of its descriptive content from Meteo France "Météorologie tropicale" textbook, whereas Chatgpt answer was mainly based on NOAA, fair enough; but, among Chatgpt suggestions there was "You will need the most accurate weather forecast you can get, like NOAA and [hear hear] Predictwind". Should anyone take this seriously ?

You need to understand what’s happening here. You didn’t get an answer from an AI. This is RAG (retrieval augmented generation). Your question was interpreted by the language model to determine intent. The intent was searched in a standard search engine, probably Google or Bing, and the results were interpreted by the language model to explain to you what it found.
The “AI” knows nothing and makes things up if allowed. It’s just a statistical language model and what you got in results were a précis of the Internet results it found, nothing more.

AI weather models will ultimately also be statistical in nature. There will be thousands of parameters considered to determine the statistically most likely weather coming up. While theoretically less accurate than physics models (although these are deeply flawed for other reasons) the results will likely be better because more parameters will be considered and less compute required.
As an example, the physics models don’t consider crop yield from the previous year, and traditional thinking weather men will tell you that’s because it’s irrelevant. Same for many of the data sets that will be added to AI models since we no longer need to know the why of it, just whether they statistically align.
I found out recently that some of the physics models use a square grid

 

[GHA]

So Frank, looking at using AI on the boat - how far off the mark is this reply?



"what type of clouds are these, how do they form, what forecast mifght be expected" (perplexity pro, it knows I'm a sailor)

The image shows classic cumulus clouds over a body of water on a bright, sunny day. These are specifically cumulus humilis clouds, characterized by their white, puffy appearance with flat bases and minimal vertical development
3
6
.

Formation Process​

These clouds form through a straightforward process:

• Warm air near the surface rises due to solar heating
• As the air ascends, it cools and its moisture condenses
• Water vapor condenses around cloud condensation nuclei (tiny particles like dust or sea spray)
• The flat bases typically form between 500-1,500 meters above the surface
  6

Weather Forecast​

These particular cumulus clouds suggest favorable sailing conditions:

• They indicate stable, fair weather conditions
  5
• The small size and minimal vertical development suggest there's little risk of storms
• Their presence over water indicates good thermal activity and stable atmospheric conditions
  8

Sailing Implications​

For a sailor, these conditions typically mean:

• Good visibility
• Stable wind conditions
• Low probability of precipitation
• Generally favorable conditions for day sailing

However, it's important to continue monitoring these clouds throughout the day. If they begin showing significant vertical growth or darkening bases, it could indicate developing instability and potential weather changes
5
8
.
And a load of links to get lost in

 

[Laser310]

While theoretically less accurate than physics

I am not aware that there is any reason to believe that AI forecasting can not be as good or better than physics-based models.

I am not predicting that they will be, assuming they are not already better...

I am only saying that the approaches are so different that I can't think of how one could argue that one has greater potential than the other.

As a scientist, I have to say I am not that thrilled with the early success of AI forecasting.

I think it raises the question of whether we need science.

Even if scientists still want to pursue scientific investigation, taxpayers and governments mostly support it because they believe there will be a material benefit.

I'm retired now, but former colleagues worked on, for example, various aspects of atmospheric chemistry, fluid dynamics.., etc.

Will that work continue to attract funding if it is not as important for forecasting?

Sure - very few of these guys actually cared even a little bit about forecasting. It's not what academic meteorologists focus on.

But I wouldn't be surprised if they mentioned application of their work to forecasting in their grant proposals...

 

[lustyd]

I am not aware that there is any reason to believe that AI forecasting can not be as good or better than physics-based models

Note that I used the word accurate. Physics models, given enough refinement could produce more accurate forecasts. Statistical models probably won’t, but they’ll be right enough in the short term that the difference is irrelevant. They’ll also provide insights on what affects weather in a way we’ve never seen before due to sheer scale of data included.

I think it raises the question of whether we need science

I think we certainly do. I’d question how many scientists these days do real science but that’s another matter. Science in all areas these days seems to start with “this is how it’s done” and “this is the answer” which is quite broken. Different ideas are often shunned and unfunded so we’re learning less while spending more. The medical community are the worst for this by a long shot.

Will that work continue to attract funding if it is not as important for forecasting?

No and that’s a huge danger. Our country isn’t set up to fund things for the good of mankind and most folk don’t realise that AI doesn’t produce anything useful and new so politicians will certainly think it’s an area to save cash.

 

[Roberto]

Completely seriously, that's such a vague question you'll get far worse answers at happy hour.

Sure, but as a first reply it's like reading "to cheer your journey bring along some of the finest liquors like Lagavulin or Johnnie Walker", it does not help to build credibility.
Edit
Reading "get good weather forecast" is understandable
Reading "get good weather forecast like from NOAA" better
Reading "get good weather forecast like from NOAA, ECMWF, ICON" even better
Reading "forecasts in the area are prone to wide errors, do not rely too much on them" still better
Reading NOAA and Predictwind?

 

[Laser310]

Note that I used the word accurate. Physics models, given enough refinement could produce more accurate forecasts. Statistical models probably won’t, but they’ll be right enough in the short term that the difference is irrelevant.

just to take the other side...

AI machine learning models are using the ERA-5 data set. I use this data set quite a bit. It's a fantastic data set.

but it is not perfect.

1) it is nearly as much the product of a model as is the forecast, and is therefore subject to the same biases.

2) it is pretty low resolution

3) relatively little observational data over the oceans and at high altitude.

i'm sure we could come up with other problems

All of these are potentially fixable (although probably not looking back) to one degree or another, and we would expect that AI forecasts would improve once there is enough of this higher quality data to learn on.

I could imagine a major funding body, say the NSF in the US, saying: "look, it's pretty clear that machine learning is the route to better forecasts. The biggest limitation is the quality of the historical data. How about we undertake a project to acquire a massively better data set, so that, say, 30 years from now, we will have the data to begin to fulfill the potential of ML forecasting. We are going to need lots of satellites, drones, ocean gliders, ships, etc to acquire this data. Naturally, this means we will have to dramatically scale back our spending on basic science."

 

[dgadee]

I am a sceptic. I saw all these sorts of claims made for AI in the 1980s and then the hype died away as reality hit. We even had it later with IBM's Watson and Jeopardy! I will wait for the coming bust of the bubble.

Some people are making shed loads of money on the back of this bubble and some will lose a lot.

 

[lustyd]

The biggest limitation is the quality of the historical data. How about we undertake a project to acquire a massively better data set, so that, say, 30 years from now, we will have the data to begin to fulfill the potential of ML forecasting. We are going to need lots of satellites, drones, ocean gliders, ships, etc to acquire this data.

This is already underway, and more importantly the data is being made more available with good metadata to actually allow it to be used. NASA’s Earth Copilot project is pretty cool and we’re working with numerous UK orgs to achieve similar.
Proper AI models won’t limit the data sets quite so much. We’ve learned from LLMs that scale can be used to overcome some problems, and that compute scale is available and online now. Weather organisations have what are now quite small compute capabilities so it’s unlikely to be them doing the training, and it’s very unlikely anyone trained in forecasting will be involved in anything but the testing phase.

 

[lustyd]

I am a sceptic. I saw all these sorts of claims made for AI in the 1980s and then the hype died away as reality hit. We even had it later with IBM's Watson and Jeopardy! I will wait for the coming bust of the bubble.

Some people are making shed loads of money on the back of this bubble and some will lose a lot.

Your phone has more compute power than Watson had. The difference this time is scale. LLMs have been trained on all accessible data and are proving their capability daily by those using them. It’s not hype, it’s a working product.
We actually still use ML algorithms from the ‘80s for predictive use-cases. Its quite simple maths compared to much of what’s going on today but still works just fine.

 

[dgadee]

Your phone has more compute power than Watson had. The difference this time is scale. LLMs have been trained on all accessible data and are proving their capability daily by those using them. It’s not hype, it’s a working product.
We actually still use ML algorithms from the ‘80s for predictive use-cases. Its quite simple maths compared to much of what’s going on today but still works just fine.

But it's just an algorithm, not intelligence. Except from the designer.

We'll see anyway. Money is pouring in, the press are going wild and proponents are hyping to the skies. A bubble.

 

[lustyd]

But it's just an algorithm, not intelligence. Except from the designer.

Nobody involved has suggested otherwise. OpenAI have a stated goal of creating an AGI but they’ve also said many times this isn’t it and always refer to GPT as a large language model.

 

[dgadee]

Nobody involved has suggested otherwise. OpenAI have a stated goal of creating an AGI but they’ve also said many times this isn’t it and always refer to GPT as a large language model.

Oh yes they do. That's the hype - "intelligence" - suggesting it is something hugely new.

As it was with Jeopardy! No-one was allowed to investigate just what was happening inside the box. Slower machines but just the same rubbish claims.

 

[franksingleton]

I am not aware that there is any reason to believe that AI forecasting can not be as good or better than physics-based models.

The ECMWF trial shows little difference out to 3 days ahead but that AIFS betters IFS from 4 to 10 days. That is based on RMS of 500 hPa height. But, as I have said, this is only a partial AI study as forecasts start from the IFS analysis, ie requiring a 12 hour “physical” model forecast.

I am not predicting that they will be, assuming they are not already better...

I am only saying that the approaches are so different that I can't think of how one could argue that one has greater potential than the other&.

You can only go on results. After all, that is how NWP has always advanced. New models, modifications to models, new data sources have only ever been introduced on the basis of extensive, rarely enough, testing and, even then been modified in the light of experience.

As a scientist, I have to say I am not that thrilled with the early success of AI forecasting.

I think it raises the question of whether we need science.

Totally agree. Scientifically, it is a sterile technique.

Even if scientists still want to pursue scientific investigation, taxpayers and governments mostly support it because they believe there will be a material benefit.

I'm retired now, but former colleagues worked on, for example, various aspects of atmospheric chemistry, fluid dynamics.., etc.

The emphasis may change on what the scientists do. There could be greater effort put into local small scale detail. With a changing climate, modelling will be needed to predict future climate. It is significant that the Met Office/Turing Institute work is envisaging some form of hybrid system.

Will that work continue to attract funding if it is not as important for forecasting?

Sure - very few of these guys actually cared even a little bit about forecasting. It's not what academic meteorologists focus on.

But I wouldn't be surprised if they mentioned application of their work to forecasting in their grant proposals...

Meteorology is so all-encompassing that demand for knowledge wil continue.

 

[franksingleton]

just to take the other side...

AI machine learning models are using the ERA-5 data set. I use this data set quite a bit. It's a fantastic data set.

but it is not perfect.

No meteorological data set can ever be perfect. There is far too much inherent variability in the atmosphere. The same applies to forecasts.

1) it is nearly as much the product of a model as is the forecast, and is therefore subject to the same biases.

Given the multiplicity of data types I cannot see another way of producing a global data set.

2) it is pretty low resolution

3) relatively little observational data over the oceans and at high altitude.

If you mean conventional in situ data, yes. When it comes to satellite derived data, there is mor over the oceans. Meteorologists have always had to use what data are available rather than what would be ideal.

i'm sure we could come up with other problems

All of these are potentially fixable (although probably not looking back) to one degree or another, and we would expect that AI forecasts would improve once there is enough of this higher quality data to learn on.

I could imagine a major funding body, say the NSF in the US, saying: "look, it's pretty clear that machine learning is the route to better forecasts. The biggest limitation is the quality of the historical data. How about we undertake a project to acquire a massively better data set, so that, say, 30 years from now, we will have the data to begin to fulfill the potential of ML forecasting. We are going to need lots of satellites, drones, ocean gliders, ships, etc to acquire this data. Naturally, this means we will have to dramatically scale back our spending on basic science."

More data are always welcome and there is continued international collaborative effort aimed at acquiring more and better data. Meteosar 12 is now operational. To achieve your 30 year data set would have required planning and development 30 years ago. However, as Meteosat 12 shows we are on the way to enhancing data sets. We have had satellite infrared and microwave data since the mid 1970s. We have other data types, eg GNSSRO and scatterometer data for shorter periods. There have been continual improvements in instrumental data quality and quantity. The scientists have to use what is there rather than wringing their hands and start re-inventing wheels. As ever, weather prediction involves the art of the possible.

 

[Laser310]

To achieve your 30 year data set would have required planning and development 30 years ago.

of course.

my point is this:

Say, you are a minister tasked with developing a long-term program for getting the nation the best possible weather forecasting. Long-term being 10 years, 20 years.

You have a fixed budget; spending on one thing necessarily means spending less on another.

What route is likely to make the biggest advance on the current state of prediction?

Where would you put your money?

Basic science?

or

Better data collection - this is in a way, low hanging fruit. I could easily see data collection being improved by an order of magnitude - coupled with research into developing better machine learning algorithms, and numerical methods?

You would have work to do to convince me that the 2nd isn't the path more likely to make the biggest leap in forecasting skill.

You will probably say; "it isn't either, or, some of each is possible"

True, but given the fixed budget, basic science ends up with less going forward.

I don't like to see this - but I am afraid it will happen.

 

[franksingleton]

of course.

my point is this:

Say, you are a minister tasked with developing a long-term program for getting the nation the best possible weather forecasting. Long-term being 10 years, 20 years.

You have a fixed budget; spending on one thing necessarily means spending less on another.

What route is likely to make the biggest advance on the current state of prediction?

Where would you put your money?

Basic science?

or

Better data collection - this is in a way, low hanging fruit. I could easily see data collection being improved by an order of magnitude - coupled with research into developing better machine learning algorithms, and numerical methods?

You would have work to do to convince me that the 2nd isn't the path more likely to make the biggest leap in forecasting skill.

You will probably say; "it isn't either, or, some of each is possible"

True, but given the fixed budget, basic science ends up with less going forward.

I don't like to see this - but I am afraid it will happen.

Maybe we are using the term data collection in different ways. As I see it, it is not so much a case of acquiring more data but understanding and using better what we do have. From the 1850s we have data from the surface of the earth. Patchy in quality in the early stages, always heavily weighted to populated areas. From the end of WW2, we have data in the free atmosphere, above the surface, again improving in quality over time and even more heavily weighted towards populated areas. Since the mid 1970s, we have space based systems. Some of the data are truly global, some are only available over the oceans.
Primarily, these data were obtained to meet the needs of operational meteorology although, in recent years, there have been sideways glances towards understanding climate. For really long term use, these data can be set in what I loosely refer to as a geological setting.
Given that you cannot invent data, improvement in prediction will only come from better use and understanding of what we have. How best to combine use of these disparate data sets is the key. With physical modelling, the weightings of the different data forms and the calculation of heat transfer processes by models are both tackled on a trial and error basis. I could well see that AI/ML could be used to improve both these current imponderables. That is the pragmatist in me talking. However, the scientist in me says that real improvements in weather prediction have, ultimately, to come from better understanding and, therefore, the better use of what we have.

 

[Laser310]

Given that you cannot invent data, improvement in prediction will only come from better use and understanding of what we have.

No - I am saying start today to build a database for better ML-based forecasting 20 years from now; a long term investment.

sure - for now, we are stuck with what we have and sure, we can do a better job with what we have..., but imagine having an order of magnitude more than what we have.

For example, we have very little actual observational data at, say, 500mb. As far as I know, present satellites can not collect wind speed and direction at 500mb. Imagine 100,000 drones flying around at different levels of the atmosphere measuring wind speed and direction, temperature, humidity, some chemistry.., and whatever else.

right now, the NOAA radiosonde program sends up maybe a few hundred balloons a day over continental US.., I don't know what UK Met does, or The European center. But, the data is pretty sparse. There are almost no balloons over the oceans or other continents.

The point I have been trying to make, is that if all you care about is forecasting skill, my 100,000 drones is probably a better investment than basic science over the next 20 years.

This is a big problem for science - and not just in meteorology.

 

[franksingleton]

No - I am saying start today to build a database for better ML-based forecasting 20 years from now; a long term investment.

sure - for now, we are stuck with what we have and sure, we can do a better job with what we have..., but imagine having an order of magnitude more than what we have.

For example, we have very little actual observational data at, say, 500mb. As far as I know, present satellites can not collect wind speed and direction at 500mb. Imagine 100,000 drones flying around at different levels of the atmosphere measuring wind speed and direction, temperature, humidity, some chemistry.., and whatever else.

right now, the NOAA radiosonde program sends up maybe a few hundred balloons a day over continental US.., I don't know what UK Met does, or The European center. But, the data is pretty sparse. There are almost no balloons over the oceans or other continents.

The point I have been trying to make, is that if all you care about is forecasting skill, my 100,000 drones is probably a better investment than basic science over the next 20 years.

This is a big problem for science - and not just in meteorology.

You would need 100,000 drone capable of flying at all heights up to 80km. Or radiosondes on a dense network over the whole globe capable of reaching 80km. It is a good exercise not to be constrained by reality in order to explore what is possible. Clearly, the only practicable solution to global monitoring has to be space based. It is, in fact possible to measure wind profiles using Doppler Lidar as on the Aeolus programme. However, you can calculate wind from temperature and humidity data. The problem here is that satellite cannot measure either. They can only measure the effects of temperature and humidity on infrared, microwave and VHF radiation.
The solution to monitoring the atmosphere in the foreseeable future has to be a combination of all available techniques.
The role of AI is still uncertain. I do not know how much radiance data has been archived or whether the ERA5 type of data are the only sources useable for AI training. I will be interested to see what comes of the Met Office/Turing Centre approach - if I am around long enough. I am way beyond my allotted span

 

[lustyd]

Imagine 100,000 drones flying around at different levels of the atmosphere measuring wind speed and direction, temperature, humidity, some chemistry.., and whatever else.

The problem would be better approached with a more standardised way to collect and integrate data. This is being worked on by various agencies. The problem right now is that your 100,000 drones would create a data set that others would need to gain access to and integrate. Often those producing data want paying too, which makes the situation worse.

We have millions of phones and watches with temperature and pressure sensors as well as light sensors. Accessing that data would be a huge step forwards, and anomalies could be spotted using ML.

Realistically crowd sourcing is the best bet for lots of data, even if we just use that to better interpolate “real” data sets.

 

[franksingleton]

The problem would be better approached with a more standardised way to collect and integrate data. This is being worked on by various agencies. The problem right now is that your 100,000 drones would create a data set that others would need to gain access to and integrate. Often those producing data want paying too, which makes the situation worse.

We have millions of phones and watches with temperature and pressure sensors as well as light sensors. Accessing that data would be a huge step forwards, and anomalies could be spotted using ML.

Realistically crowd sourcing is the best bet for lots of data, even if we just use that to better interpolate “real” data sets.

Physical modelling requires global, 3D data, surface to about 80km. Some have suggested even higher but that seems very unlikely. AI will also require global 3D data.