ECMWF AI

[franksingleton]

ECMWF has recently announced that it is now running an AI forecast model operationally rather than as a trial. AIFS runs in parallel with the IFS (Integrated Forecast Sysren). Nesr future developments will include producing an ensemble.

This is not yet a full AI system in that the starting point is the same analysis, requiring the same computer resources, as used for the IFS.


Using raw observations rather than a pre-processed data set will be more difficult to implement in terms of the necessary learning. Such a system when implemented will, in principle, greatly reduce the computer resources needed. Work is in hand on AI-DOP, AI-Direct Observation Prediction.


However, the introduction of any new system in weather prediction has always been a case of "Softlee, softlee, catchee monkey." Both ECMWF and the Met Office are thinking in terms of hybrid operational systems in order to maximise the strengths of the two approaches.

 

[lustyd]

I look forward to seeing the results of this. Sadly no longer in the business but these sorts of developments are very interesting

 

[AntarcticPilot]

I think it's important that we recognise that the term AI covers a wide range of computational techniques which differ enormously in what they do. Neural nets are very different animals to Large Language Models, for example. And there are many different flavours of Neural Net! But you can think of Neural Nets being massively parallel multidimensional correlation machines, while LLMs look for statistical predictions of what the next term in a progression will be given the previous terms. The former is what is generally used in situations like weather forecasting.

 

[newtothis]

I think it's important that we recognise that the term AI covers a wide range of computational techniques which differ enormously in what they do. Neural nets are very different animals to Large Language Models, for example. And there are many different flavours of Neural Net! But you can think of Neural Nets being massively parallel multidimensional correlation machines, while LLMs look for statistical predictions of what the next term in a progression will be given the previous terms. The former is what is generally used in situations like weather forecasting.

If it's written in R, it's statistics
If it's written in Python, it's machine learning
If it's written in PowerPoint, it's AI

 

[AntarcticPilot]

If it's written in R, it's statistics
If it's written in Python, it's machine learning
If it's written in PowerPoint, it's AI

Isn't "written in PowerPoint" an oxymoron?

 

[lustyd]

Very few people are still using R, although Uni's are still pushing it for reasons only they know. It's a nightmare when people graduate and then have to learn Python to get a job.
No comment on which is better, just the reality of what's in use out there. For the record, Powerpoint is considerably more popular than either or the others, and often gets quicker results.

 

[franksingleton]

O

I look forward to seeing the results of this. Sadly no longer in the business but these sorts of developments are very interesting

So do I. No doubt incorrectly, I had gained the impression that AI and ML were, at least to some people, all about throwing masses of numbers into the pot and seeing what came out. As far as weather forecasting is concerned, it is good to see the involvement of experts in understanding weather with those who are AI experts. That will reduce the number of blind alleys explored.
I did not find it surprising that the ECMWF trial using the ERA5 data set was positive for deterministic forecasts. From my knowledge of the many data forms, my expectation is that the gains in AI forecasting using raw data will be in better ensembles and in reducing usage of computers. Deterministic forecast accuracy may not be greatly improved.