Weather advancements

[lustyd]

Thought this might interest a few after the recent conversation about AI and weather models. This one is Google working with ECMWF to make some pretty good progress. While it may seem like a small step, note that this also helps move to faster compute so we’ll likely see rapid iterations over the coming year along with more experimentation. They’ll also pull in more seemingly unconnected data and move away from big cubes.
It’s interesting to think that this isn’t more accurate, it’s just right more often.
The secret to better weather forecasts may be a dash of AI

 

[franksingleton]

I have not been watching YBW recently and missed this.
ECMWF and the Met Office are both studying AI forecasting. ECMWF used 40 years of historical analyses to create the starting point for the AI learning process. Their trial AI forecasts have started from the IFS analyses. Results so far suggest that AI could outperform the current physical models. The unknown is whether AI based on raw observational data will also perform better than physical modelling. In other words, can the IFS analysis stage be omitted.
To get to the starting point for a physical model takes about 3 hours from the nominal observation times of 00 and 12 UTC.(my guess). The forecast takes about another 3 hours. An AI forecast would be far faster. So, even now, using AI would produce earlier forecasts and save some 6 hours a day computing effort.

 

[lustyd]

Saw your article in the CA mag, good stuff

 

[franksingleton]

Saw your article in the CA mag, good stuff

I have put a copy at AI Forecasts - Franks-Weather - The Weather Window

 

[oldbloke]

The "AI" is only doing what weather forecasters have always done. Use the available data and previous experience, Red sky at night sailors delight. If you get wet when you go outside, it's raining. Nothing new, may be quicker, may be more accurate more often. The big difference from The Met Office etc is it's not trying to understand the mechanisms. Fortunately it won't be an either/or

 

[Roberto]

AI experimental weather products

 

[franksingleton]

The "AI" is only doing what weather forecasters have always done. Use the available data and previous experience, Red sky at night sailors delight. If you get wet when you go outside, it's raining. Nothing new, may be quicker, may be more accurate more often. The big difference from The Met Office etc is it's not trying to understand the mechanisms. Fortunately it won't be an either/or

At #6 Roberto reminded us of AI experimental weather products. In that, I said -

“As a scientist, I dislike the black box approach but have to admit that if it really does work, then so be it. ECMWF and the Met Office will, for the time being, follow both lines.”

AI may provide statistically better forecasts than NWP models but there will always be some uncertainty. The atmosphere will still be governed by the laws of physics. There will still be turbulence and consequent predictability limits.

Models may cease to be operational tools.Their future role might be in understanding the atmosphere.

 

[franksingleton]

The Met Office will be trialling AI in the not too distant future. Trialling of any new system will be lengthy and thorough.
See AI crucial to tackling environmental challenges, say experts at Turing and Met Office event

 

[Ammonite]

For those who have not seen it
Google DeepMind predicts weather more accurately than leading system

 

[Laser310]

the ECMWF AIFS AI model has been downloadable within Expedition in GRIB format for at least a year now.

ECMWF | Charts

It can be used for optimal routing etc. I've used it.

BTW, one does not need a license to just install the program and use it for downloading and viewing gribs. There is a huge selection of gribs available for easy download within the program.

what the license gets you is instrument connectivity and optimal routing.

gribs downloaded in expedition can be opened in other programs

 

[franksingleton]

For those who have not seen it
Google DeepMind predicts weather more accurately than leading system

I do not think that there is anything new over and above what has been said in earlier posts. In particular, the article says -
“There is still a long way to go before machine .learning approaches can completely replace physics-based forecasting.”One reason, as I said #2 and at AI Forecasts - Franks-Weather - The Weather Window is that the training has not been done using raw data. So far, as far as I know, they have used ECMWF IFS re-analyses. The forecasts on the ECMWF site have to start from analysis data generated by the IFS.
The Met Office and the Turing centre are studying how to optimise the strengths of the two approaches.

 

[lustyd]

Training on the raw data would take months, and probably a few iterations to get right. It’s being worked on so I’d expect something to appear in a year or so. It’s likely that it’s also taking time to collate the necessary data which would require cooperation from many organisations worldwide.

 

[Roberto]

Another type of advance
Europe’s most advanced weather satellite is now fully operational

 

[franksingleton]

Training on the raw data would take months, and probably a few iterations to get right. It’s being worked on so I’d expect something to appear in a year or so. It’s likely that it’s also taking time to collate the necessary data which would require cooperation from many organisations worldwide.

I cannot guess how long training using raw data might take. Intuitively, I would guess that is far more complex than using the ERA5 data, ECMWF Reanalysis v5. I suppose that it goes without saying that the learning will have to be an ongoing process as the available data are changing continually.

I doubt that we sailors will see any great improvement in forecasts apart from them being available earlier. That would be a significant added value. The atmosphere will still be turbulent. Predictability of small detail will still only be for short periods. The larger the size of the weather system, the longer ahead it will be predictable.

Probability forecasting could well improve and that is important commercially but less so for the majority of us. Not many here will be into optimal routing.

 

[franksingleton]

Another type of advance
Europe’s most advanced weather satellite is now fully operational

See, also, Operational satellite a ‘landmark moment’ for weather forecasts. I suspect the main value here is in the very short term. An overall view of lightning will be useful. Impact on longer periods will be in the more accurate definition of the analysis whether for conventional physical modelling as well as AI models. The idea of being able to see detail of 1 - 2 km, never mind 1/2 km at 10 minute intervals from a GEOS orbiter is mind bending.

 

[lustyd]

I suppose that it goes without saying that the learning will have to be an ongoing process as the available data are changing continually

With LLMs we’ve now developed techniques to train incrementally to reduce carbon footprint. Initially the GPT models took almost a year to train using unimaginable compute capability but that’s reduced as has the compute to run the model afterwards.
I don’t think additional weather data will have a lot of impact, so unlikely that will need updating too often, but additional other data sources like mapping layers may be added as they are sourced. The OS has a great set of map layers with stuff like surface types and even roofing types on buildings (we used ML to determine this). All very useful for a big weather model

 

[GHA]

Frank - ask an AI!!
www.perplexity.ai

Cut & paste doesn't include the links though..

"i am a sailor interested in weather, i want to find any podcasts or pages describing the latests advancements in AI in weather forecasting"

Answer
As a sailor interested in weather, you'll be excited to learn about the latest advancements in AI for weather forecasting. Here are some podcasts and resources that discuss recent developments:

Podcasts​

1. The WeatherPod: This podcast recently featured an episode with Irina Sandu, Director for Destination Earth at the European Centre for Medium-Range Weather Forecasts (ECMWF). The episode explores the impact of AI on weather and climate prediction, including digital twins for weather forecasting
   1
   .
2. Triple Point Podcast: Episode 39, titled "The Accelerating AI Landscape in Weather," features Dr. Amy McGovern from the University of Oklahoma. This episode discusses recent developments in AI weather prediction, including policy implications and accelerating data assimilation
   8
   .

Online Resources​

1. NVIDIA's YouTube Channel: NVIDIA recently released a video titled "How AI is Transforming Climate Forecasting and Weather Prediction." This video discusses cutting-edge AI technologies like Earth-2 and Stormcast, which are pushing the boundaries of weather prediction with generative AI
   2
   .
2. Google DeepMind Blog: Google DeepMind has published information about GraphCast, their state-of-the-art AI model for weather forecasting. GraphCast can deliver 10-day weather predictions with unprecedented accuracy in under one minute
   6
   .
3. AIwire: This website features an article about NVIDIA's new AI model called StormCast, which aims to revolutionize extreme weather forecasting. The article discusses how StormCast compares to traditional forecasting methods and its potential impact on disaster planning and mitigation
   9
   .

Key Advancements​

1. Improved Accuracy: AI models like Google's GenCast are outperforming traditional forecasting methods, providing more accurate predictions up to 15 days in advance
   3
   7
   .
2. Faster Processing: AI-powered models can generate forecasts much quicker than conventional methods. For example, GraphCast can produce a 10-day forecast in less than a minute
   6
   .
3. Extreme Weather Prediction: AI models are showing promise in predicting extreme weather events earlier and more accurately, which could be crucial for sailors
   3
   9
   .
4. Data Assimilation: AI is accelerating the process of data assimilation, which is vital for improving weather model accuracy
   8
   .
5. Long-range Forecasts: Companies like Salient Predictions are using AI to enhance long-range weather forecasts, which could be particularly useful for planning sailing trips
   5
   .

As a sailor, these advancements in AI weather forecasting could significantly improve your ability to plan trips and stay safe on the water. Keep an eye on these resources for future updates as the field continues to evolve rapidly.

Quite amazing just how capable a tool it is and doesn't get grumpy when you point out an error, I cut n pasted an image & asked what is this -

Answer
This is a surface pressure weather chart (synoptic chart) from the German Weather Service (Deutscher Wetterdienst), dated June 6, 2024, at 17 UTC. The chart shows:

Key Features​

• High pressure systems marked with "H" and isobars showing pressure values around 1010-1025 hPa
• Several pressure systems across what appears to be Northern Europe
• Frontal systems indicated by black lines with various symbols
• Isobar lines connecting points of equal atmospheric pressure

Technical Details​

The chart appears to be a basic ICON model forecast valid for June 7, 2024, 00:00 UTC, as indicated at the bottom of the image. This type of chart is particularly useful for sailors as it helps predict weather patterns and wind conditions by showing the distribution of atmospheric pressure systems

Not much point going onto web chat rooms to find out things anymore, AI does it all so much better

 

[franksingleton]

I am not doubting that AI will be a valuable tool in weather prediction. I am trying to warn against over expectation. I note that ECMWF and the Met Office/Turing Institute are proceeding cautiously. I am also warning that the atmosphere itself is a limiting factor to deterministic weather prediction.

 

[Roberto]

ask an AI!!

As a curiosity (my first time ever to use AI tools) I asked both Chatgpt and "Claude" (a French AI tool)
"how to cross the ITCZ on a sailboat?", two friends had access to the professional versions.
"Claude" derived a lot of its descriptive content from Meteo France "Météorologie tropicale" textbook, whereas Chatgpt answer was mainly based on NOAA, fair enough; but, among Chatgpt suggestions there was "You will need the most accurate weather forecast you can get, like NOAA and [hear hear] Predictwind". Should anyone take this seriously ?

 

[GHA]

As a curiosity (my first time ever to use AI tools) I asked both Chatgpt and "Claude" (a French AI tool)
"how to cross the ITCZ on a sailboat?", two friends had access to the professional versions.
"Claude" derived a lot of its descriptive content from Meteo France "Météorologie tropicale" textbook, whereas Chatgpt answer was mainly based on NOAA, fair enough; but, among Chatgpt suggestions there was "You will need the most accurate weather forecast you can get, like NOAA and [hear hear] Predictwind". Should anyone take this seriously ?

Completely seriously, that's such a vague question you'll get far worse answers at happy hour.
To be most useful it seems best to give lots of context. keep asking & honing in to what you actually want to find, and perplexity gives you far far better links than google ever will.

"i am a long distance sailor on a 12m well found sailing boat. I aim to cross the itcz between cape verde and brazil. what questions should i be asking to ensure a safe trip"

"
As a long-distance sailor planning to cross the ITCZ (Intertropical Convergence Zone) between Cape Verde and Brazil, you should consider several key questions to ensure a safe trip. Here are the most important aspects to focus on:
Weather and Timing
When is the best time to cross?
The ITCZ is typically narrower and less active during winter and early spring3. Aim to depart from Cape Verde in late March or early April for more favorable conditions3.
How can I monitor the ITCZ's location and activity?
Regularly check weather forecasts and GRIB files, particularly as you approach 12°N latitude1. Use reliable weather routing services and satellite communication systems like Iridium Go with PredictWind offshore2.
Route Planning
What's the optimal route to minimize time in the ITCZ?
Consider staying north of the ITCZ for as long as possible, making most of your westing in the northeast trades3. Aim for an intermediate waypoint between 03°N and 07°N at about 130°W, adjusting based on current conditions3.
How can I best utilize the Equatorial Current?
Be aware that the Equatorial Current can significantly impact your course. It may run at 2.5 knots or more, setting directly west4. Plan your route to account for this strong westward push.
Boat and Crew Preparedness
Is my vessel fully equipped for tropical conditions?
Ensure all systems are operational, especially those related to weather monitoring, communications, and storm tactics3.
How prepared is my crew for potential challenges?
Train your crew in handling squalls, reefing quickly, and maintaining a proper lookout. Discuss strategies for dealing with prolonged periods of light winds or sudden storms3.
Contingency Planning
What's my strategy for dealing with prolonged calms?
Be prepared for periods of very light winds (0-5 knots)1. Consider your fuel capacity and consumption rate if motoring becomes necessary.
How will I handle potential squalls and thunderstorms?
Develop a clear plan for reducing sail quickly and securing loose gear. Be ready for sudden wind shifts and increased wind speeds up to 30-40 knots in squalls1.
Post-ITCZ Considerations
What conditions can I expect after crossing the ITCZ?
Be prepared for potentially strong headwinds and currents along the Brazilian coast, especially if crossing later in the year (June/July)4.
How will I maintain my easting for a favorable approach to Brazil?
Plan your route to retain enough easting for a good wind angle when you reach the southeast trades, which can significantly impact your landfall options4.
By addressing these questions and thoroughly preparing for your voyage, you'll be better equipped to handle the challenges of crossing the ITCZ between Cape Verde and Brazil. Remember to remain flexible and patient, as conditions in this region can change rapidly3."

And from asking the same question but telling perplexity to focus on social media >

As a long-distance sailor planning to cross the ITCZ (Intertropical Convergence Zone) between Cape Verde and Brazil on a 12m well-found sailing boat, you should consider the following questions to ensure a safe trip:

1. What is the best time of year to make this crossing, considering weather patterns and hurricane seasons?
   1
2. How can I obtain reliable weather forecasts and updates during the journey?
3. What are the prevailing winds and currents in this region, and how might they affect my route?
   1
4. How much extra fuel should I carry in case of prolonged periods of calm in the doldrums?
   1
   2
5. What safety equipment and communication devices should I have on board?
6. How can I prepare for potential equipment failures or maintenance issues during the crossing?
   2
7. What is my plan for crew management, including watch schedules and conflict resolution strategies?
   2
8. How can I ensure adequate provisioning for potentially extended periods at sea?
9. What are the nearest safe harbors or ports of refuge along my route?
10. How can I prepare mentally for the challenges of long-distance offshore sailing?
    2
11. What specific navigational challenges might I encounter in the ITCZ region?
12. How can I optimize my sail plan and boat setup for potentially variable wind conditions?
    1

Remember to consult resources like "World Cruising Routes" by Jimmy Cornell for detailed information on this specific crossing
1
. Additionally, consider seeking advice from experienced sailors who have made similar passages and studying recent crossing trends in this area
1
.