Weather Models

[lustyd]

OK what the heck is happening? Every few hours I'm seeing wild swings in predictions from good weather to storms with 8mm rain. It would be fine if this was a week out but 2 days in the future shouldn't be this unstable, surely?
For a while I thought weather modelling was improving but it's looking like I was wrong and we're back to almost complete guesswork. I can only assume the data the models trained on doesn't match with what's happening this year. Perhaps we should be getting confidence ratings, or even show blanks if the weather on a given day flip flops so much as this could be dangerous.

 

[capnsensible]

Morning. For the big picture have you tried passageweather.com?

 

[lustyd]

Morning. For the big picture have you tried passageweather.com?

Also uses GFS so the issues will be the same. Example here for West Wittering this Saturday. A few hours ago the model showed no rain at all for the whole weekend and now 5mm which is very significant rainfall. By this evening this will likely change to no or almost no rain whatsoever again. Wind speeds are just as changeable this summer.
I'm not complaining about the weather, and I'm not complaining about the weather websites, I'm stating that the weather models appear to be completely broken right now and spewing out inconsistent crap rather than relatively stable information that gets more reliable as we approach a time and date.

 

[Minerva]

They have been all over the place this season. Case in point yesterday I looked at the forecasts for Oban this weekend on windy which shows all 5 model systems.

One model had a northerly / One had a southerly wind. One model had 7kts another had 34kts.

Who knows what we’ll get.

 

[RobbieW]

There was a thread on this somewere the other day; the main thing I remember is that there is something unusual happening around Biscay that is throwing the models.

 

[capnsensible]

I reckon that the predictions for very localised weather forecasts change a lot with any small change in the big picture.

Bit like a container ship altering a few degrees on your ais changes cpa suddenly.

Personally I do look at my local stuff on yetanotherweatherapp.....windfinder. but I always look at passageweather to see what's going on at various magnifications.

 

[RobbieW]

Found the reference,

, thank you @Stemar

 

[FairweatherDave]

Indeed thanks Stemar, I watched that yesterday. It does feel unusual to not know what the weather is going to do, and shows how we have got used to consuming excellent forecasts on the whole. We'll get what we get. Be prepared....... in my case do I need waterproofs for Victorious festival and the sail to get there . Kasabian in full Musto

 

[Daedelus]

I wonder if it's because with climate change the weather is getting more extreme and violently changeable. That's my understanding of what is likely with it. This would make it less predictable.

 

[KeelsonGraham]

I think the models hve been particularly bad this year. Little to no agreement between them outside 2 days. Last year, the 5-day forecasts were pretty good. It’s a sign of just how unstable the atmosphere has been in mid latitudes this summer.

 

[LittleSister]

For a while I thought weather modelling was improving but it's looking like I was wrong and we're back to almost complete guesswork. I can only assume the data the models trained on doesn't match with what's happening this year. Perhaps we should be getting confidence ratings, or even show blanks if the weather on a given day flip flops so much as this could be dangerous.

Nothing much to do with modelling shortcomings, really, and certainly not to the point where it's purely guesswork. It's more the unpredictability of particular type of weather we're currently experiencing, inconvenient that may be for your passage plans. Just sometimes the weather is especially unstable, and hence resistant to prediction, even at short time ranges. It's your expectation of always being able to be reasonably confident of the weather days days in advance that is the problem, not the modelling which works impressively well as a general rule.

UK weather is generally more changeable than perhaps most places in the world, but occasionally it is especially so, and in such conditions you can't expect certainty about the outcome, even just a few hours in advance. . In such cases slight differences in the geography and intensity of different factors will be highly significant, and the weather experienced in one place will be much different to other places not far away, and the weather at that place may well change very quickly.

Apply different models to such situations and they will give significantly different results, because they each give greater weight to some factor or other, and each update is likely to produce different results to the previous. It's the weather at 'fault'. not the models. It's the use of models that informs us that the aether is currently especially changeable.

When the models converge, it is because of the stability, and hence predictability, of the weather, and the more unstable the weather the greater divergence between models and the more they will each vary with each update.

I think the Met Office's 10 Day Forecasts and 'Deep Dives' (such as Robbie W's linked to) are brilliant at explaining current situations and indicating the degree of confidence that can currently be given to current expectations.

Look on the bright side, if UK (area) weather was always predictable, we'd have a shortage of things to talk about!

 

[lustyd]

I don't necessarily need them to agree, but it would be nice if each model was relatively stable!

 

[LittleSister]

I don't necessarily need them to agree, but it would be nice if each model was relatively stable!

The models would not be any use at all, would they, if they were stable when the weather was not!

 

[lustyd]

The models would not be any use at all, would they, if they were stable when the weather was not!

My point being that the input data isn't changing that rapidly so the output shouldn't vary quite so wildly. Obviously the models will gradually shift towards accuracy, but the flip flopping between lovely summer evening and the apocalypse every few hours should be filtered out since it's very clearly unreliable.

I should add that I work in data science - I'm not just bashing this for the hell of it, there is clearly bad practice/laziness here or at the very least hiding critical information from consumers of the output, which is just as bad as bad practice.

 

[lustyd]

4 hours after the previous screenshot. That's the difference between "build an Ark" and "beergarden will be nice"

 

[lustyd]

OK I just spotted the problem and capn was probably right with using a more generic service. Windguru had inserted their in house model above the rest and that seems to calculate on the fly using the various other models. GFS has actually been relatively stable so I've reorganised the page so that's at the top.

 

[Aja]

They have been all over the place this season. Case in point yesterday I looked at the forecasts for Oban this weekend on windy which shows all 5 model systems.

One model had a northerly / One had a southerly wind. One model had 7kts another had 34kts.

Who knows what we’ll get.

It's been all that so far!

 

[KeelsonGraham]

My point being that the input data isn't changing that rapidly so the output shouldn't vary quite so wildly. Obviously the models will gradually shift towards accuracy, but the flip flopping between lovely summer evening and the apocalypse every few hours should be filtered out since it's very clearly unreliable.

I should add that I work in data science - I'm not just bashing this for the hell of it, there is clearly bad practice/laziness here or at the very least hiding critical information from consumers of the output, which is just as bad as bad practice.

You’re right. It’s been almost impossible to plan anything with any confidence this season. But it’s the weather - not the models.

 

[lustyd]

But it’s the weather - not the models.

No, it's definitely the models. I don't mind changeable weather, as I said previously. The models definitely need improvement regardless what angle you care to take. It's a hard problem, sure, but that doesn't mean we can't do better, and it doesn't mean we should hide behind that complexity and pretend everything is fine. If the model is very clearly producing BS then it should be labelled as such and withdrawn, or at the very least have a big visible confidence rating enforced in any usage.
What we're doing right now is misrepresenting the situation and pretending we have a prediction of the weather. As I said above that's inconvenient to some and outright danger to life for others so no, I do not accept that there isn't a problem.

 

[KeelsonGraham]

Nope, it’s the weather. Sometimes the trigger conditions are so numerous and the options for divergence in different directions so plentiful, that there is no model or supercomputer on the planet can can provide a reliable forecast. Nor is there ever likely to be one.

Take, for example, a slowly building Azores high over the UK. I can pretty much guarantee you clear skies, haze and low winds for up to 5 days. This is because this particular weather pattern is very stable.

On the other hand, predicting the locations of TS off the West African coast in summer is nigh on impossible.

Similarly, the very disturbed conditions we’re having this summer are due to the boundary between polar and tropical air being much further south than usual. This boundary generates lows almost randomly. You can be certain that a low will be generated, but not exactly where. Nor can you be certain whether the low will continue to deepen, stay the same, or fill in. All that depends on innumerable other factors. Normally in UK summer most of these pass us to the North so we don’t notice them Or bother about them. Not so this year.

The principal difference between each model is the assumptions about the initial conditions. Usually each model is run several times over with slightly different initial conditions. If there is a lot of conformity in the outcome then the forecasters will be more confident of the reliability of the forecast. If even very slight changes in the start assumptions cause large changes in the outcome then the chances are that forecast is unreliable.

The models can never promise certainty, only probabilities. Perhaps these should be published daily. On the other hand, careful study of various models in the days preceding your cruise will reveal either a relatively stable forecast and strong correlation between the models (high probability of accuracy) or highly changeable forecasts and low conformity between models (low probability of accuracy). It’s the latter we’ve been seeing this summer.