Weather Models

[franksingleton]

You say you are a data scientist. Could you please enlighten us by telling us what data and what Medellin have you done. That would enable us to judge your competence in proposing the existence of other techniques for weather modelling. My background is well known.

A truism you might like to bear in mind: to know about weather anywhere necessitates knowing about weather everywhere. Maybe an overstatement for the next few hours but there are teleconnections that make the kind of models that you probably use, impossible to apply.

 

[franksingleton]

Yes I’m sure he and his colleagues congratulate themselves every day on how clever they are. As I said though, in 10 years that knowledge will be worthless when someone comes along and shakes things up. That person probably won’t be within the profession. As demonstrated here any contrary thinking is quickly squashed and ridiculed, much like the medical community.

You really are condescending in pontificating on subjects that you clearly do not understand. Meteorologists/numerical weather modellers do not live on a different planet. They are well aware of the problems that I describe above. They are realists continually trying to improve modelling techniques, observation systems and data. The atmosphere is a physical system governed by the laws of physics. Unless you or some other genius can come up with alternative laws of physics, NWP is the only approach. Improvements can only come through refining models, improving observing techniques and better use of the data.

 

[lustyd]

Go on, tell me more about how it’s impossible to do better (I’m kidding. Don’t. )

 

[DFL1010]

All models are wrong, but some are useful.

 

[SaltyC]

Tin Hat time!
I'm afraid, I'm with Frank, he has many years (Decades) of experience in meteorology, those studying the subject have pushed boundaries to get better understanding of the weather and the and accuracy its prediction.

Computing power and predictions moved the game tremendously.

I will now go back to teaching basic navigation. We know the knowns - pressure, wind direction, the time of HW. However the predicted height and tidal streams are effected by unforseens (as is weather, but different players).

My personal thought is Weather, as with Navigation, is an artistic interpretation of a science. ie There are unpredicatbles, at which point you rely on experience to make a judgement.

Yes, Captain Kirk and Scotty may come along shortly, but that is not a given. we have to work with what we have, it does not stop us looking to improve.

 

[franksingleton]

Go on, tell me more about how it’s impossible to do better (I’m kidding. Don’t. )

It is easy to woffle on about better ways to predict weather. It is quite another cave to hazard a guess as to how. Give us a clue or stop criticising what you do not understand.

 

[franksingleton]

All models are wrong, but some are useful.

During my 10 year stint as a senior forecaster, we used to say that the only certainty in a forecast was the date. That assumed that you kept clear of the theologians.

 

[lustyd]

It is easy to woffle on about better ways to predict weather. It is quite another cave to hazard a guess as to how. Give us a clue or stop criticising what you do not understand.

There you go again telling me I don’t understand. You’re right nobody can know anything you don’t know.

For the record sometimes we can’t talk about trade secrets on public forums. Intellectual property is a thing.

 

[lustyd]

During my 10 year stint as a senior forecaster, we used to say that the only certainty in a forecast was the date. That assumed that you kept clear of the theologians.

Yes, because you were certain that it couldn’t be done better. Maybe if you had a better attitude you’d have more answers you were confident in.

 

[westernman]

A word of warning, no supplier of weather model data ever tells you that models have in-built smoothing so that effective resolution is about 5 grid lengths, regardless of model. Claims of high precision forecasts are advertising crap. ECMWF free data are only available on a 0.4 degree grid but the loss of information from the 0.1 degree data output is minimal.

Is the modelling done with a fixed grid?

I would have thought that there are good reasons for using a variable sized grid. There are some big areas where nothing much happens. But in some places 0.4 degrees one way for the other has some extreme consequences. E.g the Pyrenees mountains.

I sail a lot around the area where the Pyrenees come down to the coast. Weather forecasts for that particular 20 miles or so are particularly terrible.
But that 20 miles may be less than the grid size - in which case I cannot see how any model can get close to reality.

 

[Pye_End]

I have seen some people come into a business and make a fabulous game changing contribution.

However, I have seen far, far, far more newcomers who think they know better than the wealth of experience around them, and totally screw the whole thing up with their arrogant attitude.

 

[lustyd]

I have seen some people come into a business and make a fabulous game changing contribution.

However, I have seen far, far, far more newcomers who think they know better than the wealth of experience around them, and totally screw the whole thing up with their arrogant attitude.

I’ve seen quite the opposite. But I do work in an industry where we transform other industries. Often people trained in only one thing are very narrow minded and resistant to new ideas. When you cross train and are open to ideas it’s easier to be transformative.

 

[franksingleton]

Is the modelling done with a fixed grid?

I would have thought that there are good reasons for using a variable sized grid. There are some big areas where nothing much happens. But in some places 0.4 degrees one way for the other has some extreme consequences. E.g the Pyrenees mountains.

I sail a lot around the area where the Pyrenees come down to the coast. Weather forecasts for that particular 20 miles or so are particularly terrible.
But that 20 miles may be less than the grid size - in which case I cannot see how any model can get close to reality.

Météo France does run a global model with a variable grid length but it performs less well with other models according to objective tests on behalf of WMO. Models with short grid lengths are nested within global models taking their boundary conditions from the global model. Because of uncertainties in models and, to a lesser extent, uncertainties in data analyses, models have to filter out small detail. By small, I mea small relative to the grid used for computation. This is intended to prevent unrealistic development. In extreme cases, models ca go unstable, as frequently happened in the early days of operational NWP.

 

[johnalison]

I’ve seen quite the opposite. But I do work in an industry where we transform other industries. Often people trained in only one thing are very narrow minded and resistant to new ideas. When you cross train and are open to ideas it’s easier to be transformative.

I have an idea I would like to offer you. It is currently under wraps, but as a clue I can tell you that it involves seaweed.

 

[franksingleton]

I’ve seen quite the opposite. But I do work in an industry where we transform other industries. Often people trained in only one thing are very narrow minded and resistant to new ideas. When you cross train and are open to ideas it’s easier to be transformative.

You seem to forget that NWP is used in many countries worldwide and studied in many universities By people far more clever than I am. You seem to forget, also, that many disciplines are involved, mathematicians, physicists chemists, computer scientists, instrument technologists etc. There is plenty of scope for new ideas and new approaches.

Now, stop prevaricating. Give us a clue about a new, radical approach. As a data scientist you must have two ideas to rub together.

 

[laika]

Didn’t it used to be the case that ITV used weather models but the bbc used people who were genuine meteorologists?

 

[franksingleton]

All reputable media organisations use forecasts based on models. At some stage AI will be used to generate worded forecasts. To some extent, this is done now - Meteoconsult was a prime example. The BBC used to have a contract with the Met Office to provide presenters. The BBC uses a commercial firm to provide presenters, some of whom are not trained as meteorologists. However, the information is largely Met Office and ECMWF based.

 

[Mister E]

Yes, because you were certain that it couldn’t be done better. Maybe if you had a better attitude you’d have more answers you were confident in.

So how can weather forecasting be improved?
Don't bother with some consulting management bollocks but actual facts.

 

[boomerangben]

I think it is sad that Lustyd has started a really interesting and what might a really beneficial thread and ruined it by his single mindedness. . His thread, his rules I suppose but I thought this was a forum for the benefit of us all.

As for the weather, I wonder how many of you who use tabular weather data such as wind guru and so actually stop and look at synoptic charts and how they evolve over a period of time. At the minute (or yesterday’s weather - not looked at this mornings weather yet) had a slack area of low pressure over Scotland. The wind forecast was slightly different as we flew north clearly showing the challenges in predicting the pressure gradient in such a system. When I cycled home from work, I started in the dry and 300m later the road was soaking wet from a recent shower. That shower might have been triggered by a farmer ploughing a field miles away triggering a cumulus cloud that then formed a CB of sufficient size to cause local wind effects and so on. Weather is reported in probabilities. Aviation forecasts have multiple layers where different weather has different probabilities. All it takes is a slight change for a weather effect to change its level of probability and whether it falls off the forecast altogether. If models are being run constantly then the forecast will change. I think that might be this issue here. Tabular weather data might be running weather probabilities of say 80% and not reporting weather that probabilities are say 40%. A small change in the model might switch the two weathers in the next forecast.

As for Lustyd’s assertion that weather forecasting will change in the 10 years, I am sure that AI will change everything in life. But the problem I foresee is that because we expect weather forecasts to be cheap, we will rely less and less data derived from organic matter. We see fewer and fewer trained and experienced met observers in aviation, replaced by a laser looking straight up and three visibility machines looking across a metre or so of air. In the old days a human would look across the airfield at the fog bank rolling in from the sea and change the weather report, but the machine is telling Exeter that it’s going to be glorious. You’re in data Lustyd, tell me how a future of poorer data will result in better modelling……

 

[laika]

All reputable media organisations use forecasts based on models.

Hardly GQ but I suppose I may have seen John Kettley amongst those pictures in the window of barbers’ shops.