Synoptic charts, GRIB, stuff you should know...

[simonjk]

Hi Chaps,

I've posted elsewhere in the forum responding to a question about synoptic charts. This is a common question and so I thought that it might be useful for you to know a little more about where the forecasts you see and hear come from and are produced.

Firstly, remember I am an independent forecaster and do not work for the Met Office. I will try to be as impartial as I can be. I also can't miss saying that there is more information about this in my book which is going to be published on Friday, 'The Sailor's Book of the Weather'. Buy it now! /forums/images/graemlins/smile.gif

On with the post, and apologies for the length.

The charts you see that are issued by the Met Office, such as these, are produced by human forecasters. What happens is that the duty forecaster is presented with a best guess of pressure and frontal positions (both predicted by the computer model). He can then amend the position of fronts, highs, lows etc... and can also input some false data in order to make the forecast look more realistic (sometimes the raw output model just looks ;wrong'). Once he is happy the forecast chart is issued. Essentuially these charts are based on output from the UKMO model, but the forecast will take acount of what the GFS, DWD, GEM, Ensemble, ECMWF and other models are doing too. Generally though, up to 24-hours ahead there is very little difference between them.

What you see on the internet at sites such as Metcheck is raw forecast data from the computer model. It has not had intervention made too it by humans (although any rogue observations may have been spotted). The model is run and then this data coded and issued. Remember that the resolution of the model is still relatively coarse. On the Metcheck website I note it says that the resolution is 0.1deg, but the GFS documentation says the best resoltuion is 0.5deg (i.e. 30nm between data points). I suspect (and I could be wrong, perhaps someone can correct me) that 0.1 deg is an interpretation between model points? However, this doesn't divert from the main point that what you see here is raw output.

Anything that is beyond the resolution of the model is only interpretation, although this may be improved by improvements in the underlying topographical model.

Basically, use the rule of thumb that the most common sites on the internet giving multiple location forecasts are using the raw GFS, but may be applying some additional technique to it to extract forecasts between grid points.

I'll stop now because this could get really dull for you, but it might just give you something to think about.

Best wishes,
Simon

 

[mixmaster]

Thanks Simon. I've always wondered why synoptic charts (or at least the pressure gradients) that have the human intervention from various met offices are not converted to GRIB format and then sent out? Why is it that only the raw computer data is available this way? Are there any plans to add this feature?

 

[simonjk]

I suspect it is a matter of programming. You can actually get the analysis and t+24 hour forecast chart in a format which can be decoded in a program called Digital Atmosphere. I use this on my website at www.weatherweb.net

Best idea though is to try and draw the fronts in yourself. After some practice this becomes fairly easy, and you learn as you draw (I still learn something new when drawing the charts everyday).

Simon

 

[Clyde_Wanderer]

Simon.
Does your book cater for the complete novice in its contents, ie does it explain all the abreviasions etc that you have used in your post above? as they are double dutch to me, but I l find that I am constantly watching the signs of the weather around me and try a prediction now and then, sometimes not bat and sometimes totally wrong.
What I would really like to do is to be able to read the clues in the sky around me and make some reasonable and consistant short term predictions, and I would like to be able to decipher the synoptic charts, which at the moment I havent a clue about.
Do you think your book can teach me all that?
Good luck with the sales.

 

[captainslarty]

About time ya did something to educate the great unwashed lol.. shame its not free though Simon...

Can I suggest Dashew. Mariners weather.. no need to reinvent the wheel..

Or, dag pike....

 

[simonjk]

Yep, I hope the book has something for novices and experts alike. For example, there's a chapter on clouds and forecasting from them, a chapter on high and low pressure, why and how they form, fronts, and another chapter on DIY forecasting. I think you would find it useful.

Simon

 

[metcheck]

Hi Simon,

Good luck on the book, looks like an excellent read and the more we can educate the general public into the extent of weather forecasting the better.

Just to add a little onto the Metcheck bit you mentioned. Yes, we do use raw data from the GFS and we interpolate it down to 0.1 degrees which in an ideal world isn't the best solution as interpolation can sometimes distort the true figures (especially around coastal regions).

Saying this, we also have an algorithm which kicks in after 72 hours which takes a look at the ensembles and will tone up or down certain features like excessive rainfall or temperature trends to ensure the model looks at the "most likely" outcome as opposed to the deterministic one.

Bet we've scared everyone off now ;-)

Cheers,

A

 

[simonjk]

Nice one Andy, and you folks who are reading this, this is REALLY important information and well done Andy for telling everyone...a pat on the back there for Metcheck.

Why? Well, because Andy has now told you HOW the GFS is used at Metcheck you are better armed to interpret the forecasts. What metcheck are doing is looking at the GFS and saying okay, if the weather parameters at grid point A are this, and at point B are that, then a mean between the two is C. This doesn't increase accuracy of the model but means that forecasts can be more specific to ones location. Interpolation is okay, so long as you know it is happening and can amend your own DIY sailing forecasts accordingly.

Interesting Andy that you are using an algorithm on the ensembles after 72-hours. is it intelligent or static? This is another important note for sailors as it is a an example of how we, as forecasters, treat forecasts beyond 3-days, i.e. with EXTREME caution! But, guidance can be given so long as as many forecast models are viewed as possible. This is where an ensemble of models would be useful (sorry for the technical speak).

No-one should be 'scared' by this as all that is happening is that we are making the users of forecasts more aware of the falibility of the models and forecast based upon it. There is no way that a human forercaster could sit down and produce 3-hourly forecasts for thousands of locations on the planet out to 384-hours ahead, so models are the best way to do this.

All the sailor (and other users) have to remember is that the forecast is exactly that, a prediction using imperfect equations and techniques that can and do go wrong. That's where human forecasts come in (thank goodness /forums/images/graemlins/wink.gif )

Simon

 

[metcheck]

Hi Simon,

You're totally right. The power of weather models such as the GFS is astounding, not just that but they provide detailed global weather (and wave) forecasts for free out to 384 hours.

I don't think the algorithm could be classed as intelligent as I wrote it lol, but it works on a system where if certain values for say rainfall are greater than 5mm, then it will check the other 20 ensemble members and see the chance of this happening. If more than 10 members agree then it will go with the figure, otherwise it will nudge the figure down to the ensemble mean.

It's not an ideal solution, but in a very simplistic way it mimics what human forecasters do.

Every morning we scan through not just the GFS, but the UKMO, ECMWF and DWD to get a general picture of how each model is expecting the next few days to develop.

Sometimes (take Storm Johanna a few weeks ago as an example) few of the models agreed on the intensity and track of the system. So we delve into the ensembles to see how confident the GFS is of the track and intensity. Gradually, as the storm got closer, so the models started to agree and therefore our confidence increased on the most likely outcome.

Then, this weekend with the cold snap, virtually all the models agreed it would happen from some 5-6 days out. Which means that we can be pretty sure that it will happen and issue forecasts based on that.

There will never be a substitute for a human pair of eyes looking at the charts. This is because we're aware of model biases and can weight each one accordingly, however as you rightly say, no forecaster would sit down and work out 3hr forecasts out to 8 days ahead for every location on planet earth hehe.

Cheers,

A

 

[BlueSkyNick]

This is all great stuff for those of us trying to develop our weather knowledege, but needs quality time to sit and contemplate rather than a cursory few minutes coffee break in the office!

However, your comment that There will never be a substitute for a human pair of eyes looking at the charts is an interesting one.

There are, no doubt, people who will believe that once the models and technology are sufficiently clever and reliable, that it's best to leave human judgement right out of the equation.

It's a bit like how my boat goes consistently faster when the autohelm is on, than when somebody is helming manually. /forums/images/graemlins/wink.gif

 

[MarkGrubb]

Interesting stuff, thanks Simon and Andy.

Simon does your book include a chapter that covers these issues: the different techniques, sources of data etc. used by forecasters, how they affect the forecasts and what we should know when interpreting those forecasts for sailing.

There are lots of different websites now offering forecasts and from this discussion it seems there can be subtle differences in the way these forecasts are produced: some use the raw data; some use forecasters to check and adjust the data; some use algorithms to check and adjust data; and many, do doubt, use combinations of all those to some degree.

It would be useful to see this information in a chapter, so it can be easily digested and understood by simpletons like me, to help understand how these different techniques affect the forecasts I use for sailing.

Thanks again,

Mark.

 

[metcheck]

Hi Nick,

Well, model resolution is getting better and there is talk of developing a neural network forecast system which "learns" from its mistakes, but this is all sci-fi stuff.

There is still two major drawbacks with numerical modelling :-

1-It's impossible to get all the weather information for everywhere in the world at any one moment in time.
2-Computing power is still a few miles from where it should be if we were to have a perfect numerical model.

In theory, if we could obtain every weather observation for every 10 metres around the world, then ran this at 5 second timesteps then weather forecasting would be stunningly accurate.

However, due to lack of observations over oceans and remote areas, plus the computing power required, it's virtually impossible.

Gradually, satellites are improving the amount of information, but it would still take around 4 days to run a model at that time step, by which point the forecast would be out of date and useless.

At present, the best fit is to check forecasts from 2 or 3 sources. If they agree then it's likely that will happen, but if not then investigate further, use ensembles to check the confidence in forecasts.

HTH,

A

 

[Cutter]

Fascinating stuff thanks

 

[Thistle]

Fantastic thread! Many thanks, guys!

 

[simonjk]

Hi again all,

There are more details about my book and a contents table at http://eu.wiley.com/WileyCDA/WileyTitle/productCd-0470998032.html I really want it to be a book that you can have in the cabin, or on the coffee-table and can dip in and out of as and when you want. I got my hands on the first copy of it today and I must say Wiley have done a super job; lots of photo's and illustrations and text that is easy to read (has to be easy for me). It's for novies and experts alike, hopefully all will learn something from it.

Anyway, enough of the plug, back to the posting. Of course one should never say never, but at least for the foreseable future humans still have an improtant part to play. Anecdotally this shows it's face to a meteorologist by on most days the models being accurate enough to follow fairly well, with only minor adjestments (although I stress a forecaster will look at many models, not just one). However, on occasions when severe weather is predicted the human forecaster comes into their own. Take the summer floods last year for example. While the Met Office model was pretty well spot on in both timing, amount and location of the heaviest rain, the GFS and French models were different in their predictions. The result was that the skill of the human forecaste rcame into play in deciding which of the models to follow most closely and in monitoring the ground truth observations against radar as the event unfolded. There are other examples of the winds being smoothed too much by models, such as September 2006 when the GFS predicted 20kts off Scillys, speeds actually reached in excess of 60kts! Again, the human forecasters intervention was vital here in upgrading the forecast winds.

Andy is quite right about the current status of numerical models, after all that is excatly what they are, models of the atmosphere, and they are imperfect. Think of the Butterfly Effect and Chaos theory and that pretty well sums up forecasting. If one could accurately model the atmopshere at every level in the vertical and horizontal, the topography of the earth down to the nearest metre, the effect of billions of insects flapping their wings, and the effects of heating on individual beaches, plus many more variables, then a near-perfect (note NOT perfect) model may be achieved.

Hope that hasn't confused all too much?

And just a footnote that it is vital that providers such as Metcheck state the source of their forecasts and what they do to them in order to enable users to evaluate them.

Simon

 

[wotayottie]

Interesting. If the resolution of the models is 30nm without interpolation , does that mean that you can only forecast at the most detailed level for areas more than 30nm apart? But more interestingly, as weather systems move over us how accurate is the time element of the forecast? It often seems to me that the forecast of the weather is more accurate than the prediction of when it will arrive.

I find the forecasts of windguru and windfinder very useful and remarkably accurate even 7 days out - are they simply the computer model data or do they come from met men interpreting the data?

finally, what use do you make of observations from ships and planes transiting the atlantic in building your models?

 

[simonjk]

Basically, from model forecasting point of view the answer is yes. All that is happening is that the model is predicting a set of parameters for grid point 30nm apart (in the case of a 0.5 degree resolution such as the GFS).

From ther time point of view many factor influence the systems to make them speed up or slow down, and it is the effect of the processes taking place within the atmosphere which case the acceleration or decceleration of individual weather systems, hence the reason you need such massive computers to model the atmopshere anything like accurately.

As far as I am aware Windguro and Windfinder are using pure model output (I am sure they will coreect me if I am wrong) and so they would be no more accurate than sites also using pure model output from the same model.

Observations are the foundation of any forecast. As I start a forecasting shift the first thing I do is look at a map of observations and analyse the chart buy drawing on fronts and isobars. This gives me an immediate understanding of the current weather situation, without this there is no point in even trying to forecast the weather. This is why the Shipping Forecast Metmaps are so useful as one can stil plot a chart and draw on isobars and get an immediate understanding of current weather.

You can also then comparte the chart to the forecast charts and decise whether the model is performing within what one would perceive as an acceptable margin of error (i.e.e is the position of a high or low way out?). This aids me in my evaluation of how reliable the mdoel is.

Am I ittering again?

Simon

 

[mormargh]

This is a very useful thread. I use the GFS meteograms from the NOAA, simply because they are easy to read, and you can select what they show (e.g. dew point with air temp enabling you to forecast visibility). In the week before I'm setting off, I get the forecast based on a point in the general area I'm sailing and also get one for the nearest Met Office reporting station that provides an hourly record (in my case Culdrose or Walton). I compare the forecasts to the actual and use the information to assess how much reliance to put on the forecast. Must point out that I believe the GFS model doesn't recognise the Isle of Wight so maybe not that good in the Solent!

For those who are interested in met, have a play around in www.westwind.ch which is quite some data source.

 

[simonjk]

Yes, the meteograms are useful aren't they? Good for a quick diagnosis of what is going to happen. Sounds like a sensible plan to me.

You are right that the GFS doesn't see the Isle of Wight, or Anglesey, Isle of man for that matter. Western Isles and Channel Islands are also poorly modelled (if at all). So, basically the forecasts in these areas take no account of the Islands.

Westwind.ch is a great source of data, I use it all the time.

 

[captainslarty]

Still vote for Dashews Mariners weather, sorry Simon.. but it has been about a bit and IS good..

Also, it explains all you need to know re synoptics.. armed with that, and a few respectable and trustworthy proffesional forecasts, you can 'roll yer own'

http://www.waypoints.com/dashew.html

Simon, you ARE doing a fantastic job, but, many offshore cruisers with serious miles under their keel have done this before.. I therefore suggest purely an alternative..