zyGrib has been forked and is now continuing as XyGrib

[davidgal]

The well known open source grib reader / weather visualization software named zyGrib has not been updated since 2016. As there is community pressure for good free and open grib readers, the project has been forked and will now continue as "XyGrib" with a new team and new home site https://opengribs.org. All activity is open source and entirely without any commercialization.

The vision of OpenGribs.org (sister site to OpenSkiron.org) is to bring together the best free and open Grib reader / weather visualization software with multiple sources of quality free and open weather forecast data in Grib form.

The user community will be provided with support on the site's forum in regards to using the XyGrib software as well as the legacy zyGrib. In addition the support forum will deal with issues related to Numeric Weather Prediction models and Grib files in general.

 

[franksingleton]

Thanks for the heads-up. Is it your understanding that zyGrib will continue to be available? It is an excellent resource. I am sure that improvement will always be possible but, IMHO, I can see little need to change the current zyGrib at present. If it isn’t; btokrn, don’t fix it.

 

[Sandy]

the project has been forked

I'd be interested to understand that the above phrase means.

 

[davidgal]

Hello Frank,

The zyGrib website will continue to operation and the software in its last version 8.0.1, is available. However Jacques Zanitetti, who deserves no end of praise for bringing the software to where it is today, has indicated that he will no longer advance the software. He will continue to maintain the GFS grib server which serves zyGrib and others.

I differ from you on the need to change the current zyGrib. It does not handle Grib2 formats for sea data. It also does not handle all the new Gribs that are coming available with the new EU laws on open data. I can point to DWD-ICON Global, DWD-ICON Europe, DWD-EWAM, DWD-GWAM, DWD-COSMO, Netherlands Harmonie, Norwegian Hirlam and more.

OpenGribs.org will be launching an on-the-fly Grib server to provide most of the above integrated in the follow on of zyGrib - XyGrib. The current zyGrib only has GFS integrated in the package. We both know how good that is!

The underlying concept is that a Grib reader & visualizer is only as good at the Gribs it can read. Likewise Grib files from any particular source are only as good as the package that reads and visualizes them. This is the essence of the drive behind this new site - to fuel the symbiosis of providing the best free and open Grib reader together with the best free and open data in Grib form.

David

 

[davidgal]

Hi,

I'd be interested to understand that the above phrase means.
Read more at http://www.ybw.com/forums/showthrea...-now-continuing-as-XyGrib#mumSFPo02uyxi8xx.99

It is a well used term in the open source world.
The GNU General Public License allows anyone to branch off on a new direction with the original project source code - This know as forking . The condition for this is that the terms of the license cannot be changed. The new product must remain open source and it may be forked in the future.
David

 

[Sandy]

Hi,

It is a well used term in the open source world.
The GNU General Public License allows anyone to branch off on a new direction with the original project source code - This know as forking . The condition for this is that the terms of the license cannot be changed. The new product must remain open source and it may be forked in the future.
David

Thanks David. It was a new term to me, been a while since I looked at Open Source stuff.

 

[franksingleton]

Hello Frank,

The zyGrib website will continue to operation and the software in its last version 8.0.1, is available. However Jacques Zanitetti, who deserves no end of praise for bringing the software to where it is today, has indicated that he will no longer advance the software. He will continue to maintain the GFS grib server which serves zyGrib and others.

Thank you, David. I agree 100% Jacques has done a great job. I do not know if he ever sees this Forum but would like to record my admiration for what he has achiebed.

I differ from you on the need to change the current zyGrib. It does not handle Grib2 formats for sea data. It also does not handle all the new Gribs that are coming available with the new EU laws on open data. I can point to DWD-ICON Global, DWD-ICON Europe, DWD-EWAM, DWD-GWAM, DWD-COSMO, Netherlands Harmonie, Norwegian Hirlam and more.

OK. As Sandy is a “cynical Scottish engineer”, I am probably a “cynical English meteorologist.” I take your point for some of the products that you mention.

For the rest, I would not bother with most. The only service that I would really like to see is the UK UM and, probably, their 4km model. There is no virtue in comparing different global models in real time and the GFS is at least as useful as any other global model freely available. No doubt NCEP will migrate to a non-hydrostatic model with a 10 km, or smaller grid, sooner or later. As regards LAMs, bearing in mind the short lifetimes of small weather detail, for my sailing, I cannot say that I ever see a need for other than a global model. But, OK, for those that believe in their usefulness – racing sailors, for example - I can see a benefit in being able to use HIRLAM etc.

I can see endless rather nugatory arguments about the virtues of one model vis a vis the rest. I know that different groups will inevitably develop slightly different LAMs but the majority of lay users will forget, if they ever realised, that the main constraint in weather prediction is the atmosphere itself. Further, many sailors and some providers of forecasts do not realise (or will not acknowledge) that models filter out wavelength smaller than about 5 grid lengths.

OpenGribs.org will be launching an on-the-fly Grib server to provide most of the above integrated in the follow on of zyGrib - XyGrib. The current zyGrib only has GFS integrated in the package. We both know how good that is!

The underlying concept is that a Grib reader & visualizer is only as good at the Gribs it can read. Likewise Grib files from any particular source are only as good as the package that reads and visualizes them. This is the essence of the drive behind this new site - to fuel the symbiosis of providing the best free and open Grib reader together with the best free and open data in Grib form.

David

I would add that GRIB files from any source are only as good as the data that are input and how well those data are analysed. There are sources of GRIB and GRIB products, no names no pack drill, that run LAMs with no detailed data.

Despite my cynicism, I welcome your initiative. I look forward to seeing it in operation. As you are not selling a service you will not, I hope, be making claims of unattainable accuracy or superiority of any one service versus the rest.

 

[davidgal]

Despite my cynicism, I welcome your initiative. I look forward to seeing it in operation. As you are not selling a service you will not, I hope, be making claims of unattainable accuracy or superiority of any one service versus the rest.

Thanks for the good wishes.
I agree with your closing point as to not trying to make claims on accuracy of forecast data. In my opinion, the best way to provide valuable service to the sailing community is to provide several optional sources and let everyone decide what works the best for him or her.

 

[Joker]

I've downloaded it and given it a try.

Windows 10 said 'This is an unrecognised app! Don't do it!'. But I did.

The German Hydrographic Office produces GRIB files for the currents in the N Sea, Elbe and Baltic at ftp://ftp.bsh.de/Stroemungsvorhersagen/grib2/

I can't find any program that will display properly [OpenCPN comes the closest] and they are potentially very useful.

Zygrib doesn't, and neither does Xygrib. Any chance of this changing in the near future?

 

[franksingleton]

Thanks for the good wishes.
I agree with your closing point as to not trying to make claims on accuracy of forecast data. In my opinion, the best way to provide valuable service to the sailing community is to provide several optional sources and let everyone decide what works the best for him or her.

David. I have sent a PM to you.

 

[franksingleton]

Thanks for the good wishes.
I agree with your closing point as to not trying to make claims on accuracy of forecast data. In my opinion, the best way to provide valuable service to the sailing community is to provide several optional sources and let everyone decide what works the best for him or her.

David, I am minded of the early days of commercial TV in the U.K. and hearing someone say “They speak very well of it on TV.” That syndrome is alive and kicking today. I have no problem with sailors using whatever GRIB service they find easy to access and with a presentation that they like.
I worry when I see words like

“....exceptionally accurate
....high-resolution
....precise high quality,
Incredible detail....
....down to 1km resolution.”

When they think about it, most sailors will realise that weather is simply not precise; that claims of meaningful prediction on such small scales are nonsensical. I hope that, when they become readily and easily available, access to HIRLAM, HARMONIE, ICON, etc will put some of the claims into perspective. That is why I wish you well.


I am sure that you will agree that a current major lacuna is the U.K. UM and 4km models. I live in hope. Meanwhile, I will make a prediction that will make me seem a “Cynical English meteorologist.” That is that we will never be able to make accurate westher forecasts on either global or local scales. Forecasts will become increasingly useful but never reliably accurate or precise.

 

[davidgal]

The German Hydrographic Office produces GRIB files for the currents in the N Sea, Elbe and Baltic at ftp://ftp.bsh.de/Stroemungsvorhersagen/grib2/

I was not aware of this resource. I'll take a look and include it in the next update.

David

 

[JumbleDuck]

That is that we will never be able to make accurate westher forecasts on either global or local scales. Forecasts will become increasingly useful but never reliably accurate or precise.

I think you are too pessimistic, but time scales are crucial. Accurate forecasts a year ahead? Never. Accurate forecasts ten minutes ahead? Sure, if you have the density of data needed.

At gliding competitions it is (or now maybe was) customary to send a tug to windward with a sonde before flying started. Suitably processed, that would give spot-on forecasts for cloudbase, convection, thermal strength, cover and so on for a few hours ahead, over a very small area.

 

[franksingleton]

I think you are too pessimistic, but time scales are crucial. Accurate forecasts a year ahead? Never. Accurate forecasts ten minutes ahead? Sure, if you have the density of data needed.

At gliding competitions it is (or now maybe was) customary to send a tug to windward with a sonde before flying started. Suitably processed, that would give spot-on forecasts for cloudbase, convection, thermal strength, cover and so on for a few hours ahead, over a very small area.

I am being realistic, not pessimistic. Yes, obviously it is a matter of time and space scale. Your 10 minutes is in the nowcasting range. Here, we are talking about weather forecast services that can be distributed to sailors or even provided on a tailor made basis to a specific user.

If we could measure the atmosphere accurately on a 1km grid and if we had a perfect weather model, then by the time the data had been collected, fed into a computer, a forecast run and the information issued, then the smallest details defined by that grid (circa 4 or 5 km) would have disappeared. Even if the time-scale of the mechanics could be reduced to tens of minutes rather than a few hours, predictability would still be such that a forecast would have limited usefulness. As a sailor you will know well that even away from land, there are structures in the atmosphere well under 4-5 km.

Small weather details have short lifetimes. The extreme is a gust of size around 100m. There is no way of predicting a gust explicitly until it is about to hit you. Models such as HIRLAM and the U.K. LAM can be fed with radar data to describe an individual thunderstorm cloud but cannot predict well how that cloud will change over the next few hours. A thunderstorm cloud will be, typically, 10-20km in size. Your glider flying a sonde might work well on some occasions but not all by any means. You might have a front 30 miles away moving at 30 knots. In two hours time, the weather would bear little relationship to your sounding.

On the larger scales, lows, highs and fronts can be fairly well forecast but the noise in the atmospheric system means that detail, even hours ahead, will not be accurately predicted. It will be usefully predicted but that is not the same as accurately. As a forecaster I used to say that the only certainty in any forecast was the date - as long as you kept the theologians apart.

 

[JumbleDuck]

On the larger scales, lows, highs and fronts can be fairly well forecast but the noise in the atmospheric system means that detail, even hours ahead, will not be accurately predicted.

So ultimately, and without wanting to be contentious, whether "accurate" forecasts ere aver available depends on how you define (one defines) accurate. I suspect that it's a Heisenberg-like system:

delta T / delta X ~ C/P​

where delta X is the distance scale over which you'd like to resolve, delta T is the time for which you'd like a useful forecast, C is the desired accuracy and P is the processing power you have available. For practical purposes, I'd say that C/P is about unity at the moment: forecasts six hours in advance are pretty good if you don't need to resolve better than six miles and forecasts a day ahead are fine if you accept that the world is divided into 25 miles wide pixels. There may well be better forecasts ahead if you pay - those are my impressions of public forecasts.

Twenty years ago I was camping at a glider site (Le Blanc) in central France. It was a bakingly hot day, with very unstable air, and when I struck my tent it started a dust devil which tore across the campsite, sent the bedding and possessions of some nice Dutch people a hundred feet in the air and then grew into a rather impressive thermal. Ten minutes later there were half a dozen gliders in it, 3000' up. I should have charged them

I agree that effects on that sort of scale will never be predictable just as - and for very similar reasons - it will never be possible to predict the flow patterns in boiling liquids. Far too many variables and the system is almost certainly chaotic anyway. Similarly, computational fluid dynamics can predict lift and drag in turbulent systems very well, even though the actual flow is random.

You might have a front 30 miles away moving at 30 knots. In two hours time, the weather would bear little relationship to your sounding.

The tug would typically fly direct upwind for at least 50nm to look for this sort of thing. The aim was to choose an optimal launch time for gliders which might then fly 100km away and back. And of course it was combined with normal weather forecasts. A front wouldn't have been a surprise.

 

[franksingleton]

Jumble Duck. You are correct in that much depends on how you define accuracy. Some providers of forecasts claim great accuracy and/or precision in their forecasts even, sometimes, that they are the most accurate. Even some users make similar statements. My reaction is to ask how they define their terms and how they measure them.

Just about every maritime nation from the largest to the smallest issues offshore/sea area and coastal/inshore waters forecasts. All are similar to the often derided U.K. Shipping and Inshore forecasts. A paper
https://rmets.onlinelibrary.wiley.com/doi/full/10.1002/met.1341 Is a study by the Met Office of the “accuracy” of wind in inshore forecasts. As we all know, they can be grossly inaccurate for a particular location but they are clearly useful in a general sense.

Your formula has very limited application. Likewise your gliding example. Such a reconnaissance flight would confirm the large scale forecast; for example, “A cold front will clear by 1000 hours, there will then be good convective soaring conditions”. It would not enable an assessment of when or where the first showers would occur. It would not tell you where showers would be in 6-12 hours time. It would give a good indication of gust strengths but not where and when gusts would occur. You, as a scientist, would not expect that degree of accuracy/precision but that is what some providers of forecasts claim.

As regards cost, a non-trivial forecast for a short time (say 3+ hours ahead) for an area as small as you like, requires a meso-scale model. These models are nested within larger area models. Take a look at the Met Office app showing radar images of rain and predictions using a highly detailed model fed by a highly detailed 4-d data analysis, including radar data. These forecasts are produced twice a day at 0300 and 1500 UTC. While writing this, I looked at the actual and predicted rain areas for 0700 UTC. For the most part, they were good, “Very good, in parts.” But, not totally accurate. By midday, 9 hours after the initial data time, they will be less good. Those highly detailed forecasts are only possible using mega computing power with the attendant costs.

I wish that there was some way in which the user community and providers could get away from discussing “accuracy.” On our summer cruise ~ 4-5 months, we get great help from forecasts several days, 7 or 8, ahead. They are certainly useful in that we do not generally find ourselves stuck in a place that we do not want to be in. On the other hand, in the short term (6-12 hours) we can, very occasionally, be wishing that we were in port.

It goes without saying that meteorologists necessarily have to assess accuracy of models in order to see if upgrading has actually had a beneficial effect. Such assessments, eg RMS of surface pressure and RMSVE of wind at 500hPa, do not resonate with users at sea level. Accuracy and usefulness are not synonymous and that is really rhe basis for my assertions about accuracy.

 

[JumbleDuck]

Jumble Duck. You are correct in that much depends on how you define accuracy.

Thanks, and I agree completely with your later point that the lack of a common understanding of "accurate" causes problems.

Your formula has very limited application.

It's more of a mental model, and the notion that working to smaller scales or higher precision in either space or time costs computing power is pretty fundamental in numerical simulation.

Such a reconnaissance flight would confirm the large scale forecast; for example, “A cold front will clear by 1000 hours, there will then be good convective soaring conditions”. It would not enable an assessment of when or where the first showers would occur.

On the contrary, it was very good at predicting when the first showers would occur, because it gave very good prediction of likely convection and cloudbase. That was the point of it. Where the showers would occur was a different matter, of cou

 

[franksingleton]

Thanks, and I agree completely with your later point that the lack of a common understanding of "accurate" causes problems.

It's more of a mental model, and the notion that working to smaller scales or higher precision in either space or time costs computing power is pretty fundamental in numerical simulation.

I think that my problem with your conceptual model is that decreasing grid length (requiring more power) does not mean that you can predict increasingly smaller structures further and further ahead. There are physical limits set by the natural lifetimes of all meteorological structures.

As with global models, shorter grid lengths in three dimensions allow better defined initial analyses. Running ensembles from better defined analyses leads to a smaller spread in the computed outcomes. As a specific example, decreasing grid lengths will never mean that the Boscastle storm could by predicted explicitly 24 hours ahead. It should mean better warning of such a storm in that area. It should mean better prediction a few hours ahead. On the global scale, decreasing grid lengths will not enable explicit prediction of squall lines two or three days ahead. It will mean that ensembles will have a smaller spread.

On the contrary, it was very good at predicting when the first showers would occur, because it gave very good prediction of likely convection and cloudbase. That was the point of it. Where the showers would occur was a different matter, of course.

But, surely, your last sentence is the point that I am making. A shower cloud, typically 10-15 km across, is a significant feature. It can have effects on wind direction and strength over a substantial area and time. Hence my assertions regarding nonsensical claims by some “unofficial” modellers some of whom offer output claimed to be at 1 km resolution. Their output data may well be on a 1 km grid, but accurate?

 

[davidgal]

I've downloaded it and given it a try.

Windows 10 said 'This is an unrecognised app! Don't do it!'. But I did.

The German Hydrographic Office produces GRIB files for the currents in the N Sea, Elbe and Baltic at ftp://ftp.bsh.de/Stroemungsvorhersagen/grib2/

I can't find any program that will display properly [OpenCPN comes the closest] and they are potentially very useful.

Zygrib doesn't, and neither does Xygrib. Any chance of this changing in the near future?

Hi,
We have just released XyGrib version 1.0.0-beta-c for download. The mentioned Gribs are now readable in the update.
David

 

[JumbleDuck]

I think that my problem with your conceptual model is that decreasing grid length (requiring more power) does not mean that you can predict increasingly smaller structures further and further ahead. There are physical limits set by the natural lifetimes of all meteorological structures.

Of course, and I did say that, it being a chaotic system, there will almost certainly be absolute limits to the predictions can be made - as there are, for example, for planetary systems (more than three objects and you're stuffed) or compound pendulums. Are we at that limit for meteorology yet? As a layman, I doubt it. I expect that finer detail and longer ranges will both be possible ... and since finer detail runs into the problem of unpredictable Scottish glider pilots starting thermals, I think range is the way to go. Maybe an ultimate goal of forecasts a week ahead as good as we currently have for a day ahead? Better data, better algorithms (I knocked two orders of magnitude off the steps needed to solve some 3-d diffusion problems), bigger computers ... but to reiterate, I agree completely that great accuracy is never going to happen.

But, surely, your last sentence is the point that I am making. A shower cloud, typically 10-15 km across, is a significant feature. It can have effects on wind direction and strength over a substantial area and time. Hence my assertions regarding nonsensical claims by some “unofficial” modellers some of whom offer output claimed to be at 1 km resolution. Their output data may well be on a 1 km grid, but accurate?

I agree. textural information ("frequent showers" ... "occasional sunny spells") is useful, but the only way to forecast at 1km is to be there, looking at the skiy and confining the forecast to an hour ahead. Which ain't really much use.

By the way, and just for completeness, I was thinking of the showers under summer cumulus clouds, which are typically only a few hundred metres across but scattered over much wider areas, like acne on a teenage face. I did once have great fun trying to escape from a French cu-nim which was 30km long and growing almost as fast as I could fly. Happy times.