gributils
gributils is a set of utilities on top of pygrib for manipulating and indexing gribfiles. Current features include:
- Grib index
- Store and query a large set (historical dataset) of grib files
- Elastic Search
- Query by geographical location, timestamp and parameter name, parameter unit, level and level type.
- Parameter name normalization using a simple CSV file
- Optional REST based API server
- Extract polygons of areas covered with valid values in a grib layer
- Provide accurate grid coordinates, even for grib version 1 files where too low precision in Dx / Dy have resulted in a distorted grid.
- Conversion from values with U and V components (e.g. for wind) to magnitude and degree north.
Command line usage
Lookup parameter values for a certain point in space and time, across multiple indexed grib files:
ex@ample:~# gributils index --database="http://elasticsearch:9200" interp-timestamp --timestamp "2018-08-21 18:30:00" --lat 63 --lon 10
[{'parameterName': 'P Pressure', 'parameterUnit': 'Pa', 'typeOfLevel': 'heightAboveGround', 'level': 0, 'value': 101633.03125000001},
{'parameterName': 'U component of wind', 'parameterUnit': 'm s-1', 'typeOfLevel': 'heightAboveGround', 'level': 10, 'value': -2.898160457611084},
{'parameterName': 'V component of wind', 'parameterUnit': 'm s-1', 'typeOfLevel': 'heightAboveGround', 'level': 10, 'value': 2.705005645751954}]
Python usage
Lookup parameter values for a certain point in space and time, across multiple indexed grib files:
>>> import gributils.gribindex, datetime
>>> index = gributils.gribindex.GribIndex("http://elasticsearch:9200")
>>> index.interp_timestamp(lat=63, lon=10, timestamp=datetime.datetime(2018, 8, 21, 19, 32, 00))
[{'parameterName': 'P Pressure', 'parameterUnit': 'Pa', 'typeOfLevel': 'heightAboveGround', 'level': 0, 'value': 101673.95000000001},
{'parameterName': 'U component of wind', 'parameterUnit': 'm s-1', 'typeOfLevel': 'heightAboveGround', 'level': 10, 'value': -2.0344434102376305},
{'parameterName': 'V component of wind', 'parameterUnit': 'm s-1', 'typeOfLevel': 'heightAboveGround', 'level': 10, 'value': 1.9993160883585617}]
Extracting layer boundaries from a grib layer:
>>> import gributils.bounds, pygrib
>>> polygon = gributils.bounds.bounds(pygrib.open("2018-08-20T14:27:30+00:00.mywavewam4_weatherapi_sorlandet.grb")[1])
>>> polygon
<shapely.geometry.polygon.Polygon object at 0x7fcd15abb208>
>>> # poly_real_coords only has coordinates in the ranges ]-90,90[, ]-180,180[
>>> poly_real_coords = gributils.bounds.unwrap_dateline(gributils.bounds.split_dateline(polygon))
>>> poly_real_coords
<shapely.geometry.multipolygon.MultiPolygon object at 0x7fcd0d4aab38>
>>> # Calculate a unique id identifying this polygon that can be used for database lookups etc
>>> gributils.bounds.polygon_id(poly_real_coords)
'36cf52a2cd2ee4aed01095e7cd831f5f2c9a96c292eaca40de1b2d63c62ab1b8'
REST usage
The rest API is described in the swagger documentation available at http://localhost:1028/
ex@ample:~# gributils server --database="http://elasticsearch:9200"
ex@ample:~# curl 'http://localhost:1028/index/parametermap/add?name=smhi_arome' --data-binary "@parametermaps/smhi/arome/parametermap.csv"
{"status": "success"}
ex@ample:~# curl 'http://localhost:1028/index/add?parametermap=smhi_arome&extra=\{\}' --data-binary "@file.grib"
{"status": "success"}
ex@ample:~# curl 'http://localhost:1028/index/interpolate/timestamp?lat=63&lon=10×tamp=2018-08-21T19:32:00.000000Z'
{'parameterName': 'P Pressure', 'parameterUnit': 'Pa', 'typeOfLevel': 'heightAboveGround', 'level': 0, 'value': 101673.95000000001}
{'parameterName': 'U component of wind', 'parameterUnit': 'm s-1', 'typeOfLevel': 'heightAboveGround', 'level': 10, 'value': -2.0344434102376305}
{'parameterName': 'V component of wind', 'parameterUnit': 'm s-1', 'typeOfLevel': 'heightAboveGround', 'level': 10, 'value': 1.9993160883585617}
Installation
apt install libgrib-api-dev libeccodes-dev
pip install pyproj numpy flask
python setup.py install
Additional tools
Gributils annotator lets you annotate streams of positional data with weather using gributils.