@jufaua/geobuf v3.0.2
Geobuf
Geobuf is a compact binary encoding for geographic data.
This is a fork of mapbox/geobuf that adds support for packing array of numbers instead of handling them as object, resulting in a more compact protobuf for use cases with large arrays of int/float.
Geobuf provides nearly lossless compression of GeoJSON data into protocol buffers. Advantages over using GeoJSON alone:
- Very compact: typically makes GeoJSON 6-8 times smaller.
- 2-2.5x smaller even when comparing gzipped sizes.
- Very fast encoding and decoding — even faster than native JSON parse/stringify.
- Can accommodate any GeoJSON data, including extensions with arbitrary properties.
The encoding format also potentially allows:
- Easy incremental parsing — get features out as you read them, without the need to build in-memory representation of the whole data.
- Partial reads — read only the parts you actually need, skipping the rest.
Think of this as an attempt to design a simple, modern Shapefile successor that works seamlessly with GeoJSON. Unlike Mapbox Vector Tiles, it aims for nearly lossless compression of datasets — without tiling, projecting coordinates, flattening geometries or stripping properties.
Note that the encoding schema is not stable yet — it may still change as we get community feedback and discover new ways to improve it.
"Nearly" lossless means coordinates are encoded with precision of 6 digits after the decimal point (about 10cm).
Sample compression sizes
| Data | JSON | JSON (gz) | Geobuf | Geobuf (gz) |
|---|---|---|---|---|
| US zip codes | 101.85 MB | 26.67 MB | 12.24 MB | 10.48 MB |
| Idaho counties | 10.92 MB | 2.57 MB | 1.37 MB | 1.17 MB |
API
encode
var buffer = geobuf.encode(geojson, new Pbf());Given a GeoJSON object and a Pbf object to write to,
returns a Geobuf as UInt8Array array of bytes.
In Node@4.5.0 or later, you can use Buffer.from to convert back to a buffer.
decode
var geojson = geobuf.decode(new Pbf(data));Given a Pbf object with Geobuf data, return a GeoJSON object. When loading Geobuf data over XMLHttpRequest, you need to set responseType to arraybuffer.
Install
Node and Browserify:
npm install @jufaua/geobufBrowser build CDN links:
Building locally:
npm install
npm run build-dev # dist/geobuf-dev.js (development build)
npm run build-min # dist/geobuf.js (minified production build)Command Line
npm install -g @jufaua/geobufInstalls these nifty binaries:
geobuf2json: turn Geobuf fromstdinor specified file to GeoJSON onstdoutjson2geobuf: turn GeoJSON fromstdinor specified file to Geobuf onstdoutshp2geobuf: given a Shapefile filename, send Geobuf onstdout
json2geobuf data.json > data.pbf
shp2geobuf myshapefile > data.pbf
geobuf2json data.pbf > data.jsonNote that for big files, geobuf2json command can be pretty slow, but the bottleneck is not the decoding,
but the native JSON.stringify on the decoded object to pipe it as a string to stdout.
On some files, this step may take 40 times more time than actual decoding.
See Also
- geojsonp — the prototype that led to this project
- pygeobuf — Python implementation of Geobuf
- twkb — a geospatial binary encoding that doesn't support topology
and doesn't encode any non-geographic properties besides
id - vector-tile-spec
- topojson — an extension of GeoJSON that supports topology
- WKT and WKB — popular in databases
- EWKB — a popular superset of WKB