Readers & Writers
GeoBrix provides Spark DataSource V2 readers and writers for geospatial file formats,
across both execution tiers — lightweight (pyrx, pure-Python, no JAR) and heavyweight
(rasterx, GDAL/OGR-backed). Both tiers scale by partitioning work across the cluster rather
than reading or writing sequentially on a single node.
Each lightweight *_gbx format pairs with a heavyweight counterpart (*_ogr / gdal /
gtiff_gdal), and the two tiers are held to row-count / byte parity as a hard gate in
benchmarking.
Readers
Load geospatial files into a distributed DataFrame. See the Readers Overview for the full list, tier differences, output schemas, and options.
| Format | Lightweight | Heavyweight |
|---|---|---|
| Raster (generic) | raster_gbx | gdal |
| GeoTIFF | gtiff_gbx | gtiff_gdal |
| NetCDF | netcdf_gbx | netcdf_gdal |
| PMTiles | pmtiles_gbx | — (light-only) |
| Vector (generic) | vector_gbx | ogr |
| Shapefile | shapefile_gbx | shapefile_ogr |
| GeoJSON | geojson_gbx | geojson_ogr |
| GeoPackage | gpkg_gbx | gpkg_ogr |
| File Geodatabase | file_gdb_gbx | file_gdb_ogr |
Writers
Write DataFrames back out to geospatial files. See the Writers Overview for the column contract, single-file vs sharded trade-offs, and per-format details.
| Format | Lightweight | Heavyweight |
|---|---|---|
| Raster (generic) | raster_gbx | gdal |
| GeoTIFF | gtiff_gbx | gtiff_gdal |
| PMTiles | pmtiles_gbx | pmtiles |
| NetCDF | netcdf_gbx | — (light-only) |
| Vector (generic) | vector_gbx | — |
| Shapefile | shapefile_gbx | — |
| GeoJSON | geojson_gbx | — |
| GeoJSONL | geojsonl_gbx | geojsonl_ogr |
| GeoPackage | gpkg_gbx | — |
| File Geodatabase | file_gdb_gbx | — (hybrid; needs native GDAL) |
Registering the lightweight tier
Heavyweight readers and writers are auto-discovered from the JAR. The lightweight Python
DataSources are not auto-registered — call register(spark) once per session before using
any *_gbx format:
from databricks.labs.gbx.ds.register import register
register(spark)
Next Steps
- Readers Overview — every reader, tier differences, output schemas, options.
- Writers Overview — the column contract, single-file vs sharded writers, per-format details.
- Benchmarking — tier-vs-tier timing and parity methodology.