Shapefile Writer

shapefile_gbx — pyogrio-backed, pure-Python DataSource V2 writer with the OGR driver preset to ESRI Shapefile. It round-trips with the matching shapefile_gbx reader; the schema (attrs..., geom_0, geom_0_srid, geom_0_srid_proj) is shared across all lightweight vector readers and writers.

Compute & scale

• Large Shapefiles: a Shapefile assembles on a single node, and the format hard-caps each .shp/.dbf at 2 GB. For large outputs, see Choosing a writer for large datasets — use a classic cluster (high-memory Serverless does not help here) or the scalable geojsonl_gbx / gpkg_gbx writers.
• Tier & compute: the lightweight (*_gbx) writers need no JAR or init script and are the only option on Serverless, standard (shared), and ARM clusters. The heavyweight raster/PMTiles writers require a classic x86 cluster (JAR + GDAL init script). Your compute usually decides the tier — then data scale. See Benchmarking for timings and methodology.

Before you write

• Register first — call register(spark) once before using any *_gbx format (see the Writers Overview).
• Lightweight-only — this single-file writer isn't implemented in the heavyweight tier; for output in both tiers, use the sharded GeoJSONL writer (geojsonl_gbx / geojsonl).
• Input geometry — provide geometry as WKB (binary) or WKT (text), with the CRS in the companion *_srid / *_srid_proj columns. EWKB, EWKT, and GeoJSON-encoded geometry are not accepted. Writing from a Databricks GEOMETRY / GEOGRAPHY column? Export to WKB first with ST_AsBinary (or ST_AsText for WKT) — see the Schema section below — and avoid ST_GeomAsEWKB / ST_AsEWKT / ST_AsGeoJSON.

Options

Option	Default	Behavior
driverName	ESRI Shapefile (preset)	OGR driver. Preset by this named writer; override only if needed.
mode	overwrite	overwrite only; append is rejected.
fileName	(none)	Name the output archive explicitly. Only honored when zip=true: when set, .save(path) treats path as the parent directory (created if missing) and writes path/<fileName>.shp.zip. For non-zip shapefile (directory bundle), fileName is not applied — use zip=true for single-archive naming. See Output naming.
zip	false	When true, produce a single .shp.zip archive containing the .shp/.shx/.dbf/.prj/.cpg files at the archive root instead of a sidecar directory. The output path is normalised automatically: .save("roads") → roads.shp.zip; .save("roads.shp") → roads.shp.zip. Useful when the destination requires a single portable file (e.g. uploading to a GIS portal). Ignored for non-Shapefile drivers.
geometryType	inferred from the data	Override the OGR geometry type.
layerName	driver default	Output layer name where supported.
geomCol	auto-detected from the *_srid column	Override the geometry column name. Locates the geometry and its SRID companion (<geomCol>_srid). See Named Vector Formats. This is a lightweight-tier writer.
sridCol	<geomCol>_srid	Override the SRID column name. Required — supplies the CRS authority code (e.g. "4326", or "0" if unknown).
projCol	<geomCol>_srid_proj	Override the PROJ4 column name (optional fallback CRS when sridCol is "0").

Schema

shapefile_gbx uses the shared vector column contract — a geometry column plus its *_srid (required) / *_srid_proj (optional) CRS companions; every other column becomes an attribute. Point the writer at your columns with geomCol / sridCol / projCol, or coerce to the geom_0 convention.

(df  # geometry + CRS under your own column names
 .write.format("shapefile_gbx")
 .option("geomCol", "the_geom")   # WKB or WKT
 .option("sridCol", "epsg")       # REQUIRED — CRS authority code
 .mode("overwrite").save("/Volumes/cat/sch/vol/parcels"))

For coercing an arbitrary frame to this shape and for exporting Databricks GEOMETRY / GEOGRAPHY columns, see the Vector Writer schema.

Output: an ESRI Shapefile — a .shp plus its .shx / .dbf / .prj / .cpg sidecars. Note the Shapefile format constraints: attribute field names are truncated to 10 characters, attribute types are limited by the dBASE (.dbf) format, and a file holds a single geometry type.

How it scales

Unlike a single-node pyogrio.write_* call that serializes one file on one machine, the shapefile_gbx writer runs as a Spark DataSource V2 two-phase write: each partition is written concurrently by its executor to a scratch fragment, then the driver merges the fragments into one output file. The merge is sequential and rename-free, so it is safe on FUSE-mounted cloud storage (Unity Catalog Volumes, DBFS). Repartition the input to control write parallelism.

Output naming

shapefile_gbx applies the standard single-file output naming contract. When zip=true the canonical extension is .shp.zip; without zip the output is a sidecar directory (not a single file, so the naming contract applies to the directory name rather than a file extension). Rules for the zip=true (single-archive) case:

Case	.save(path) / fileName	Resolved output
fileName given	.option("fileName","roads").save("/out/exports")	/out/exports/roads.shp.zip
No fileName; path is an existing directory	.save("/out/exports")	/out/exports/exports.shp.zip
No fileName; path is a stem	.save("/out/exports/roads")	/out/exports/roads.shp.zip

Extension auto-completion: roads → roads.shp.zip; roads.shp → roads.shp.zip; roads.shp.zip → unchanged. Passing a name ending in a different recognized geo extension (e.g. .gpkg) raises a clear error.

Example

# Lightweight Shapefile writer (pyogrio; OGR driver preset to "ESRI Shapefile")
from databricks.labs.gbx.ds.register import register
register(spark)
src = f"{SAMPLE_DATA_BASE}/nyc/subway/nyc_subway.shp.zip"
df = spark.read.format("shapefile_gbx").load(src)
out = "/tmp/nyc_subway.shp"  # a single named output file (.shp + sidecars)
df.write.format("shapefile_gbx").mode("overwrite").save(out)
back = spark.read.format("shapefile_gbx").load(out)
assert back.count() == df.count()

Typical pipeline: export a table to a single file

Export a table of vector data to a Shapefile — no coalesce needed (the writer merges partitions into a single file on commit):

df = spark.table("main.geo.parcels")  # vector data in a (Delta) table
df.write.format("shapefile_gbx").mode("overwrite").save("/Volumes/main/geo/exports/parcels.shp")

Each partition is written concurrently, then merged into one output file. See Benchmarking for light-vs-heavy export figures.

Producing a portable ZIP archive

Pass zip=true to write a single self-contained .shp.zip file — the .shp, .shx, .dbf, .prj, and .cpg sidecars are packed flat at the archive root. The output path is normalised automatically regardless of the extension you supply — .save("roads"), .save("roads.shp"), and .save("roads.shp.zip") all write roads.shp.zip:

df.write \
    .format("shapefile_gbx") \
    .mode("overwrite") \
    .option("zip", "true") \
    .save("/Volumes/main/geo/exports/roads")
# -> /Volumes/main/geo/exports/roads.shp.zip

The resulting archive is readable by shapefile_gbx (and any tool that handles .shp.zip / /vsizip/), and can be uploaded directly to GIS portals that expect a single compressed Shapefile.