Skip to main content

Functions the JAR speeds up

The GeoBrix JAR adds a JVM level (L1) to some functions. On a dedicated cluster with the JAR, these functions run in the JVM instead of in a Python worker.

The env wheel alone runs every function. The JAR is optional: install it only where you want the JVM level.

The JAR is a performance add-on that dedicated compute can load, not an API. You call every function from SQL or Python, and GeoBrix picks the JVM level where it is faster. The Scala tag on a function's API page links here. For the measured speed-ups, see Performance with the JAR and the benchmark results by function.

Where the JAR loads​

  • Dedicated access mode loads the JAR. The functions on this page run on the JVM there.
  • Standard access mode runs Spark Connect, so no function has a JVM level there. GeoBrix runs the Python level and says why.
  • Serverless cannot load the JAR. Every function runs at its Python level.
  • A SQL warehouse runs Unity Catalog functions only. See Unity Catalog functions.

What needs dedicated compute compares the computes feature by feature.

Which functions run on the JVM​

The keep rule

GeoBrix makes the JVM level the default where it measured at least 1.2x faster than the best Python form, with the same output.

A few functions are not measured yet. They run the JVM level until a measurement says otherwise.

Performance has the measured speed-ups. Every function not listed here runs at its Python level, with or without the JAR.

A note "A virtual tile runs in Python" marks a tile function that reads bytes on the JVM. GeoBrix sends a virtual tile to the Python level, and a materialized tile to the JVM. See Virtual tiles and the JVM level.

48 functions run on the JVM when the JAR is present.

RasterX​

FunctionPackageNotes
gbx_rst_bng_rasterize_aggRasterX
gbx_rst_boundingboxRasterXA virtual tile runs in Python.
gbx_rst_custom_rasterize_aggRasterX
gbx_rst_fromcontentRasterX
gbx_rst_h3_rasterize_aggRasterX
gbx_rst_quadbin_rasterize_aggRasterX
gbx_rst_tilexyzRasterXA virtual, NULL or error tile runs in Python.
gbx_rst_worldtorastercoordyRasterXA virtual tile runs in Python.

GridX​

FunctionPackageNotes
gbx_bng_aswkbGridX
gbx_bng_aswktGridX
gbx_bng_cellareaGridX
gbx_bng_cellfillGridX
gbx_bng_cellintersectionGridX
gbx_bng_cellunionGridX
gbx_bng_centroidGridX
gbx_bng_distanceGridX
gbx_bng_euclideandistanceGridX
gbx_bng_geomkloopGridX
gbx_bng_geomkloopexplodeGridX
gbx_bng_geomkringGridX
gbx_bng_geomkringexplodeGridX
gbx_bng_kloopGridX
gbx_bng_kloopexplodeGridX
gbx_bng_kringGridX
gbx_bng_kringexplodeGridX
gbx_bng_pointascellGridX
gbx_bng_polyfillGridX
gbx_bng_tessellateGridX
gbx_bng_tessellateexplodeGridX
gbx_custom_cellaswkbGridX
gbx_custom_cellaswktGridX
gbx_custom_cellfillGridX
gbx_custom_centroidGridX
gbx_custom_distanceGridX
gbx_custom_geomkloopGridX
gbx_custom_geomkloopexplodeGridX
gbx_custom_geomkringGridX
gbx_custom_geomkringexplodeGridX
gbx_custom_gridGridX
gbx_custom_kloopGridX
gbx_custom_kringGridX
gbx_custom_pointascellGridX
gbx_custom_polyfillGridX
gbx_h3_cellfillGridX

VectorX​

FunctionPackageNotes
gbx_st_interpolateelevationbboxVectorX
gbx_st_legacyaswkbVectorX
gbx_st_transformcrsVectorX
gbx_st_triangulateVectorX

Check the level GeoBrix picked​

register returns the level it chose for each name:

from databricks.labs.gbx import register

chosen = register(spark)
chosen["gbx_rst_boundingbox"].level # 'L1' on dedicated compute with the JAR, else 'L3'

When the JVM level cannot run, register prints one line that names the level and the reason. See GeoBrix says why.

Install the JAR​

  • Choose an install path compares the env wheel alone, the env wheel and the JAR, and a container image.
  • databricks.labs.gbx.stage_jar(vol_dir) copies the JAR of the env wheel to a Volume folder. See Stage the JAR.
  • A DCS image starts a dedicated DBR 19 cluster with the JAR in place.