Functions the JAR speeds up
The GeoBrix JAR adds a JVM level (L1) to some functions. On a dedicated cluster with the JAR, these functions run in the JVM instead of in a Python worker.
The env wheel alone runs every function. The JAR is optional: install it only where you want the JVM level.
The JAR is a performance add-on that dedicated compute can load, not an API. You call every function from SQL or Python, and GeoBrix picks the JVM level where it is faster. The Scala tag on a function's API page links here. For the measured speed-ups, see Performance with the JAR and the benchmark results by function.
Where the JAR loads
- Dedicated access mode loads the JAR. The functions on this page run on the JVM there.
- Standard access mode runs Spark Connect, so no function has a JVM level there. GeoBrix runs the Python level and says why.
- Serverless cannot load the JAR. Every function runs at its Python level.
- A SQL warehouse runs Unity Catalog functions only. See Unity Catalog functions.
What needs dedicated compute compares the computes feature by feature.
Which functions run on the JVM
GeoBrix makes the JVM level the default where it measured at least 1.2x faster than the best Python form, with the same output.
A few functions are not measured yet. They run the JVM level until a measurement says otherwise.
Performance has the measured speed-ups. Every function not listed here runs at its Python level, with or without the JAR.
A note "A virtual tile runs in Python" marks a tile function that reads bytes on the JVM. GeoBrix sends a virtual tile to the Python level, and a materialized tile to the JVM. See Virtual tiles and the JVM level.
48 functions run on the JVM when the JAR is present.
RasterX
GridX
VectorX
| Function | Package | Notes |
|---|---|---|
gbx_st_interpolateelevationbbox | VectorX | |
gbx_st_legacyaswkb | VectorX | |
gbx_st_transformcrs | VectorX | |
gbx_st_triangulate | VectorX |
Check the level GeoBrix picked
register returns the level it chose for each name:
from databricks.labs.gbx import register
chosen = register(spark)
chosen["gbx_rst_boundingbox"].level # 'L1' on dedicated compute with the JAR, else 'L3'
When the JVM level cannot run, register prints one line that names the level and the reason. See GeoBrix says why.
Install the JAR
- Choose an install path compares the env wheel alone, the env wheel and the JAR, and a container image.
databricks.labs.gbx.stage_jar(vol_dir)copies the JAR of the env wheel to a Volume folder. See Stage the JAR.- A DCS image starts a dedicated DBR 19 cluster with the JAR in place.