Install Library on Cluster

Install Library on Cluster

Usage

db_libs_install(
  cluster_id,
  libraries,
  host = db_host(),
  token = db_token(),
  perform_request = TRUE
)

Arguments

cluster_id

Unique identifier of a Databricks cluster.

libraries

An object created by libraries() and the appropriate lib_*() functions.

host

Databricks workspace URL, defaults to calling db_host().

token

Databricks workspace token, defaults to calling db_token().

perform_request

If TRUE (default) the request is performed, if FALSE the httr2 request is returned without being performed.

Details

Installation is asynchronous - it completes in the background after the request.

This call will fail if the cluster is terminated. Installing a wheel library on a cluster is like running the pip command against the wheel file directly on driver and executors.

Installing a wheel library on a cluster is like running the pip command against the wheel file directly on driver and executors. All the dependencies specified in the library setup.py file are installed and this requires the library name to satisfy the wheel file name convention.

The installation on the executors happens only when a new task is launched. With Databricks Runtime 7.1 and below, the installation order of libraries is nondeterministic. For wheel libraries, you can ensure a deterministic installation order by creating a zip file with suffix .wheelhouse.zip that includes all the wheel files.

See also

lib_egg(), lib_cran(), lib_jar(), lib_maven(), lib_pypi(), lib_whl()

Other Libraries API: db_libs_all_cluster_statuses(), db_libs_cluster_status(), db_libs_uninstall()