Skip to contents

This is a long-running operation, which blocks until Clusters on Databricks reach RUNNING state with the timeout of 20 minutes, that you can change via timeout parameter. By default, the state of Databricks Clusters is reported to console. You can change this behavior by changing the callback parameter.

Usage

resize_cluster_and_wait(
  client,
  cluster_id,
  autoscale = NULL,
  num_workers = NULL,
  timeout = 20,
  callback = cli_reporter
)

Arguments

client

Required. Instance of DatabricksClient()

cluster_id

Required. The cluster to be resized.

autoscale

Parameters needed in order to automatically scale clusters up and down based on load.

num_workers

Number of worker nodes that this cluster should have.

timeout

Time to wait for the operation to complete in minutes.

callback

Function to report the status of the operation. By default, it reports to console.

Details

Resizes a cluster to have a desired number of workers. This will fail unless the cluster is in a RUNNING state.