Skip to main content

databricks.labs.dqx.installer.workflow_task

Task Objects​

@dataclass
class Task()

job_cluster​

cluster key for job clusters; if None, uses serverless environment

run_job_name​

when set, this task will run another job

dependencies​

def dependencies()

List of dependencies

Workflow Objects​

class Workflow()

name​

@property
def name()

Name of the workflow

spark_conf​

@property
def spark_conf() -> dict[str, str] | None

Spark configuration for the workflow

override_clusters​

@property
def override_clusters() -> dict[str, str] | None

Override clusters for the workflow

tasks​

def tasks() -> Iterable[Task]

List of tasks

workflow_task​

def workflow_task(
        fn=None,
        *,
        depends_on=None,
        job_cluster=Task.job_cluster,
        run_job_name: str | None = None) -> Callable[[Callable], Callable]

Decorator to register a task in a workflow.

remove_extra_indentation​

def remove_extra_indentation(doc: str) -> str

Remove extra indentation from docstring.

Arguments:

  • doc str - Docstring to process.

Returns:

  • str - Processed docstring with extra indentation removed.