databricks.labs.dqx.checks_storage

ChecksStorageHandler Objects

class ChecksStorageHandler(ABC, Generic[T])

Abstract base class for handling storage of quality rules (checks).

load

@abstractmethod
def load(config: T) -> list[dict]

Load quality rules from the source. The returned checks can be used as input for apply_checks_by_metadata or apply_checks_by_metadata_and_split functions.

Arguments:

• config - configuration for loading checks, including the table location and run configuration name.

Returns:

list of dq rules or raise an error if checks file is missing or is invalid.

save

@abstractmethod
def save(checks: list[dict], config: T) -> None

Save quality rules to the target.

TableChecksStorageHandler Objects

class TableChecksStorageHandler(ChecksStorageHandler[TableChecksStorageConfig]
                                )

Handler for storing quality rules (checks) in a Delta table in the workspace.

load

@telemetry_logger("load_checks", "table")
def load(config: TableChecksStorageConfig) -> list[dict]

Load checks (dq rules) from a Delta table in the workspace.

Arguments:

• config - configuration for loading checks, including the table location and run configuration name.

Returns:

list of dq rules or raise an error if checks table is missing or is invalid.

Raises:

• NotFound - if the table does not exist in the workspace

save

@telemetry_logger("save_checks", "table")
def save(checks: list[dict], config: TableChecksStorageConfig) -> None

Save checks to a Delta table in the workspace.

Arguments:

• checks - list of dq rules to save
• config - configuration for saving checks, including the table location and run configuration name.

Raises:

• InvalidCheckError - If any check is invalid or unsupported.

WorkspaceFileChecksStorageHandler Objects

class WorkspaceFileChecksStorageHandler(
        ChecksStorageHandler[WorkspaceFileChecksStorageConfig])

Handler for storing quality rules (checks) in a file (json or yaml) in the workspace.

load

@telemetry_logger("load_checks", "workspace_file")
def load(config: WorkspaceFileChecksStorageConfig) -> list[dict]

Load checks (dq rules) from a file (json or yaml) in the workspace. This does not require installation of DQX in the workspace.

Arguments:

• config - configuration for loading checks, including the file location and storage type.

Returns:

list of dq rules or raise an error if checks file is missing or is invalid.

Raises:

• NotFound - if the checks file is not found in the workspace.
• InvalidCheckError - if the checks file cannot be parsed.

save

@telemetry_logger("save_checks", "workspace_file")
def save(checks: list[dict], config: WorkspaceFileChecksStorageConfig) -> None

Save checks (dq rules) to yaml file in the workspace. This does not require installation of DQX in the workspace.

Arguments:

• checks - list of dq rules to save
• config - configuration for saving checks, including the file location and storage type.

FileChecksStorageHandler Objects

class FileChecksStorageHandler(ChecksStorageHandler[FileChecksStorageConfig])

Handler for storing quality rules (checks) in a file (json or yaml) in the local filesystem.

load

def load(config: FileChecksStorageConfig) -> list[dict]

Load checks (dq rules) from a file (json or yaml) in the local filesystem.

Arguments:

• config - configuration for loading checks, including the file location.

Returns:

list of dq rules or raise an error if checks file is missing or is invalid.

Raises:

• FileNotFoundError - if the file path does not exist
• InvalidCheckError - if the checks file cannot be parsed

save

def save(checks: list[dict], config: FileChecksStorageConfig) -> None

Save checks (dq rules) to a file (json or yaml) in the local filesystem.

Arguments:

• checks - list of dq rules to save
• config - configuration for saving checks, including the file location.

Raises:

• FileNotFoundError - if the file path does not exist

InstallationChecksStorageHandler Objects

class InstallationChecksStorageHandler(
        ChecksStorageHandler[InstallationChecksStorageConfig])

Handler for storing quality rules (checks) defined in the installation configuration.

load

@telemetry_logger("load_checks", "installation")
def load(config: InstallationChecksStorageConfig) -> list[dict]

Load checks (dq rules) from the installation configuration.

Arguments:

• config - configuration for loading checks, including the run configuration name and method.

Returns:

list of dq rules or raise an error if checks file is missing or is invalid.

Raises:

• NotFound - if the checks file or table is not found in the installation.
• InvalidCheckError - if the checks file cannot be parsed.

save

@telemetry_logger("save_checks", "installation")
def save(checks: list[dict], config: InstallationChecksStorageConfig) -> None

Save checks (dq rules) to yaml file or table in the installation folder. This will overwrite existing checks file or table.

Arguments:

• checks - list of dq rules to save
• config - configuration for saving checks, including the run configuration name, method, and table location.

VolumeFileChecksStorageHandler Objects

class VolumeFileChecksStorageHandler(
        ChecksStorageHandler[VolumeFileChecksStorageConfig])

Handler for storing quality rules (checks) in a file (json or yaml) in a Unity Catalog volume.

load

@telemetry_logger("load_checks", "volume")
def load(config: VolumeFileChecksStorageConfig) -> list[dict]

Load checks (dq rules) from a file (json or yaml) in a Unity Catalog volume.

Arguments:

• config - configuration for loading checks, including the file location and storage type.

Returns:

list of dq rules or raise an error if checks file is missing or is invalid.

Raises:

• NotFound - if the checks file is not found in the workspace.
• InvalidCheckError - if the checks file cannot be parsed.
• CheckDownloadError - if there is an error downloading the file from the volume.

save

@telemetry_logger("save_checks", "volume")
def save(checks: list[dict], config: VolumeFileChecksStorageConfig) -> None

Save checks (dq rules) to yaml file in a Unity Catalog volume. This does not require installation of DQX in a Unity Catalog volume.

Arguments:

• checks - list of dq rules to save
• config - configuration for saving checks, including the file location and storage type.

BaseChecksStorageHandlerFactory Objects

class BaseChecksStorageHandlerFactory(ABC)

Abstract base class for factories that create storage handlers for checks.

create

@abstractmethod
def create(config: BaseChecksStorageConfig) -> ChecksStorageHandler

Abstract method to create a handler based on the type of the provided configuration object.

Arguments:

• config - Configuration object for loading or saving checks.

Returns:

An instance of the corresponding BaseChecksStorageHandler.

create_for_location

@abstractmethod
def create_for_location(
    location: str,
    run_config_name: str = "default"
) -> tuple[ChecksStorageHandler, BaseChecksStorageConfig]

Abstract method to create a handler and config based on checks location.

Arguments:

• location - location of the checks (file path, table name, volume, etc.)
• run_config_name - the name of the run configuration to use for checks (default is 'default').

Returns:

An instance of the corresponding BaseChecksStorageHandler.

ChecksStorageHandlerFactory Objects

class ChecksStorageHandlerFactory(BaseChecksStorageHandlerFactory)

create

def create(config: BaseChecksStorageConfig) -> ChecksStorageHandler

Factory method to create a handler based on the type of the provided configuration object.

Arguments:

• config - Configuration object for loading or saving checks.

Returns:

An instance of the corresponding BaseChecksStorageHandler.

Raises:

• InvalidConfigError - If the configuration type is unsupported.

is_table_location

def is_table_location(location: str) -> bool

True if location points to a Delta table (catalog.schema.table) and is not a file path with a known checks serializer extension.

Arguments:

• location str - The checks location to validate.

Returns:

• bool - True if the location is a valid table name and not a file path, False otherwise.