Skip to main content

Metadata and access control

Partner integrations should use serverless SQL warehouses for all metadata operations—reading catalogs, schemas, tables, and access policies—to ensure consistent governance and optimal performance. Review Unity Catalog privileges and securable objects, apply least-privilege access, and filter out system assets for a cleaner user experience.

Reading metadata​

General metadata lookup​

Use Databricks SQL driver metadata APIs, GET calls on SDKs, or SHOW commands to retrieve high-level metadata such as catalogs, schemas, tables, and other assets.

Detailed metadata for specific objects​

For more detailed metadata, use SHOW and DESCRIBE EXTENDED SQL commands.

Documentation: SHOW statements | DESCRIBE statements | Information Schema

note

Metadata accessed through your integration must comply with the Databricks Acceptable Use Policy (AUP).

Writing metadata​

  1. The recommended starting point for creating and modifying metadata (e.g., tables, views, catalogs, tags) is via SQL DDL statements.
  2. To include external assets in data lineage, use the bring-your-own-lineage API.

Documentation: SQL DDL statements | External Lineage (BYOL)

Managing access control​

Databricks provides multiple layers of access control. Use SQL to write policies to Unity Catalog and let Unity Catalog handle enforcement. To manage access at scale without granting permissions on individual objects, use ABAC with governed tags. For advanced scenarios, use the Databricks REST API.

Documentation: Access control layers | ABAC | Data access control | REST API

What's next​