Workflow and automation

Delta Sharing operations can be fully automated using the Databricks REST API, Python SDK, or SQL commands. This enables self-service onboarding, scheduled maintenance, and integration with CI/CD pipelines.

Common workflows

Workflow	Description	Automation approach
Recipient onboarding	Create recipients, assign to shares, distribute credentials	SDK/API triggered by request form or CRM
Share provisioning	Create shares, add tables/views, configure partitions	Terraform or SDK in CI/CD pipeline
Token rotation	Rotate bearer tokens on schedule	Scheduled Workflow with SDK
Access revocation	Remove recipients or revoke grants	API call triggered by contract termination
Schema updates	Add new tables or columns to existing shares	SDK in data pipeline post-deployment

Automation tools

Tool	Best for	Documentation
REST API	Direct HTTP calls, language-agnostic integrations	Shares, Recipients, Providers
Python SDK - Sharing	Shares, recipients, grants	Sharing module
Python SDK - Marketplace	Listings, consumer requests, exchanges	Marketplace module
SQL	Ad-hoc operations, notebook-based workflows	Delta Sharing SQL
Terraform	Infrastructure-as-code, declarative provisioning	Databricks Terraform Provider

Use the Sharing module for D2D or open sharing outside of Marketplace. Use the Marketplace module when you need to programmatically manage listings, fulfill consumer requests, or work with private exchanges.

Example scripts

note

These scripts are provided as starting points for your automation. Review, test, and adapt them for your environment before use in production.

Self-service onboarding

Automate recipient onboarding with a Databricks Workflow triggered by a request form:

from databricks.sdk import WorkspaceClient

def onboard_recipient(customer_name: str, sharing_id: str, license: str):
    """Automated recipient onboarding."""
    w = WorkspaceClient()
    
    # Create recipient with properties
    recipient = w.recipients.create(
        name=customer_name.lower().replace(" ", "_"),
        sharing_code=sharing_id,
        properties_kvpairs={"properties": {"customer_code": customer_name, "license": license}}
    )
    
    # Grant access to appropriate shares based on license
    w.shares.update_permissions(
        name=f"{license}_data_share",
        changes=[{"principal": recipient.name, "add": ["SELECT"]}]
    )
    return recipient

Audit all shares and recipients

Generate a compliance report of all shares, their objects, and who has access:

def audit_shares():
    """Generate a complete audit of shares and recipients."""
    w = WorkspaceClient()
    audit = []
    
    for share in w.shares.list():
        share_detail = w.shares.get(share.name)
        permissions = w.shares.share_permissions(share.name)
        
        audit.append({
            "share": share.name,
            "objects": [obj.name for obj in (share_detail.objects or [])],
            "recipients": [p.principal for p in (permissions.privilege_assignments or [])]
        })
    
    return audit

Scheduled token rotation

Rotate tokens for open sharing recipients on a regular schedule:

def rotate_tokens_for_recipients(recipient_names: list[str]):
    """Rotate tokens for specified recipients."""
    w = WorkspaceClient()
    for name in recipient_names:
        w.recipients.rotate_token(name=name)
        # In production: send new activation link via secure channel

Schedule this as a Databricks Workflow to run monthly or quarterly.

Bulk revoke access

Revoke access for all recipients matching a naming pattern (e.g., during offboarding):

import re

def revoke_recipients_matching(pattern: str, dry_run: bool = True):
    """Revoke all shares from recipients matching a regex pattern."""
    w = WorkspaceClient()
    regex = re.compile(pattern)
    
    for recipient in w.recipients.list():
        if regex.match(recipient.name):
            if dry_run:
                print(f"[DRY RUN] Would revoke: {recipient.name}")
            else:
                w.recipients.delete(recipient.name)
                print(f"Revoked: {recipient.name}")

# Preview what would be revoked
revoke_recipients_matching(r"^test_.*", dry_run=True)

Export metadata for backup

Export all share configurations to JSON for disaster recovery:

import json
from datetime import datetime

def export_share_metadata(output_file: str = None):
    """Export all share metadata to JSON."""
    w = WorkspaceClient()
    
    export = {"exported_at": datetime.now().isoformat(), "shares": [], "recipients": []}
    
    for share in w.shares.list():
        detail = w.shares.get(share.name)
        export["shares"].append({
            "name": share.name,
            "objects": [{"name": o.name, "type": o.data_object_type} for o in (detail.objects or [])]
        })
    
    for recipient in w.recipients.list():
        export["recipients"].append({"name": recipient.name, "type": recipient.authentication_type})
    
    if output_file:
        with open(output_file, "w") as f:
            json.dump(export, f, indent=2)
    return export

SQL quick reference

Shares

-- Create
CREATE SHARE my_share COMMENT 'Description';

-- Add assets
ALTER SHARE my_share ADD TABLE catalog.schema.table_name;
ALTER SHARE my_share ADD TABLE catalog.schema.table_name WITH HISTORY;
ALTER SHARE my_share ADD TABLE catalog.schema.table_name 
  PARTITION (region = 'us-west');
ALTER SHARE my_share ADD VIEW catalog.schema.view_name;
ALTER SHARE my_share ADD VOLUME catalog.schema.volume_name;

-- Remove assets
ALTER SHARE my_share REMOVE TABLE catalog.schema.table_name;

-- List and describe
SHOW SHARES;
DESCRIBE SHARE my_share;
SHOW ALL IN SHARE my_share;

-- Delete
DROP SHARE my_share;

Recipients

-- Create D2D recipient
CREATE RECIPIENT acme_corp USING ID '<sharing-identifier>';

-- Create D2O recipient (token-based)
CREATE RECIPIENT external_partner COMMENT 'Partner XYZ';

-- Create with properties
CREATE RECIPIENT acme_corp
USING ID '<sharing-identifier>'
PROPERTIES ('customer_code' = 'ACME', 'license' = 'enterprise');

-- Update properties
ALTER RECIPIENT acme_corp SET PROPERTIES ('license' = 'professional');

-- Rotate token (D2O only)
ALTER RECIPIENT external_partner ROTATE TOKEN;

-- List and describe
SHOW RECIPIENTS;
DESCRIBE RECIPIENT acme_corp;

-- Delete
DROP RECIPIENT acme_corp;

Grants

-- Grant access
GRANT SELECT ON SHARE my_share TO RECIPIENT acme_corp;

-- Revoke access
REVOKE SELECT ON SHARE my_share FROM RECIPIENT acme_corp;

-- View grants
SHOW GRANTS ON SHARE my_share;
SHOW GRANTS TO RECIPIENT acme_corp;

Information schema queries

-- List all shares
SELECT * FROM system.information_schema.shares;

-- List share contents
SELECT * FROM system.information_schema.share_objects
WHERE share_name = 'my_share';

-- List recipients
SELECT * FROM system.information_schema.recipient_tokens;

-- List recipient permissions
SELECT * FROM system.information_schema.share_recipient_privileges;

For complete syntax, see the Delta Sharing SQL reference.

SDK quick reference

The Databricks Python SDK provides programmatic access to Delta Sharing and Marketplace operations.

Installation

pip install databricks-sdk

Delta Sharing operations

from databricks.sdk import WorkspaceClient
w = WorkspaceClient()

# Shares
w.shares.list()                    # List all shares
w.shares.get(name="my_share")      # Get share details
w.shares.create(name="...", ...)   # Create share
w.shares.update(name="...", ...)   # Update share
w.shares.delete(name="...")        # Delete share
w.shares.share_permissions(name)   # Get share permissions
w.shares.update_permissions(...)   # Update permissions

# Recipients
w.recipients.list()                # List all recipients
w.recipients.get(name="...")       # Get recipient details
w.recipients.create(name="...", ...)  # Create recipient
w.recipients.rotate_token(name)    # Rotate token
w.recipients.delete(name="...")    # Delete recipient

Full reference: Sharing module documentation

Marketplace operations

# Provider operations
w.provider_listings.list()                    # List your listings
w.provider_listings.get(id="...")             # Get listing details
w.provider_personalization_requests.list()    # List access requests
w.provider_exchanges.list()                   # List private exchanges

# Consumer operations
w.consumer_listings.list()                    # Browse listings
w.consumer_listings.get(id="...")             # Get listing details

Full reference: Marketplace module documentation

What's next

• Set up monitoring to track automated operations
• Review the operations runbook for governance around automation
• Learn about recipients and credential management