Tech Summit DEMO
Databricks Tech Summit FY2024 DEMO:
This demo will launch auto generated tables(100s) inside single bronze and silver Lakeflow Declarative Pipeline using dlt-meta.
Launch Command Prompt
Install Databricks CLI
- Once you install Databricks CLI, authenticate your current machine to a Databricks Workspace:
databricks auth login --host WORKSPACE_HOST
Install Python package requirements:
# Core requirements pip install "PyYAML>=6.0" setuptools databricks-sdk # Development requirements pip install flake8==6.0 delta-spark==3.0.0 pytest>=7.0.0 coverage>=7.0.0 pyspark==3.5.5
Clone dlt-meta:
git clone https://github.com/databrickslabs/dlt-meta.git
Navigate to project directory:
cd dlt-meta
Set python environment variable into terminal
dlt_meta_home=$(pwd)
export PYTHONPATH=$dlt_meta_home
Run the command:
python demo/launch_techsummit_demo.py --uc_catalog_name=<<Unity Catalog name>> --cloud_provider_name=aws
- uc_catalog_name : Unity Catalog name
- cloud_provider_name : aws or azure
- you can provide
--profile=databricks_profile name
in case you already have databricks cli otherwise command prompt will ask host and token