Skip to main content

Installation

Table of Contents​

Pre-requisites​

  1. Install Databricks CLI - Ensure that you have the Databricks Command-Line Interface (CLI) installed on your machine. Refer to the installation instructions provided for Linux, MacOS, and Windows, available here.

Installing the Databricks CLI in different OS:

macos-databricks-cli-install
  1. Configure Databricks CLI - Details can be found here. Additionally, Lakebridge requires the profile used for the Databricks CLI to specify a cluster_id, to do this, you can either:
  • Edit your ~/.databrickscfg file directly and enter a cluster_id for the profile you're using or
  • The flag --configure-cluster gives you the prompt to select the cluster_id from the available clusters on the workspace specified on the selected profile.
databricks configure --host <host> --configure-cluster --profile <profile_name>
  • Alternatively you can use the environment variable DATABRICKS_CLUSTER_ID to set the cluster id you would want to use for your profile before running the databricks configure command.
export DATABRICKS_CLUSTER_ID=<cluster_id>
databricks configure --host <host> --profile <profile_name>
  1. Python - Verify that Python 3.10 or above is installed
  • Windows - Install python from here. Your Windows computer will need a shell environment (GitBash or WSL)
  • MacOS/Unix - Use brew to install python in macOS/Unix machines

Check Python version on Windows, macOS, and Unix​

check-python-version
  1. Java - Verify that Java 11 or above is installed. This is required for the Morpheus transpiler

[back to top]

Install Lakebridge​

Upon completing the environment setup, install Lakebridge by executing the following command:

databricks labs install lakebridge
lakebridge-install

Verify Installation​

Verify the successful installation by executing the provided command; confirmation of a successful installation is indicated when the displayed output aligns with the example below:

 databricks labs lakebridge --help
verify-install

[back to top]

Install Transpile​

Upon completing the environment setup, you can install the out of the box transpilers by executing the following command. This command will also prompt for the required configuration elements so that you don't need to include them in your command-line call every time.

databricks labs lakebridge install-transpile
transpile-install

Verify Installation​

Verify the successful installation by executing the provided command; confirmation of a successful installation is indicated when the displayed output aligns with the example screenshot provided:

 databricks labs lakebridge transpile --help
transpile-help

[back to top]

Configure Reconcile​

Once you're ready to reconcile your data, you need to configure the reconcile module.

databricks labs lakebridge configure-reconcile
reconcile-configure

Verify Configuration​

Verify the successful configuration by executing the provided command; confirmation of a successful configuration is indicated when the displayed output aligns with the example screenshot provided:

 databricks labs lakebridge reconcile --help
reconcile-help

[back to top]