Amazon Redshift Profiler Details

• Prerequisites
• Configure Connection to Redshift
• Run the profiler
• Profiler output

Prerequisites

1. Environment

• Lakebridge CLI installed and configured for your Databricks workspace (same as other profiler sources).

2. Network connectivity

The machine running the profiler must reach the Redshift cluster endpoint (hostname) on the cluster port (default 5439), subject to your security groups / VPC / routing rules.

3. Authentication

During configuration you choose an authentication type and where secrets are read from (local, env, or file):

Authentication type	Typical use
sql_authentication	Native database user and password
iam	IAM-authenticated connection using AWS credential resolution

For IAM authentication you typically need:

• AWS credentials available to the process (for example AWS_PROFILE, environment variables for keys, or instance/profile credentials).
• IAM permissions allowing Amazon Redshift credential APIs appropriate for your setup (for example redshift:GetClusterCredentials where applicable).

Use local to store plaintext values in ~/.databricks/labs/lakebridge/.credentials.yml, env to substitute values from environment variables (with fallback), or file to reuse an existing credential file when it already contains valid Redshift entries.

4. Database privileges

The profiler connects as your configured user and runs read-only extracts. The pipeline includes source_ddl steps that create a helper view query_view in the database (via CREATE OR REPLACE VIEW). The connecting user therefore needs permission to:

• Create (and replace) views in the target database used for profiling.
• Select from the Amazon Redshift system relations used by the extracts (see below).

The unified pipeline standardizes on SYS monitoring views for query history and CPU charts. Objects referenced by the bundled SQL include, among others:

• sys_query_history, sys_query_detail
• sys_external_query_detail

Deployment-specific (optional) metrics — these steps are marked optional and degrade to absent when the underlying objects are not available:

Metric	Single-AZ provisioned	Multi-AZ provisioned	Serverless
Managed storage / nodes	stv_node_storage_capacity, stv_partitions	absent	sys_serverless_usage
Cost incurred	absent	absent	sys_serverless_usage

Exact object access is determined by Amazon Redshift documentation for your edition; grant minimum read access consistent with those views/tables.

tip

If you cannot grant broad catalog access, narrow to the relations used in resources/assessments/redshift/ in the Lakebridge package.

Configure Connection to Redshift

databricks labs lakebridge configure-database-profiler

Select redshift when prompted for the source system. The wizard will ask for authentication type, credential source (local | env | file), and connection details — for password auth, for example:

• Redshift cluster endpoint (host)
• Port (default 5439)
• Database name
• User and password

For IAM auth, expect prompts for optional fields such as DB user, cluster identifier, AWS profile, and region.

Example-style transcript (values are illustrative):

databricks labs lakebridge configure-database-profiler

Please select the source system you want to configure
[0] bigquery
[1] legacy_synapse
[2] mssql
[3] oracle
[4] redshift
[5] snowflake
[6] synapse
[7] teradata
Enter a number ...: 4

Redshift authentication: sql_authentication or iam.

Authentication type
[0] iam
[1] sql_authentication
Enter a number between 0 and 1: 1
Credential source (local | env | file)
[0] local
[1] env
[2] file
...

Enter the Redshift cluster endpoint (host): mycluster.abc123.us-east-1.redshift.amazonaws.com
Enter the port details (default: 5439): 5439
Enter the database name: dev
Enter the user details: profiler_reader
Enter the password details: ********

Do you want to test the connection to redshift? [y/n]: y

Run the profiler

After configuration and a successful connection test (optional):

databricks labs lakebridge execute-database-profiler --help

Run the profiler (interactive source selection):

databricks labs lakebridge execute-database-profiler

You can pass the source explicitly:

databricks labs lakebridge execute-database-profiler --source-tech redshift

There is no deployment-variant prompt. A single pipeline covers serverless, single-AZ provisioned, and multi-AZ provisioned clusters; capacity and cost extracts that do not apply to the connected deployment are reported as expected-absent.

Execution will:

1. Load pipeline_config.yml for Redshift.
2. Run source_ddl steps on the cluster (including creating/updating query_view).
3. Run SQL extracts and persist results into a timestamped DuckDB file under the selected --output-folder.

Profiler output

The profiler extract is written as a timestamped DuckDB file under the selected --output-folder and uploaded to the configured UC Volume for downstream analysis.

Back to Configure Profiler