> ## Documentation Index
> Fetch the complete documentation index at: https://docs.wisdom.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Connect to Databricks

WisdomAI connects to Databricks using a **Client ID** and **Client Secret** for authentication. This method ensures secure and granular control over the data WisdomAI can access.

## Prerequisites

Before you begin, make sure you have the following:

* An active Databricks workspace.
* Permissions to create and manage **Service Principals** within your Databricks account or workspace.
* A Databricks **SQL Warehouse** set up and running.

## 1. Obtain Databricks connection details

You will need the following four pieces of information from your Databricks environment:

1. **Host**: The URL of your Databricks workspace, which typically looks like `https://<region>.cloud.databricks.com/`. You can find this in your SQL Warehouse connection details.
2. **HTTP Path**: The specific path to your SQL Warehouse, usually starting with `/sql/1.0/warehouses/`. You can find this in the "Connection Details" tab of your SQL Warehouse.
3. **Client ID**: The Application ID of your Databricks Service Principal.
4. **Client Secret**: The secret key generated for your Databricks Service Principal.

### How to find your connection details

Follow these steps to find the necessary credentials.

#### Host and HTTP path

1. Navigate to your Databricks Workspace.
2. Go to **SQL Warehouses** (or SQL Endpoints).
3. Select the warehouse you want to connect to.
4. Click on the **Connection Details** tab. Here you will find the **Server Hostname** (your Host) and the **HTTP Path**.

#### Client ID and client secret (using a Databricks-managed Service Principal)

You'll first need to create a Service Principal in Databricks.

**Step 1: Create a Service Principal**

1. Log in to your Databricks environment.
   * **A) account-level permissions:** Log in to your Databricks **account console**.
   * **B) administrator-level permissions:** Log in to your Databricks **workspace** as an administrator.
2. Navigate to the Service Principals section.
   * **A) From Account Console:** Navigate to **User management > Service principals**.
   * **B) From Workspace:** Navigate to **Admin Settings** (click your username in the top right, then Admin Settings) > **Identity and access > Service principals**.
3. Click **Add service principal**.
4. Select **Databricks managed**.
5. Provide a descriptive **Name** for the service principal (e.g., WisdomAI-ServicePrincipal).
6. Click **Add service principal**.
7. Once created, you will see the **Application ID** for this service principal. This is your **Client ID**. Copy this value.

**Step 2: Generate a Client Secret for the Service Principal**

1. From the service principal's page, go to the **Secrets** tab.
2. Under OAuth secrets, click **Generate secret**.
3. Set a lifetime for the secret (up to 730 days) and click **Generate**.
   <Note>
     Choose an expiration that balances security and operational convenience.
   </Note>
4. Immediately copy the displayed secret. This is your **Client Secret**.

<Warning>
  The Client Secret is only shown once and cannot be retrieved later. Please store it in a secure location immediately.
</Warning>

## 2. Assign permissions in Databricks

The Service Principal needs specific permissions to access the required data that WisdomAI needs.

* **SQL Warehouse/Cluster Permissions**: Grant **CAN USE** permission to the Service Principal on the SQL Warehouse you are connecting to. You can do this from the **Permissions** tab of the SQL Warehouse.
* **Data Privileges**: The service principal requires the following privileges in Unity Catalog (or Hive Metastore):
  * `USE CATALOG` on the target catalog.
  * `USE SCHEMA` on the target schema.
  * `SELECT` on the tables and views you want WisdomAI to access.

You can grant these permissions using SQL commands. Here are some examples:

```sql theme={null}
-- Grant permission to use a specific catalog
GRANT USAGE ON CATALOG <your_catalog_name> TO `<service_principal_id>`;

-- Grant permission to use a specific schema
GRANT USAGE ON SCHEMA <your_catalog_name>.<your_schema_name> TO `<service_principal_id>`;

-- Grant select permission on table
GRANT SELECT ON TABLE <your_catalog_name>.<your_schema_name>.<your_table_name> TO `<service_principal_id>`;

-- Grant select permission on all tables in a schema
GRANT SELECT ON ALL TABLES IN SCHEMA <your_catalog_name>.<your_schema_name> TO `<service_principal_id>`;
```

Replace `<service_principal_id>` with the Application ID of your service principal, and `<your_catalog_name>`,`<your_schema_name>`, and `<your_table_name>` with your specific values.

## 3. Connect WisdomAI to Databricks

Once you have gathered the required information, you can configure the connection in WisdomAI:

1. In WisdomAI, navigate to the **Connections** section and click **Add Connection**.
2. Select **Databricks** as the data source type.
3. Fill in the connection details in the "Databricks connection details" section.
   * **Connection Name**: Choose a descriptive name for your connection (e.g., "Databricks Prod").
   * **Host**: Paste the Server Hostname you obtained from Databricks.
   * **HTTP Path**: Paste the HTTP Path for your SQL Warehouse.
   * **Client ID**: Enter your Client ID (Application ID). This field is mandatory.
   * **Client Secret**: Enter your Client Secret (the value copied immediately after creation). This field is mandatory.
   * **Catalog Filters (Optional)**: Specify any Catalog Filters if you want to further restrict the catalogs/schemas WisdomAI crawls.
4. Click **Save and Sync metadata**. WisdomAI will use the provided credentials to connect to Databricks and scan the metadata of the specified catalogs and tables.

<img src="https://mintcdn.com/wisdomai/FPeFp9c8fz1r_sxU/dark-img/databricks-dark.png?fit=max&auto=format&n=FPeFp9c8fz1r_sxU&q=85&s=00c52c1a34183e262b9cca97602c4f52" alt="image.png" lightAlt="image.png" darkAlt="image.png" className="hidden dark:block" width="3002" height="1720" data-path="dark-img/databricks-dark.png" />

<img src="https://mintcdn.com/wisdomai/XePopA6v78ArzXvA/images/databricks.png?fit=max&auto=format&n=XePopA6v78ArzXvA&q=85&s=d86994e56626335aaf732bd93d0943ce" alt="image.png" lightAlt="image.png" darkAlt="image.png" className="dark:hidden" width="3002" height="1720" data-path="images/databricks.png" />

## Security considerations and best practices

* **Least Privilege**: Always follow the principle of least privilege. Only grant the necessary permissions to the Service Principal.
* **Secure Key Management**: Treat your Client ID and Client Secret like passwords. Avoid sharing them via insecure channels, such as unencrypted email. Use secure methods, such as LastPass or other secure file-sharing services.
* **Credential Rotation**: Regularly rotate your Client Secrets to enhance security, especially before their expiration.

## Troubleshooting common issues

Having trouble? Here are solutions to some frequently encountered problems.

### Authentication failed or invalid credentials

* Ensure the Host, HTTP Path, Client ID, and Client Secret are complete and correct. No extra characters, spaces, or missing lines.
* Verify that the Client Secret has not expired or been revoked in Databricks.
* Confirm that the Service Principal exists and is enabled in Databricks.

### Permission denied errors

* Double-check that the Service Principal has the necessary `CAN USE` permission on the SQL Warehouse.
* Verify that the Service Principal has the correct `USE CATALOG`, `USE SCHEMA`, and `SELECT` privileges on the Databricks catalogs, schemas, and tables you are trying to access.
* Confirm that the SQL Warehouse is running and accessible.

### Catalog not found or table not found

* Verify the spelling of the catalog or table names.
* If using catalog filters in WisdomAl, ensure the catalog is included in the filter.
* Confirm that the Service Principal has permissions on the specific catalog you are trying to access.

### Connection timeout or network errors

* Verify that WisdomAl's CIDR block (3`5.238.115.103/32` or `34.82.248.105/32`) is allowlisted in your Snowflake network policies or any corporate firewalls.

## Next steps

<CardGroup cols={3}>
  <Card title="Connect to Snowflake" icon="snowflake" href="/getting-started/connect-data-sources/connect-to-snowflake">
    Connect to Snowflake directly or via a proxy using key-pair authentication and a Snowflake virtual warehouse.
  </Card>

  <Card title="Connect to Google BigQuery" icon="google" href="/getting-started/connect-data-sources/connect-to-google-bigquery">
    Connect to Google BigQuery using a Google Cloud Platform Service Account for secure and granular access to your data.
  </Card>

  <Card title="Connect and Test Tutorial" icon="rocket" href="/setting-up-wisdom-ai/basic-tutorial-connect-and-test">
    Walk through the initial setup to connect a data source and run your first query.
  </Card>
</CardGroup>