Databricks Setup Guide

Follow our setup guide to connect Databricks to Fivetran.

---

Prerequisites

To connect Databricks to Fivetran, you need:

• A Databricks account
• A Databricks cluster (version 7.1, 7.2, or 7.3)

---

Setup instructions

Create new destination

1. Go to your Fivetran account page.

2. Click + Destination.

3. In the destination drop-down menu, select Databricks on AWS or Databricks on Azure to launch the setup form. Keep this tab open.

Create a Databricks Cluster

1. In a new tab, log in to your Databricks account.

2. In the Databricks console, click Clusters in the left menu.

   

3. Click + Create Cluster.

   

4. Enter a Cluster name of your choice.

5. Set the Databricks Runtime Version to 7.1 or later.

   

6. Under Advanced Options, select Spark.

   

7. Paste the following code in the Spark Config box:

   spark.databricks.delta.alterTable.rename.enabledOnAWS true
   spark.hadoop.fs.s3a.impl shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystem
   spark.hadoop.fs.s3n.impl shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystem
   spark.hadoop.fs.s3n.impl.disable.cache true
   spark.hadoop.fs.s3.impl.disable.cache true
   spark.hadoop.fs.s3a.impl.disable.cache true
   spark.hadoop.fs.s3.impl shaded.databricks.org.apache.hadoop.fs.s3a.S3AFileSystem
   

8. Paste the following code in the Environment Variables box:

   AWS_STS_REGIONAL_ENDPOINTS="us-east-1"
   

Get credentials

1. In the same Advanced Options window, select JDBC/ODBC.

   

2. Make a note of the following values. You will need them to fill in your Fivetran Databricks setup form.

   • Server Hostname
   • Port
   • HTTP Path

Create a personal access token

Follow Databricks' token management guide to create a new personal access token.

Finish Fivetran configuration

1. Return to the tab with your Fivetran setup form.

2. Enter the Server Hostname, Port, and HTTP Path you found in Step 3.

3. Enter the Personal Access Token you created in Step 4.

4. (Optional) Set the Create Delta tables in an external location toggle to ON to create Delta tables as external tables. You can:

   • Enter the External Location you want to use. We will create the Delta tables in the {externallocation}/{schema}/{table} path.

   • Use the default Databricks File System location registered with the cluster. Do not specify the external location. We will create the external Delta tables in the /{schema}/{table} path.

5. Click Save & Test.