Databrickslink
The Databricks Lakehouse Platform combines the key features of data lakes and data warehouses. It supports multiple data workloads including analytics, business intelligence, data engineering, data science, and machine learning. The platform is built on open source and open standards.
Supported implementationslink
Fivetran supports connecting with three different Databricks implementations:
Setup guidelink
Follow our step-by-step setup guide to connect Databricks on AWS , Databricks on Azure, or Databricks on Google Cloud Platform to Fivetran.
Type transformation mappinglink
The data types in Databricks follow Fivetran's standard data type storage.
We use the following data type conversions:
Fivetran Data Type | Destination Data Type | Notes |
---|---|---|
BOOLEAN | BOOLEAN | |
SHORT | SMALLINT | |
INT | INT | |
LONG | BIGINT | |
BIGDECIMAL | DECIMAL | |
FLOAT | FLOAT | |
DOUBLE | DOUBLE | |
LOCALDATE | DATE | |
LOCALDATETIME | TIMESTAMP | Databricks requires time zone value |
INSTANT | TIMESTAMP | |
STRING | STRING | |
JSON | STRING | Databricks doesn't support JSON |
BINARY | BINARY |
Column nameslink
Fivetran ignores the case of column names in your destination tables as Databricks is case-insensitive.
Table maintenancelink
We perform weekly maintenance operations on the Delta tables. We run the following operations during the weekend:
NOTE: You may observe a sync delay for your connectors while the destination table maintenance operations are in progress.
Liquid clusteringlink
Fivetran does not support liquid clustering for Delta tables.