Capabilities for Databricks

This section lists the Capabilities of Fivetran HVR when using 'Databricks'. For more information about the pre-requisites, access privileges, and other configuration requirements, see Databricks Requirements.

Capture

HVR does not support Capture changes from location on Databricks.

Repository Database

HVR does not support Repository database on Databricks.

Integrate

HVR supports the following capabilities on Databricks:

• Integrate changes into location (Databricks from version 9.0 to version 11.2).
• Burst integration (Integrate with parameter Method=BURST) (Databricks from version 9.0 to version 11.2).
• Integrate with parameter BurstCommitFrequency (Databricks from version 9.0 to version 11.2).
• Continuous integration (Integrate with parameter Method=CONTINUOUS) (Databricks from version 9.0 to version 11.2).
• ColumnProperties with parameter SoftDelete (Databricks from version 9.0 to version 11.2).
• Bulk load requires a staging area (location property Staging_Directory) (Databricks from version 9.0 to version 11.2).
• Creation and update of HVR state tables (Databricks from version 9.0 to version 11.2).

HVR does not support the following capabilities on Databricks:

• Tables without a key and without TableProperties with NoDuplicateRows for continuous integration.
• Continuous integrate with parameter OnErrorSaveFailed.
• Disable/enable database triggers during integrate (with parameter NoTriggerFiring).
• Integrate with parameter DbProc.

Bi-directional Replication

HVR does not support Bi-directional Replication on Databricks.

Refresh and Compare

HVR supports the following capabilities on Databricks:

• hvrrefresh or hvrcompare from source location (Databricks from version 9.0 to version 11.2).
• hvrrefresh into target location (Databricks from version 9.0 to version 11.2).
• Row-wise hvrrefresh into target location (option -g) (Databricks from version 9.0 to version 11.2).

HVR does not support the following capabilities on Databricks:

• Bulk load of multiple slices of same table can be done in parallel.

Other Capabilities

HVR supports the following capabilities on Databricks:

• International table and column names where DBMS is not configured with UTF-8 encoding (Databricks from version 9.0 to version 11.2).
• Always treat DBMS table names and column names as case sensitive, even if not configured (equivalent to location property Case_Sensitive_Names always set to TRUE) (Databricks from version 9.0 to version 11.2).
• Always treat DBMS schema names case sensitive (Databricks from version 9.0 to version 11.2).

HVR does not support the following capabilities on Databricks:

• Use distribution key for parallelizing changes within a table (ColumnProperties with parameter DistributionKey).
• Lossless binary float datatypes. No data loss when transporting float values (because base 2 fractions are never converted to base 10). The values for these datatypes, when selected back from the database, might be slightly different from the ones that were inserted. This leads to Compare differences..