Error: Invalid Protocol Version

Issue

After enabling Change Data Feed (CDF) on an Azure Data Lake Storage (ADLS) Delta Lake table through Databricks, Fivetran fails to write to the table. The following error appears:

Invalid protocol version

Environment

• Destination: Managed Data Lake Service (ADLS)
• Fivetran-managed Delta Lake tables with Change Data Feed (CDF) enabled through Databricks

Resolution

To resolve this issue:

1. In Databricks, disable CDF on the affected Delta Lake tables.

2. Revert the Delta table protocol to a version we support:

   • minReaderVersion = 1
   • minWriterVersion = 2

3. Restore each table's Delta log to its last working state before CDF was enabled. This ensures that the table metadata is compatible with Fivetran.

4. In Fivetran, re-sync the affected tables:

   i. Go to your connection page.

   ii. Select the Schema tab.

   iii. Find the affected table and click Re-sync.

   iv. In the confirmation pop-up window, click Re-sync table.

If multiple tables are affected, perform a connection-level re-sync instead. For instructions, see How to Trigger Historical Re-Syncs.

Do not re-enable CDF on Fivetran-managed Managed Data Lake Service (MDLS) tables. This configuration isn't supported. If you need to use Delta Lake CDF, use a Databricks destination, where it is supported by Fivetran.

Cause

This issue occurs when you enable CDF on a Fivetran-managed Delta Lake table through Databricks. When this happens, Delta Lake increases the table's writer protocol version. However, the Delta Standalone Java library that we use doesn't support this higher version. As a result, Fivetran can't write to the table