Error: Delta Unsupported Time Travel Beyond Deleted File Retention Duration

Issue

Syncs fail with the following error:

Error running query: [DELTA_UNSUPPORTED_TIME_TRAVEL_BEYOND_DELETED_FILE_RETENTION_DURATION] Cannot time travel beyond delta.deletedFileRetentionDuration (168 HOURS) set on the table

Environment

Connector: Databricks

Resolution

To resolve this issue:

1. Re-sync the affected tables individually, or re-sync all historical data if multiple tables are affected. For instructions, see How to Trigger Historical Re-Syncs for Fivetran Connections and Tables.
2. Increase delta.deletedFileRetentionDuration on the source Delta tables so that it exceeds the longest expected sync lag or pause duration. Ensure that VACUUM operations or lifecycle policies don't reduce the effective retention period:

   ALTER TABLE <catalog>.<schema>.<table> SET TBLPROPERTIES ('delta.deletedFileRetentionDuration' = 'interval 30 days');
   

Cause

This issue occurs when we attempt to read a Delta table version that is older than the table's file retention window. Incremental syncs rely on Delta Change Data Feed and versioned reads. We read data at a specific table version, then use table_changes() to read changes between versions.

If you pause your connection or it falls behind long enough for the required older files to be deleted, Databricks blocks the read and returns the above error. The delta.deletedFileRetentionDuration property controls the file retention window and defaults to 168 hours.