Error: Data Lake Tables Are Corrupted

Issue

The sync fails with the following error for one or more tables:

We couldn't write the following tables as their underlying files are missing in your cloud storage bucket: <table_names>

Environment

Destination: Managed Data Lake

Resolution

Before you recover the affected tables, stop any external process that modifies files in the Fivetran-managed storage location. These processes include lifecycle policies, cleanup jobs, external maintenance operations, and recursive sync jobs with delete options.

Do not manually modify files in the storage location except as instructed in the recovery steps below.

Use one of the following recovery methods, depending on whether object versioning is enabled for your cloud storage bucket or container.

Recover tables without object versioning

If object versioning is not enabled, do the following:

1. Contact Fivetran Support to identify the affected tables.
2. Delete the underlying files associated with the affected tables from your cloud storage.
3. Remove the corresponding table entries from the Fivetran Catalog. See Remove table entries from the Fivetran Catalog.
4. Re-sync the affected tables.

Remove table entries from the Fivetran Catalog

Use one of the following methods to remove the affected table entries.

Use Fivetran Support

Fivetran Support can remove the table entries from the Fivetran Catalog on your behalf.

Use Fivetran write credentials

With Fivetran write credentials, you can remove the table entries yourself.

On the catalog integration page for your Managed Data Lake destination, find the catalog host and catalog name. Use the catalog host for <catalog_host> and the catalog name for <catalog_prefix> in the following commands.

1. Get an Apache Polaris™ OAuth token using your write-enabled credentials.

   curl -s -X POST "<catalog_host>/v1/oauth/tokens" \
     -d "grant_type=client_credentials" \
     -d "client_id=<client_id>" \
     -d "client_secret=<client_secret>" \
     -d "scope=PRINCIPAL_ROLE:ALL"
   

2. Unregister the affected table through the Polaris REST API. This operation does not delete the underlying files.

   curl -s -w "\nHTTP_STATUS:%{http_code}\n" -X DELETE \
     "<catalog_host>/v1/<catalog_prefix>/namespaces/<namespace>/tables/<table>?purgeRequested=false" \
     -H "Authorization: Bearer <token>"
   

Recover tables with object versioning

If object versioning is enabled for your cloud storage bucket or container, restore the missing files to recover the affected tables.

Contact Fivetran Support to get a complete list of the affected tables. Then, restore the deleted files using the recovery options provided by your cloud storage provider:

• For Amazon S3, restore previous versions using object versioning.
• For Azure Data Lake Storage, restore previous versions using blob versioning or soft delete.
• For Google Cloud Storage, restore previous versions using versioned objects or soft delete.

Fivetran Support can also help restore the missing files if you provide the required access to your destination.

If the issue persists after you restore the missing files, follow the steps in Recover tables without object versioning to recreate and re-sync the affected tables.

Cause

This issue occurs when one or more files referenced by Fivetran-managed table metadata are deleted from your cloud storage bucket or container.

Managed Data Lake relies on data and metadata files in cloud storage to maintain and query tables. If a referenced file is missing, Fivetran cannot safely write to the affected table.

A sync may continue to succeed after a file is deleted if the incoming data does not require Fivetran to access that file. The error occurs when a subsequent sync needs to read or modify data associated with the missing file.

Prevention

To prevent this issue from recurring, ensure only Fivetran-managed processes modify files in the Managed Data Lake storage location:

• Exclude the Fivetran-managed bucket, container, or prefix from lifecycle management, retention cleanup, cost optimization, backup synchronization, and external table maintenance jobs.
• Do not manually delete, move, rename, overwrite, compact, or vacuum files in the Fivetran-managed storage location.

Legal notices

Apache Polaris is a trademark of the Apache Software Foundation.