Increase in Fail-Safe Backup Storage When Using Fivetran To Copy Data to the Destination

Issue

There is an increase in fail-safe backup storage when using Fivetran to copy data to the Snowflake destination.

Environment

Destination: Snowflake

Resolution

The solution is to use transient tables.

New Connectors

Registering Fivetran with a transient database will ensure that all Fivetran-created tables are transient by default. This will help in reducing the additional costs and storage space associated with fail-safe. However, transient tables cannot be recovered in case of operational or system failures.

Existing Connectors

Follow the steps below:

1. Pause the Fivetran database connector.
2. Clone the target table into a transient table. This will result in an identical, transient table. Make sure that the schema names, table names, column names, and column data types are the same.
3. Swap the tables.
4. Resume the Fivetran database connector.
5. Confirm the sync is working fine.
6. Drop the swapped-out, permanent table.

See Snowflake's Fail-safe Documentation and our Data Load Costs documentation for more information.

Cause

Snowflake defaults to fail-safe. Fail-safe is a backup/storage mechanism from Snowflake that ensures historical data is protected in the event of a system failure or other similar event.