What Are the Best Practices for Cleansing Data in the Source and Destination?

Question

I would like to cleanse test data that is present in the source (and has synced to the destination). What are the best practices?

Environment

• Data source
• Fivetran dashboard
• Data destination

Answer

Do the following:

1. Remove the unwanted (test) data from the destination.
2. Pause the connector.
3. Cleanse the source of all testing data.
4. Resume the connector. The subsequent sync will fetch recent data that does not contain the test data.

Consideration

• We assume that you want to avoid writing test data to your destination or having it exist in your destination.
• A historical re-sync is not needed because the re-sync will only refresh all the existing data in the destination. However, you can perform a historical re-sync as part of the cleansing process.
• We suggest pausing the connector, considering that cleansing the data in the source and destination will take longer than the connector's sync frequency.