Records Are Missing From the Destination

Issue

Some records that exist in an Epic Clarity source table are missing from the corresponding destination table. Incremental syncs complete successfully, but the missing records aren't captured.

Environment

• Connector: Epic Clarity
• Incremental sync method: Change Sync Administration (CSA)

Resolution

To troubleshoot this issue:

1. In Fivetran, check whether the connection was paused or unable to sync for an extended period.

   i. Go to your Epic Clarity connection page.

   ii. In the top right, ensure the Enabled toggle is set to ON. If it is off, the connection is paused.

   iii. Select the Status tab.

   iv. In the Sync History chart, look for Connection Paused by <user> actions and failed syncs.

   v. Make a note of the relevant dates and times.

2. Ask your database administrator to confirm the configured retention period for the applicable CSA_* pair table.

3. Run the following query on the source to determine the range of change records still available. Replace <table_name> with the relevant CSA table name and <timestamp_column> with the column containing each row's update timestamp:

   SELECT
     MIN(<timestamp_column>) AS oldest_record,
     MAX(<timestamp_column>) AS newest_record,
     COUNT(*) AS row_count
   FROM EPIC_UTIL.CSA_<table_name>;
   

4. Compare the query results with the period identified in step 1. If the connection couldn't sync for longer than the retention period, the required change records may have been purged.

5. Re-sync the affected tables. For instructions, see How to Trigger a Historical Table Re-sync.

6. Confirm that the missing records are present by comparing record counts or unique record identifiers in the source and destination.

To prevent the issue from recurring, don't pause the connection for longer than the CSA retention period. If you expect a longer pause, ask your database administrator whether they can increase the retention period.

Cause

For source tables synced using CSA, Fivetran uses the corresponding EPIC_UTIL.CSA_<table_name> pair table to identify changes. For more information, see CSA and Teleport.

If the connection is paused or unable to sync for longer than the CSA retention period, Epic Clarity may purge change records before we capture them. As a result, records are no longer available when syncing resumes, and a historical re-sync is required to retrieve the missing source data.