Why Is My Sync Taking Longer Than Normal?
Question
Why is my sync taking longer than normal?
Environment
All Connectors
Answer
You may experience longer syncs due to the following reasons:
Increase in data volume
Increasing the volume of data we need to sync may increase your sync time. To check whether an increase in data volume is impacting your sync time, do the following:
- In your connector dashboard, view the Sync History chart.
- Hover over a sync that took a long time.
- Note the sync's data volume.
- Compare the data volume to a shorter sync.
Manual schema change
Your sync time may increase if you have included an additional column or table. You can use the Fivetran Platform Connector to check whether a new column or table was included in your sync. See our track schema changes documentation for more information.
Automatic schema change
If your connector's schema change settings are set to Allow all, we will automatically sync all newly detected columns and tables. If we detect and sync new, large tables, you may experience longer sync times. To view each connector's new tables and columns, do the following:
- In your Fivetran dashboard, select Account Settings and then click Billing & Usage.
- Click the Usage tab.
- Go to the Usage breakdown section and click the MAR type dropdown.
- Select the Initial Sync (Free) checkbox.
Significant change in source data
Significant changes in your source data, such as mass deletes or mass record updates, may increase the number of changes we need to sync and, thus, the sync time. You can use the Fivetran Platform Connector to check for significant changes that may impact your sync time. Fivetran generates records_modified
log events that include a count
field, representing the number of operations modified.
You can also check how many records were updated in your destination by querying the updated_at
column with a WHERE
clause. However, note that child tables may not include an updated_at
column if they update automatically in conjunction with their associated parent record.
Network latency
Intermittent network latency may impact your sync speed. This may occur when we connect to your source database connector or your destination database. Check with your network team to verify if there are known or potential network latency issues.
API rate limits (application connectors only)
If you have reached the source's API rate limit, we may delay your sync until we can query the API again. You can use the Fivetran Platform Connector to check whether an API rate limit may have impacted your sync time. Fivetran generates warning
log events that provide detailed information about the warning and affected tables.