Intermittent Reconnect or Timeout Errors

Issue

The Amazon S3 connector periodically fails with reconnect or timeout errors even though connection tests succeed and syncs complete successfully when new files are available.

Environment

Connector: Amazon S3

Resolution

To resolve this issue, reduce the number of objects your connection scans during each sync:

1. In Fivetran, go to your Amazon S3 connection page.
2. Select the Settings tab.
3. Click Edit connection.
4. In the Base folder path field, enter the lowest common folder path that contains all the files you want to sync. For more information, see our Amazon S3 Setup Guide.
5. Click Save & Test.
6. In your S3 bucket, organize files into partitioned folders, such as folders by date, to reduce listing overhead.
7. Archive or move older files that you no longer want to sync.

After you reduce the number of files scanned, Fivetran can complete operations more efficiently and avoid intermittent reconnect or timeout events.

Cause

This issue occurs when we scan an entire S3 bucket that contains a large number of files. The increased object listing and processing overhead can extend sync duration, causing intermittent timeout or reconnect errors.