Too Long Since the Last Incremental Update

Issue

The dashboard shows the following error: It has been too long since the last incremental update.

Environment

Connector: MongoDB

Resolution

This error indicates that the MongoDB oplog retention period is not long enough and needs to be increased.

Cause

If we are importing a very large table, especially during historical syncs, the amount of time it takes to import the table may be longer than the retention period for the oplogs, which we use for incremental updates.

You need to increase the retention period for the oplogs to ensure that they are still available after the table import is complete.