Error Handling

When building connectors with the Connector SDK, it's important to plan how your connector behaves when something goes wrong. Effective error handling helps ensure reliable syncs, prevent silent data loss, and surface issues quickly.

Error handling determines what the connector does next when an error occurs. It is commonly used together with logging, which helps you understand connector behavior during a sync, including progress, retry attempts, skipped records, and errors. Different log severity levels make it easier to diagnose issues and understand connector behavior. For log levels, rate limits, and how logs appear in your Fivetran dashboard, see Connector SDK Logging.

Choose the right behavior

During a sync, Fivetran runs your connector code. If your code raises an unhandled exception, the sync fails and Fivetran shows the stack trace in the dashboard. You can catch errors and log clear, actionable messages instead to make issues easier to diagnose. In some cases, you can keep the sync running by retrying failed requests or skipping bad rows or endpoints to increase the amount of valuable data that syncs despite the error. If you do this, add clear log messages so users know what happened and which data the connector skipped.

Error handling falls into one of three response patterns:

• Retry when you suspect that the the issue is temporary and likely to succeed on a later attempt.
• Warn, log, and continue when the sync can still deliver useful partial data but the user should know that something was skipped or degraded.
• Fail the sync when you suspect that the data delivered will have critical data integrity issues.

The following sections outline effective error-handling patterns; adapt them to your source's specific behavior.

Retry temporary errors

Some errors are temporary, and the code that failed may succeed if you retry it during the same sync, helping you avoid unnecessary delays in delivering data. A common approach is exponential backoff — retry logic with increasing delays between attempts, such as 1 second, then 2, then 4 seconds. If the source returns a Retry-After header, use that value instead. See the complex error handling with multithreading example for an implementation using exponential backoff.

Warn and continue for partial sync failures

Sometimes part of a sync cannot complete, but the rest can still deliver usable data. In these cases, a good approach is to handle the error, warn the user, and continue the sync. Ensure that any data you deliver remains consistent and accurate.

Examples include:

• One endpoint fails while the rest of the source syncs successfully
• A small number of malformed rows can be skipped
• Optional enrichment or auxiliary data cannot be fetched
• One child object type fails while the main dataset succeeds

In these cases, continue only if the unaffected data is still useful and correct, make the condition visible through logging. Avoid silently dropping important data.

Skip only when the problem affects a small number of records and the rest of the sync can continue to deliver useful data. Consider what frequency of skipped rows or partial failures justifies failing the sync instead of continuing. The right choice depends on your data source and business context.

Fail fast for unrecoverable errors

Immediately fail for errors that require user or developer action. Failing fast with an error on the dashboard increases visibility and prevents the connector from spending time on tasks that cannot succeed.

To fail a sync and surface the error in the dashboard, raise an exception such as:

raise RuntimeError("Missing required config key: api_token")

Error handling examples

For working implementations of the common error handling patterns, see:

• Errors: Configuration validation, state validation, and centralized error handling.
• Complex error handling with multithreading: Retry logic, HTTP 429 error handling, HTTP 5xx vs. 4xx behavior, and timeout protection.