Webhooks

Our Webhook API is a RESTful API accessed by using either an HTTP client, such as wget or curl, or an HTTP library, which is a part of most modern programming language packages.

Our webhooks test project is a simple example that demonstrates how to ingest webhooks from Fivetran.

Create webhooks

Learn how to create and manage webhooks for Fivetran events.

TIP: For examples of how to set up and use webhooks, see the code sample for your environment:

• Python/Flask
• NodeJS/Express

Creating a webhook is a two-step process:

1. Set up your application to receive webhook payloads at a secure, public URL.
2. Create a webhook with Fivetran to fire in response to particular events.

The Fivetran Webhook REST API lets you manage webhooks. You can also use the REST API to change the configuration of your webhook. For example, you can modify the payload URL and the list of events.

URL

The payload URL must be a secure site (HTTPS). If an insecure (HTTP) payload URL is received, Fivetran returns an HTTP 400 Bad Request error. During the webhook creation or activation, we perform a single POST request to the specified URL with a testing payload. If endpoint is not reachable or returns an unsuccessful status code, Fivetran does not create the webhook and returns an HTTP 400 Bad Request error. You can skip this check by setting active to false during the webhook creation, but we still perform this test during the webhook activation.

Active

You can disable the delivery of webhook payloads by setting active to false.

Events

When configuring a webhook, you can use the API to choose which events send you payloads. Subscribing to only specific events you want to listen for minimizes the number of HTTPS requests to your server.

We support the following events:

• sync_start
• sync_end
• status (deprecated)
• dbt_run_start
• dbt_run_succeeded
• dbt_run_failed
• connection_successful
• connection_failure
• create_connector
• pause_connector
• resume_connector
• edit_connector
• delete_connector
• force_update_connector
• resync_connector
• resync_table

NOTE: Fill out our survey if you want more event types supported by our Webhook Management API.

Retries

Sometimes, a webhook that we sent for a particular event instance (for example, a transformation's start or a connector's sync end) may fail for various reasons, such as the relevant endpoint being unavailable, etc. In that case, we retry the webhook for up to 24 hours in increasing time intervals. We stop sending the webhook for the given event instance after retrying for 24 hours.

We use an exponential algorithm to calculate the intervals between the retries. We wait 6 minutes before the first retry, 21 minutes before the second retry, and so on. This approach allows us to make up to five webhook retries within the 24-hour period.

See how Fivetran calculates the times of retries following a failed initial attempt in the following example:

NOTE: For each webhook event type within a given context (a given connector/transformation in a given group in your account), we send webhooks for each event instance independently of the other instances of that event. For example, you created a sync start webhook for a connector with the sync frequency of 6 hours. At the sync start at 00:12, Fivetran sent a sync start webhook that failed. Regardless of the number of retries and the final status of this webhook, Fivetran still will send another sync start webhook at the next sync start at 6:12.

Connection request timeout

Webhooks have a 10-second connection request timeout. If your endpoint doesn't return an HTTP 200 OK status in 10 seconds, the webhook request fails, and Fivetran retries it based on the schedule outlined in Retries.

Automatic deactivation

If a webhook consistently fails for more than three days and only responds with non-2xx codes, we deactivate it.

To activate the webhook again, ensure that the setup is correct by retrieving the webhook details and updating them with corrected information along with passing "active": true in the request. Alternatively, you can create a new webhook.

Signing

The webhook payload can be signed if you specified a secret during the webhook creation. The signature is calculated based on the payload body using the SHA-256 HMAC algorithm.

Example of webhook payload

Fivetran sends webhooks with the following payload structure:

{
  "event" : "sync_end",
  "created" : "2021-08-18T11:38:34.386Z",
  "connector_type" : "asana",
  "connector_id" : "mystified_presiding",
  "connector_name" : "mystified_presiding",
  "sync_id" : "mystified_presiding",
  "destination_group_id" : "deck_enjoy",
  "data" : {
    "status" : "SUCCESSFUL"
  }
}

Fields