Webhooks Setup Guide

Follow our setup guide to connect Webhooks to Fivetran.

Prerequisites

To connect Webhooks to Fivetran, you need an HTTP client setup on your website, application, or any other place.

IMPORTANT: Only use an HTTPS URL with TLS version 1.2 or above.

Setup instructions

Configure container service

Fivetran allows you to store your data in multiple storage container services. We support the following storage services for storing your data:

Fivetran's container service
AWS S3
Azure Blob Storage
Google Cloud Storage

If you select Fivetran's storage service, you do not need to configure anything. By default, we use this storage service to store your data. However, if you choose any other external storage service, you need to configure it to connect to Fivetran.

Follow the instructions below to configure external container services:

Configure Amazon S3 bucket

Find external ID

If you choose AWS S3, find the automatically-generated External ID and make a note of it. You will need it to configure AWS to connect with Fivetran.

Create IAM policy

To allow Fivetran access to your S3 bucket, follow the instructions below.

Open the Amazon IAM console.
Go to Policies, then select Create Policy.
Go to the JSON tab.

Copy the following policy, replacing {your-bucket-name} with the name of your S3 bucket.

{
   "Version": "2012-10-17",
   "Statement": [
            {
            "Effect": "Allow",
            "Action": [
               "s3:Get*",
               "s3:List*",
               "s3:PutObject"
            ],
            "Resource": "arn:aws:s3:::{your-bucket-name}*"
         }
   ]
}

Click Review Policy.
Name the policy "Fivetran-S3-Access".
Click Create Policy.

Create IAM role

Go to Roles, then select Create New Role.
Select Another AWS Account, then enter Fivetran's account ID, 834469178297, in the Account ID field.
Select the Require external ID checkbox.
Enter the External ID you found in Step 1.
Click Next: Permissions.
Select the 'Fivetran-S3-Access' policy that you created in here.
Click Next: Tags, which is optional.
Click Next: Review.
Name your new role 'Fivetran', then click Create Role.
Select the Fivetran role you just created.
Find the Role ARN and make a note of it. You will need to enter this value in the connector setup form.

Configure Azure Blob storage container

Generate SAS URL

If you have selected Azure Blob Storage as custom external bucket, you need to generate a SAS URL as follows:

Inside your Azure Blob Storage, click Shared Access Token and select the Read, Write and List permissions.
Click Generate SAS token and URL and copy the value displayed in the Blob SAS URL field.

Configure Google Cloud Storage bucket

Find email

Find the automatically generated email in your connector setup form and make a note of it. You will need it to edit your Google Cloud Storage bucket permissions.

Authorize service account

Go to your Google Cloud Storage console.
Select the bucket you want to use.
Go to Permissions and then click Add Members.
In the Add members window, enter the service account you found in Step 1.
From the Select a role drop-down, select Storage Object Admin.

NOTE: Make sure you don't have any Life Cycle Management Policies configured in your external storage service.

Finish Fivetran configuration

In the connector setup form, enter the Destination schema name of your choice.
Enter the Destination table name.
Select the Sync format in which form you'd like your data to be loaded, Packed or Unpacked. For more information, see the Sync Overview section.
Find the Webhook URL and make a note of it. The URL is in the format https://webhooks.fivetran.com/webhooks/<endpoint_ID>. You will need it to configure Webhooks to connect with Fivetran.
Select the container service you configured.
(Optional) Select the Authentication mechanism you want to use to authenticate incoming events:
IMPORTANT: If you choose HMAC or token-based authentication, a Secrets field will be displayed. This secret will be masked after you click Save & Test in the setup form.
- None: Fivetran doesn't authenticate incoming events.
- HMAC: Fivetran uses an HMAC (Hash-based Message Authentication Code) signature to authenticate.
  - HMAC signature header: Provide the header key that will be used to retrieve the HMAC signature.
  NOTE: You must pass the signature in the header field you provided, in each webhook event.
  - HMAC algorithm: Choose the algorithm used to generate the HMAC signature. Possible values are as follows: Use the HMAC signature header, MD5, SHA_1, SHA_224, SHA_256, SHA_384, SHA_512
  NOTE: If you select the Use the HMAC signature header, Fivetran will expect the header to follow the pattern {algorithm}={signature}. The algorithm will be inferred from the header itself.
  - Signature encoding format: Choose the signature-encoding format used to generate the HMAC signature. Possible values Base64, Hexadecimal
  - Secret key: Enter the shared secret key to be used with the hash algorithm to create signatures.
- Token Based: Fivetran uses a secret token to authenticate.
  - Secret token header name: Provide the header key that will be used to retrieve the secret.
  - Secret: Shared secret which will be passed in the header specified above, in each webhook event.
    NOTE: You must pass the secret in the header field you provide, in each webhook event. Fivetran accepts the event only if the secret matches the one provided in the setup form.
If you have selected a custom external bucket:
- For AWS S3 - Enter the S3 bucket name and the Role ARN that you created in AWS.
- For Azure Blob Storage - Enter the Azure Blob SAS URL.
- For Google Cloud Storage - Enter the GCS Bucket.
Click Save & Test. Fivetran will load events into your destination.

IMPORTANT: It takes us around 10 to 15 minutes to load the first event into your destination.

Fivetran tests and validates the Webhooks connection. On successful completion of the setup tests, you can sync your Webhooks data to your destination.

Setup tests

Fivetran performs the following Webhooks connection tests:

The Validating read and write access test for Azure checks whether the connector can successfully read and write data to your Azure Blob Storage container. We perform this test only if you choose Azure Blob Storage as the container service.
The Validating read and write access test for GCS checks whether the connector can successfully read and write data to your GCS bucket. We perform this test only if you choose Google Cloud Storage as the container service.
The Validating read and write access tests for AWS S3 checks whether the connector can successfully read and write data to your S3 bucket. We perform this test only if you choose AWS S3 as the container service.

Connector Overview

Schema Information

Release Notes

API Connector Configuration

Documentation Home