March 2024

Fivetran Azure Central India Servers

We have added new servers in the Azure Central India (Pune) region.

If you're on a Business Critical plan, you can now select this region in your destination setup form. Connectors configured in a destination run in their designated cloud and region, meaning that your data doesn't leave our region-specific servers during processing. For example, if you configure your connectors to use our servers in India, your data does not leave India during processing.

We now support Azure Private Link connections to sources and destinations in the Azure Central India (Pune) region.

For more information, see our data residency documentation.

Fivetran Azure Japan Servers

We have added new servers in the Azure Japan East (Tokyo) region.

If you're on a Business Critical plan, you can now select this region in your destination setup form. Connectors configured in a destination run in their designated cloud and region, meaning that your data doesn't leave our region-specific servers during processing. For example, if you configure your connectors to use our servers in Japan, your data does not leave Japan during processing.

We now support Azure Private Link connections to sources and destinations in the Azure Japan East (Tokyo) region.

For more information, see our data residency documentation.

Lite Connectors

Lite connectors are now available for the following sources:

• Adobe Workfront
• Aumni
• Brightcove
• CallRail
• Culture Amp
• Kisi
• Mambu
• Megaphone
• REVIEWS.io
• Rocketlane
• SparkPost
• StarRez
• Trelica
• Udemy Business
• VWO
• Webconnex
• Younium

Organization Management

Organization Management is now generally available.

Setup Tests

Setup tests after editing connection details

We have made the following changes to the setup tests that run after you edit your connector or destination's connection details:

• We no longer remove or replace reconnection errors created after a sync failure.
• If setup tests fail, we do not add a new reconnection alert.

Transformations for dbt Core

dbt Core version 1.7.9 support

We now support dbt Core version 1.7.9. You can use this version to run your projects.

NOTE: For dbt Core versions later than 1.1.5, we don't support MySQL destinations. For dbt Core versions later than 1.4.9, we don't support SQL Server and Azure Synapse destinations. These destinations are not currently supported by custom dbt Core adapters.

* dbt Core is a trademark of dbt Labs, Inc. All rights therein are reserved to dbt Labs, Inc. Fivetran Transformations is not a product or service of or endorsed by dbt Labs, Inc.

Error management

We have enhanced our error management and have made the following improvements:

• We now handle the following errors in Quickstart data models:

  • Required fields not available error:

    

  • Source data missing error:

    

• We have added a warning for instances when removing columns from the schema results in transformation failure.

  

• We have added an option to check the list of tables and columns required for the transformation to work successfully. To see the required data, click View required fields on the transformation creation page.

  

Retries for transformations and connections managed by Transformation Scheduling

Transformations and connections managed by Transformation Scheduling may be automatically retried if they fail. Since connection syncs and transformation jobs depend on each other, the rules and conditions of their retries are complex and may change in the future. We may decide to not restart connections or transformation jobs if they have failed or if their restart attempts would cause significant delays to other scheduled jobs. We may restart both connection syncs and transformation jobs up to 4 times within their scheduled sync or run interval. For connector syncs, we follow the rules described in our Sync Overview documentation whenever possible. We restart transformation jobs 5 minutes after their failure. We are gradually rolling out this feature over the next few weeks.

SQL Server and Azure Synapse for dbt Core version 1.7.3 and 1.7.9 support

We now support SQL Server and Azure Synapse destinations for dbt Core* versions 1.7.3 and 1.7.9. You can use these versions to run your projects targeting those destinations.

NOTE: For dbt Core versions later than 1.1.5, we don't support MySQL destinations. For dbt Core versions later than 1.4.9 but older than 1.7.3, we don't support SQL Server and Azure Synapse destinations. These destinations are not currently supported by custom dbt Core adapters.

* dbt Core is a trademark of dbt Labs, Inc. All rights therein are reserved to dbt Labs, Inc. Fivetran Transformations is not a product or service of or endorsed by dbt Labs, Inc.

Support multiple targets in dbt for Core* transformations

In addition to being able to change the default target, you can now define multiple additional targets when setting up a dbt for Core* transformation.

Deleting a custom target deletes the models associated with it.

When you set up a transformation, you assign one of the targets you defined for your dbt project to be used with the model. You can set up several transformations using the same model but different targets.

Transformations that use additional targets have a pill displaying the name of the additional target used.

Target management for transformations for dbt Core

You can now edit targets when using dbt for Core* transformations. This allows you to change certain properties when using transformations without changing the associated destination's settings.

The following Snowflake destination settings are now also managed via targets:

• Host
• Port
• Database
• User
• Role
• Auth

We removed them from the dbt Transformations setup form.

To manage targets from the dbt Transformations setup form, set the Show Advanced Options toggle to ON.

When adding or editing a target, you can add and define additional parameters, and override and remove the existing ones. Refer to you destination's documentation to check which parameters stored as key-value pairs it supports by clicking the link to the specific destination's documentation in the target details pane.

To override the destination's authentication settings, set the Custom destination authentication settings toggle to ON.

To revert the destination's settings use the Restore default values button.

* dbt Core is a trademark of dbt Labs, Inc. All rights therein are reserved to dbt Labs, Inc. Fivetran Transformations is not a product or service of or endorsed by dbt Labs, Inc.

New connectors

Adyen

Adyen is a payment, revenue, and financial management platform. Read our Adyen connector documentation.

Azure DevOps

Azure DevOps is a suite of tools that help teams manage their entire software development lifecycle. Read our Azure DevOps connector documentation.

Schema changes

ADP Workforce Now

We have added the following new tables:

• WORKER_LEAVE
• WORKER_TIME_CARD
• WORKER_TIME_CARD_DAILY_TOTAL
• WORKER_TIME_CARD_HOME_LABOR_ALLOCATION
• WORKER_TIME_CARD_PERIOD_TOTAL

Adyen

We have changed the data type of the exchangeRate column in the PAYMENTS_ACCOUNTING_REPORT table from INT to DOUBLE.

Aircall

We have removed the foreign key relationship of the id column to the NUMBERS table, in the CONTACT_NUMBER table.

Appfigures

We have removed the REPLY table.

Attentive

We have changed the primary key of the SMS_INBOUND table. The new primary key is the _fivetran_id column. Previously, we used subscriber_external_id and _fivetran_id columns as the composite primary key for the table.

Auth0

We have changed the primary key of the USER_IDENTITIES table. The new primary key is the _fivetran_id column. Previously, we used index and user_id columns as the composite primary key for the table.

Braze

We have added a new column, message_extras, to the following tables:

• CONTENT_CARD_EVENT
• EMAIL_EVENT
• PUSH_NOTIFICATION_EVENT
• SMS_EVENT
• WEBHOOK_EVENT

Chameleon

We have added the following new columns to the MICRO_SURVEY_RESPONSE table:

• comment_step_dropdown_item
• comment_step_other_text
• other_text

Checkr

We have renamed the ruleset_id column to id in the ASSESSMENT table .

We have changed the primary key of the ASSESSMENT table. The new primary key is the _fivetran_id column. Previously, the primary key was a combination of the report_id and ruleset_id columns.

We have changed the primary key of the REPORT_ADDRESS table. The new primary key is the _fivetran_id column. Previously, the primary key was a combination of the _fivetran_id and report_id columns.

We have changed the composite primary key of the ASSESSMENT_RESULT table. The new composite primary key is a combination of the index and _fivetran_id columns. Previously, we used the index, report_id, and ruleset_id columns as the composite primary key.

We have changed the composite primary key of the ASSESSMENT_RESULT_ASSESSED_OBJECT table. The new composite primary key is a combination of the object_id, assessment_result_index, and _fivetran_id columns. Previously, we used the index, assessment_result_index, report_id, and ruleset_id columns as the composite primary key.

ClickUp

We have changed the primary key of the GOAL_GROUP_MEMBER_IN_GOAL_FOLDER table. We now use the index column as the primary key. Previously, we used the id column as the primary key. Be sure to update your queries.

Employment Hero

Facebook Pages

We have added the following new tables to support Facebook Reels:

• LIFETIME_REEL_METRICS_BY_TYPE
• LIFETIME_REEL_METRICS_TOTAL
• REEL_HISTORY
• UNIQUE_LIFETIME_REEL_METRICS_TOTAL

You can view the full list of metrics for reel insights in Facebook's documentation.

We are gradually rolling out this change to all our Facebook Pages connectors.

Firebase

We have updated subcollection names to begin with a forward slash ("/").

Fivetran Platform Connector

We have removed the free_type column from the deletion key of the INCREMENTAL_MAR table. Previously, the deletion key included this column, which caused data discrepancy.

Front

We have changed the primary key of the CONTACT_NOTE table. The new composite primary key is a combination of the contact_id and index columns. Previously, the index column was the primary key for this table.

Greenhouse

We have made the following schema changes:

• added the responsible column to the HIRING_TEAM table
• added the opened_at column to the JOB table

Be sure to re-sync the JOB table to backfill the historical data for these tables or contact Fivetran's support team to re-sync it for you.

HappyFox

Instagram Business

We have added a new field, story_swipe_forward, to the MEDIA_INSIGHTS table. The story_navigation_action_type field breaks down the navigation field in the MEDIA_INSIGHTS table into the following navigation actions:

• story_exits
• story_taps_back
• story_taps_forward
• story_swipe_forward

Jotform

Klaviyo

We have added a new column, joined_group_at, to the LIST_PERSON table to store the dates when a person is added to the list.

Kustomer

We have added a new column, _fivetran_deleted, to the QUEUE table to capture deletes and replace the deleted column. Also, we have removed the deleted and deleted_at columns from the QUEUE table. Be sure to update your queries.

Linear

We have added the following new tables:

• ISSUE_HISTORY_1
• ISSUE_HISTORY_2
• ISSUE_HISTORY_3
• ISSUE_HISTORY_4

We have removed the ISSUE_HISTORY table.

LinkedIn Ad Analytics

We have added the following new tables:

• LEAD_FORM_RESPONSE
• LEAD_FORM_RESPONSE_ANSWER
• LEAD_FORM_RESPONSE_CONSENT
• LEAD_FORM_RESPONSE_MULTIPLE_CHOICE_ANSWERS

NOTE: You need to re-authorize your existing LinkedIn Ad Analytics connectors to access this data.

Mux

PartnerStack Vendor

We have changed the primary key in the FORM_TEMPLATE_FIELD table. The new composite key is a combination of the form_template_id and index columns.

Salesforce Commerce Cloud

We have changed the primary key in the ORDER_SHIPMENT_ADDRESS table. The new primary key is a combination of the order_id and shipment_id columns. Previously, the shipment_id column was the primary key for this table.

TikTok Ads

We have added a new column, tiktok_item_id, to the AD_HISTORY table. For more information on this field, see TikTok ads' documentation.

Improvements

Adobe Analytics

We now support the OAuth Server-to-Server credentials authentication method.

Effective January 1, 2025, Adobe will deprecate authentication using Service Account (JWT) credentials. Therefore, we recommend that you migrate to OAuth Server-to-Server credentials to continue syncing your Adobe Analytics data.

Adobe Commerce

Our Adobe Commerce connector now supports table-level re-syncs.

ADP Workforce Now

We now support syncing the Worker TimeCard and Worker Leave endpoints. Be sure to provide the required scopes to sync data from these endpoints. For more information, see our setup instructions.

Alchemer

We have changed the connector type of our Alchemer connector from Lite to Standard. As part of the upgrade, we have added the following new tables:

• SSO
• SSO_ATTRIBUTE
• SURVEY_CAMPAIGN_EMAIL_MESSAGE

Alchemer connectors created on or after March 14, 2024, use a new schema. For more information about the new schema, see our Alchemer documentation. We are gradually rolling out these changes to all existing connectors.

Amazon Selling Partner

Our Amazon Selling Partner connector is now generally available. Read our Amazon Selling Partner documentation.

Amplitude

Fivetran now supports extracting event data from Amplitude using Cloud Storage Export. We are gradually rolling out this feature to all existing connectors.

For more information, see our Amplitude setup guide.

Assembled

We have changed the connector type of our Assembled connector from Lite to Standard. As part of this upgrade, we have made the following schema changes:

• Added the following new tables:

  • DAILY_REPORT_METRIC
  • EVENT_CHANGE
  • FORECAST
  • HOURLY_REPORT_METRIC
  • TIME_OFF_REQUEST
  • TIME_OFF_UPDATE

• Added two new columns, productivity and staffable, to the PEOPLE table

• Removed the REPORT and REPORT_METRIC tables

We are gradually rolling out these changes to all existing connectors.

BigCommerce

Our BigCommerce connector is now generally available. Read our BigCommerce documentation.

BigQuery

We are rolling out the optimized sync feature for the following types of BigQuery tables:

• regular tables
• partitioned tables that do not require a partition filter.

Once the feature is rolled out, we will automatically re-sync all the tables you have selected. These re-syncs will count towards free MAR. Be sure to grant the required service account permissions to enable this feature.

CircleCI

We have renamed the API key field to Personal API token in the CircleCI connector setup form and API configuration parameters.

Close

We have changed the sync strategy for the CONTACT and LEAD tables and their child tables. We no longer capture deletes for these tables.

commercetools

We now support syncing data from text columns in all locales in our commercetools connector. Previously, we supported only the de-DE and en-US locales for syncing data from text columns.

Customer.io

We now re-import the MESSAGE table once a week to capture updates and deletes.

Duoplane

We have renamed the API Secret field to API password in the connector setup form.

Email

While syncing your email attachments, we now store the date and time when the email was sent in the _modified column. Previously, this column contained the date and time when the connector uploaded the attachments to our internal S3 bucket.

NOTE: This change is not applicable to the syncs already completed before March 16, 2024.

Facebook Ads

We have upgraded our Facebook Ads connector to version 18.0 of the Facebook API. For more information, see Facebook's summary of Graph API v18.0 and Marketing API v18.0.

We have deprecated the following insight metrics:

• conversion_rate_ranking
• engagement_rate_ranking
• quality_ranking

The existing reports won't be affected, but these fields will not be available for new reports.

Firebase

We have enhanced the connector to enable you to sync subcollections with the same name as top-level collections. You can now sync both subcollections and collections that share identical names.

Google Calendar

We now re-import the EVENT table to capture updates and deletes.

Google Cloud Storage

We have added a new configuration option, Primary Key used for file process and load, to the connector setup form to support syncing files with custom primary keys. You can now select the custom primary keys during the connector configuration and use them to update or insert new rows in your destination. We process the most recently modified files to your destination. For more information, see our setup instructions.

We have added a new toggle, Non-standard character escaping, to the connector setup form to support the non-standard ways of escaping characters. If your CSV generator uses non-standard ways of escaping character, set the toggle to ON and in the Character Escaping options drop-down menu, select one of the following options:

• Custom Escape Character: To process CSV files that use custom escape character to escape quotation marks.
• Delimited Only: To process CSV files that don't use escape character to escape quotation marks and process with delimiter only.

For more information, see our setup instructions and REST API config. We are gradually rolling out this change to all existing connectors.

High-Volume Agent SQL Server

Our High-Volume Agent SQL Server connector is now generally available. Read our High-Volume Agent SQL Server connector documentation.

We now support named instances in our High-Volume Agent SQL Server connectors. We are gradually rolling out this improvement to all existing connectors.

Instagram Business

We have upgraded our Instagram Business connector to version 18.0 of the Graph API. We are gradually rolling out this improvement to all existing connectors. For more information about the changes, see Graph API's changelog.

Iterable

We have added support for EU-based Iterable accounts. For more information, see our setup instructions.

Jotform

You can now specify your custom API base URL while setting up your connector. To enable you to specify your custom base URL, we have made the following changes to the connector setup form:

• Added a new option, Custom, to the Base URL drop-down menu.
• Added a new field, Custom base URL, which allows you to specify your custom base URL.

  NOTE: The Custom base URL field appears only when you select Custom in the Base URL drop-down menu.

For more information, see our setup instructions.

We now capture deletes for the FORM table.

Kantata

You can now exclude tables from the sync. On your connector details page, go to the Schema tab and deselect the tables you want to exclude from the sync.

IMPORTANT: You can exclude any table except the USER table.

LearnUpon

We now incrementally sync the ENROLLMENT table to capture updates.

Maxio Chargify

We now incrementally sync the COMPONENT_PRICE_POINT table and its child tables to capture new records and also capture deletes for the table.

PostgreSQL

Fivetran Teleport Sync for PostgreSQL now utilizes a hybrid of XMIN updates and Teleport deletes to reduce incremental update failures:

• Inserts and updates are captured using XMIN updates.
• The deletes are captured using a simplified Teleport query.

With this change, Teleport now processes some DDL events differently:

Impact on MAR

• XMIN wraparounds result in a MAR overage. To avoid an overage, enable the pageinspect and pg_visibility XMIN extensions. Learn how in our XMIN wraparound causing excess MAR documentation.
• Changes in excluded columns contribute towards MAR.
• Changes in a row's xmin column are captured as a change and contribute to MAR. Learn more in our XMIN system column documentation.
• MAR should decrease for tables with composite primary keys, because the previous Teleport update query synced more rows than the new query.

All connectors with Teleport enabled run setup tests to verify that XMIN extensions are enabled. You can see a list of your PostgreSQL connector's setup tests in the setup guide. If there is a risk of a MAR overage, you receive a warning to enable XMIN extensions.

History Mode

We detect any change to the XMIN value of a row and replicated that change in the destination.

Rithum

We have renamed our ChannelAdvisor connector to Rithum, following ChannelAdvisor's name change. This name change applies to the Fivetran dashboard and our documentation.

Salesforce

We can now translate the formula fields that reference queue fields. Previously, translations for such fields were unsuccessful with the LOOKUP_FIELD_NOT_SYNCED failure as the reason. To enable the translation of these formula fields, you must select the GROUP table as a dependent table.

SAP ERP on HANA

We can now sync empty tables to your destination.

Shopify

We have upgraded our Shopify connector from version 2023-04 to 2024-01 of the Shopify API. As part of the upgrade, we have deprecated the delivery_category column of the ORDER_SHIPPING_LINE table. We are gradually rolling out this change to all existing connectors. For more information about the change, see Shopify API's changelog.

Snowflake

We now support syncing materialized views. You can access these views from the Schema tab of your connector details page.

To sync materialized views, run the following commands in your worksheet. Replace <YOUR_SCHEMA> and <YOUR_ROLE> with your schema name and role name.

GRANT SELECT on all MATERIALIZED VIEWS in schema <YOUR_SCHEMA> to role <YOUR_ROLE>;`
GRANT SELECT on future MATERIALIZED VIEWS in schema <YOUR_SCHEMA> to role <YOUR_ROLE>;`

Stripe

We have changed the sync strategy for the PLAN table. To capture new records and updates, we now re-import this table every time we perform a sync for the connector. Previously, we re-imported this table once a day.

Support for syncing Kafka record headers

You can now sync record headers for Kafka connectors. We have added a new column, headers, that stores record metadata as JSON key-value pairs.

This change applies to the following connectors:

• Apache Kafka
• AWS MSK
• Confluent Cloud
• Heroku Kafka

NOTE: For the existing connectors, we will now sync record headers only for new data.

Support for TRUNCATE command

Our MariaDB and MySQL connectors now support the TRUNCATE command.

TikTok Ads

We now use STATUS_ALL filtering option for basic reports which include all statuses. Previously, by default, we used the STATUS_NOT_DELETE filtering option for basic reports which includes all statuses except Deleted. All basic report aggregations will now also include the deleted entities.

WorkRamp

We now incrementally sync the EVENT table and its child tables to capture new records.

Wrike

We now incrementally sync the ATTACHMENT table and its child tables to capture new records.

Zuora

We have implemented new weekly lookback windows to update the data and improve data integrity for the following:

• tables with a 14-day lookback period:

  • CREDIT_MEMO_APPLICATION_ITEM
  • CREDIT_TAXATION_ITEM
  • DEBIT_MEMO_ITEM
  • DEBIT_TAXATION_ITEM
  • INVOICE_ITEM
  • INVOICE_ITEM_ADJUSTMENT
  • PAYMENT_APPLICATION_ITEM
  • REFUND_APPLICATION_ITEM
  • REVENUE_EVENT_ITEM
  • REVENUE_EVENT_ITEM_DEBIT_MEMO_ITEM
  • REVENUE_EVENT_ITEM_INVOICE_ITEM
  • REVENUE_EVENT_ITEM_INVOICE_ITEM_ADJUSTMENT
  • TAXATION_ITEM

• tables with a 90-day lookback period:

  • CREDIT_MEMO_ITEM
  • REVENUE_EVENT_ITEM_CREDIT_MEMO_ITEM

REST API Improvements

Added new field in Destinations API

We have added a new field, daylight_saving_time_enabled, for the following endpoints:

• Create a destination
• Retrieve destination details
• Modify a destination

If the field value is set to true, the UTC offset is shifted in accordance with the daylight saving time. You can use this option only for the US region.

Duoplane

You can now configure your Duoplane connector using the Fivetran REST API. This feature is available for Free, Standard, Enterprise, and Business Critical accounts.

Freshdesk Contact Center

You can now configure your Freshdesk Contact Center connector using the Fivetran REST API. This feature is available for Free, Standard, Enterprise, and Business Critical accounts.

Proxy Agent Management API

Our Proxy Agent Management API resource is now in beta. You can now leverage the following features:

• Create a Proxy Agent
• List all Proxy Agents
• Retrieve Proxy Agent Details
• Delete a Proxy Agent
• Returns all connections attached to the proxy agent
• Attach connection to the proxy agent
• Detach connection from the proxy agent

Log events

New Log Event

We have added a new log event, schema_migration_end. For more information, see our Logs documentation.