June 2023

Dashboard Improvements

New Data Notification Settings

You can now choose to have Fivetran send you a notification when we detect new data in your source, regardless of your schema change setting. Previously, you could only subscribe to notifications if you selected the Block All setting. Learn more in our Schema Change settings documentation.

Fivetran Azure US Servers

We have added new servers in the Azure Central US (Iowa) region.

If you're on a Business Critical plan, you can now select this region in your destination setup form. Connectors configured in a destination run in their designated cloud and region, meaning that your data doesn't leave our region-specific servers during processing. For example, if you configure your connectors to use our servers in the US, your data does not leave the US during processing.

We now support Azure Private Link connections to sources and destinations in the Central US region.

For more information, see our data residency documentation.

Lite Connectors

Lite connectors are now available for the following sources:

• Brevo
• Campaign Monitor
• Cin7 Core
• Collibra
• Dixa
• Docebo
• Eventbrite
• FreshTeam
• Friendbuy
• Gladly
• Gorgias
• Mux
• PostHog
• SkuVault
• Tempo
• Workable

Transformations for dbt Core

dbt Core version 1.5.1 support

We now support version 1.5.1 of dbt Core. You can use this version to run your projects.

NOTE: For this version of dbt Core, we don't support SQL Server, Azure Synapse, and MySQL. These destinations aren't currently supported by custom dbt Core adapters.

* dbt Core is a trademark of dbt Labs, Inc. All rights therein are reserved to dbt Labs, Inc. Fivetran Transformations is not a product or service of or endorsed by dbt Labs, Inc.

New connectors

Amazon Selling Partner

Amazon Selling Partner is a leading ecommerce platform. Read our Amazon Selling Partner connector documentation.

Talkdesk

Talkdesk is a cloud-based call center solution. Read our Talkdesk connector documentation.

Schema changes

Coupa

We have made the following changes to the INVOICE_LINE_ALLOCATION table:

• Added a new column, invoice_header_id.

• Changed the primary key of the table. The new composite key is a combination of the following columns:

  • id
  • invoice_header_id
  • invoice_line_id

Be sure to re-sync the table for the latest changes or contact our support team to re-sync it for you.

We have renamed the savings-pact column of the CONTRACT table to savings-pct. Be sure to re-sync the table for the latest changes or contact our support team to re-sync it for you.

Facebook Ads

We have removed the has_advertiser_opted_in_odax column from the ACCOUNT_HISTORY table.

We have removed the ad_strategy_id and ad_strategy_group_id columns from the CAMPAIGN_HISTORY table.

We have added a new value, mixed, to the action_report_time parameter.

Fivetran Platform Connector

We have removed the following obsolete tables that we previously used for our Role-Based Access Control:

• ACCOUNT_MEMBERSHIPS
• MEMBERSHIPS
• MEMBERSHIPS_ROLES
• MEMBERSHIPS_ROLES_SOURCES

Google Ads

We have added a new field, metrics.cost_per_conversion, to the AD_STATS prebuilt report.

We have added a new field, metrics.cost_per_conversion, to the AD_GROUP_STATS prebuilt report.

To fetch historical data for these fields, trigger a table re-sync for these tables.

LinkedIn Company Pages

We have added a new table, TOTAL_FOLLOWER_STATISTIC. The table contains information about the total followers count by organization.

Mailchimp

We have changed the composite primary key of the UNSUBSCRIBE table and performed a table re-sync to ensure data integrity. The new composite key is a combination of the list_id and member_id columns.

Marketo

We have added the following new columns to the PROGRAM table:

• folder_id
• folder_name
• folder_type
• folder_value
• head_start

We now sync custom properties to the PROGRAM table. We sync the properties with the prefix custom_.

QuickBooks

We have changed the data type from FLOAT to BIGDECIMAL for all tables. This change applies to both new and existing connectors. However, to apply this change to existing connectors, you must delete the tables and perform a full historical re-sync.

SAP Concur

We no longer upsert data into the following columns of the EXPENSE_ENTRY table because SAP Concur does not provide the data for these columns in its Get Expense Entry API response:

• billing_amount
• comment_count
• entry_image_id
• is_credit_card_charge
• receipt_required
• report_entry_vendor_name
• transaction_currency_name

Split

The environment_id column now has a foreign key relationship to the ENVIRONMENT table in the following tables:

• SPLIT_DEFINITION_RULE
• SPLIT_DEFINITION_RULE_CONDITION
• SPLIT_DEFINITION_TREATMENT

Stripe

We have added the following fields to the CHECKOUT_SESSION_LINE_ITEM_DISCOUNT table:

• checkout_session
• coupon_id
• customer_id
• end
• invoice_id
• invoice_item_id
• promotion_code
• start
• subscription_id

We have added a new column, reconciliation_status, to the PAYOUT table.

Be sure to re-sync these tables for the latest changes or contact our support team to re-sync it for you.

Talkdesk

We have changed the composite primary key of the STUDIO_FLOW_EXECUTION table to ensure data integrity. The new composite key is a combination of the following columns:

• flow_id
• interaction_id
• step_name

Twitter Ads

We have added the following new tables:

• ACCOUNT_METROS_REPORT
• CAMPAIGN_METROS_REPORT
• FUNDING_INSTRUMENT_METROS_REPORT
• LINE_ITEM_METROS_REPORT
• PROMOTED_TWEET_METROS_REPORT
• PROMOTED_ACCOUNT_METROS_REPORT

We have updated the following tables:

NOTE: This change is enabled by default for connectors created after June 12, 2023. We are gradually rolling this out to all existing connectors and will notify all users beforehand.

Workday HCM

We have changed the primary key of the PAYROLL table. The primary key is now a Fivetran-generated column and is the hash of the employee_id and period_id columns. Previously, we populated the primary key with the Payroll_ID_Reference.WID value that we obtained from the WorkdayHCM API response. Be sure to update your queries.

Zendesk Chat

We have added two new tables, AGENT_ENGAGEMENT_HISTORY and AGENT_STATUS_HISTORY, to store agent-related events of your Zendesk Chat account.

Improvements

Amazon Ads

We have upgraded our Amazon Ads connector from version 2 to 3 of the Sponsored Products API to support Amazon's upcoming deprecations. As part of the upgrade, we have made the following schema changes:

• Added the following new tables:
  • AD_GROUP_SERVING_STATUS_DETAIL
  • CAMPAIGN_NEGATIVE_KEYWORD_SERVING_STATUS_DETAIL
  • CAMPAIGN_PLACEMENT_BIDDING
  • CAMPAIGN_SERVING_STATUS_DETAIL
  • CAMPAIGN_TAG
  • KEYWORD_SERVING_STATUS_DETAIL
  • NEGATIVE_KEYWORD_SERVING_STATUS_DETAIL
  • NEGATIVE_TARGETING_SERVING_STATUS_DETAIL
  • TARGETING_SERVING_STATUS_DETAIL
• Changed the data type of the id column from INT to STRING in the following tables:
  • AD_GROUP_HISTORY
  • CAMPAIGN_HISTORY
  • CAMPAIGN_NEGATIVE_KEYWORD_HISTORY
  • KEYWORD_HISTORY
  • NEGATIVE_KEYWORD_HISTORY
  • PRODUCT_AD_HISTORY
  • TARGETING_CLAUSE_HISTORY
• Deprecated the BIDDING_ADJUSTMENT table
• Added a new column, native_language_locale, to the KEYWORD_HISTORY table
• Added two new columns, native_language_keyword and native_language_locale, to the NEGATIVE_KEYWORD_HISTORY table
• Made the following changes to the CAMPAIGN_HISTORY table:
  • Added three new columns, budget, budget_type, and effective_budget
  • Removed the campaign_type, daily_budget, placement, and premium_bid_adjustment columns
• Made the following changes to the PRODUCT_AD_SERVING_STATUS_DETAIL table:
  • Added a new column, help_url
  • Removed two columns, _fivetran_id and level

We are rolling out these changes to all existing connectors before June 30, 2023. For more information about the changes, see Amazon's migration guide.

Amazon DynamoDB

We have renamed our Sync mode feature to Pack mode.

Amazon S3

We now support multi-character delimiters for CSV files. You can specify a multi-character delimiter in the Delimiter field of the connector setup form. For more information, see our setup instructions.

AWS PrivateLink Support

We now support AWS PrivateLink for the following destinations:

• Amazon Aurora PostgreSQL
• Amazon RDS PostgreSQL
• Aurora MySQL
• MariaDB MySQL
• Amazon RDS for MariaDB MySQL
• MySQL
• Amazon RDS for MySQL
• PostgreSQL
• SQL Server
• Amazon RDS for SQL Server

Azure Blob Storage

We now support multi-character delimiters for CSV files. You can specify a multi-character delimiter in the Delimiter field of the connector setup form. For more information, see our setup instructions.

We now support syncing PGP (Pretty Good Privacy) encrypted files. We have added a new advanced configuration, PGP Encryption Options, to the connector setup form. For more information, see our setup instructions and the REST API configuration parameters.

Azure Event Hubs

Our Azure Event Hubs connector is now generally available. Read our Azure Event Hubs connector documentation.

We have suspended our support for Avro and Protobuf message types as we have not seen the use cases for these formats. This change applies to both the connector setup form and API configuration parameters. Contact our support team if you want us to support these message formats.

Azure Private Link Support

We now support Azure Private Link for the following destinations:

• Azure Database for PostgreSQL
• Azure SQL Database
• Azure SQL Managed Instance

BambooHR

We have fixed a bug that caused data discrepancies in the EMPLOYEE table columns. We recommend that you re-sync your connector if you observe any data integrity issues with this table.

BigCommerce

You can now configure your BigCommerce connector using the Fivetran REST API. This feature is available only for Free, Standard, Enterprise, and Business Critical accounts.

Box

Our Box connector can now sync packed JSON files in Merge Mode. We have added a new drop-down menu, JSON Delivery Mode, to the connector setup form. The drop-down menu provides you the option to choose how Fivetran should handle your JSON data. For more information, see our setup instructions.

Calendly

The Calendly connector now supports table-level re-syncs.

Close

The Close connector now supports table-level re-syncs.

Criteo

We have upgraded our Criteo connector from version 2022.07 to 2023.01 of the Criteo API. For more information about the changes, see Criteo API's changelog.

Dropbox

Our Dropbox connector can now sync packed JSON files in Merge Mode. We have added a new drop-down menu, JSON Delivery Mode, to the connector setup form. The drop-down menu provides you the option to choose how Fivetran should handle your JSON data. For more information, see our setup instructions.

We have removed the Root Folder in Team Spaces toggle from the Merge Mode sync strategy, in the connector setup form. We now infer the values of this field automatically.

Eloqua

The Eloqua connector now supports data blocking for all Eloqua tables. You can now choose which tables to sync and which to block from syncing to your destination. On your Connector Details page, go to the Schema tab and select or deselect the tables. If you want to sync a child table, you must include the parent table in the sync.

Email

We now support multi-character delimiters for CSV files. You can specify a multi-character delimiter in the Delimiter field of the connector setup form. For more information, see our setup instructions.

Facebook Ads

We have upgraded our Facebook Ads connector to version 17.0 of the Facebook API. For more information, see Facebook's release of Graph API v17.0 and Marketing API v17.0.

Facebook Pages

We have upgraded our Facebook Pages connector to version 17.0 of the Graph API. We are gradually rolling out this change to all existing connectors. For more information about the changes, see Graph API's changelog.

FTP

We now support multi-character delimiters for CSV files. You can specify a multi-character delimiter in the Delimiter field of the connector setup form. For more information, see our setup instructions.

Gladly

The Gladly connector now supports table-level re-syncs.

Google Ads

We are migrating connector metadata tables to Fivetran history mode over the next few weeks. We will keep updating the updated_at column for metadata tables during the migration period and some time after that for backward compatibility.

Google Cloud Storage

We now support multi-character delimiters for CSV files. You can specify a multi-character delimiter in the Delimiter field of the connector setup form. For more information, see our setup instructions.

Google Drive

Our Google Drive connector can now sync packed JSON files in Merge Mode. We have added a new drop-down menu, JSON Delivery Mode, to the connector setup form. The drop-down menu provides you the option to choose how Fivetran should handle your JSON data. For more information, see our setup instructions.

Impact

The Impact connector now supports table-level re-syncs.

Instagram Business

We have upgraded our Instagram Business connector to version 17.0 of the Graph API. We are gradually rolling out this change to all existing connectors. For more information about the changes, see Graph API's changelog.

LinkedIn Ad Analytics

We have fixed a bug where the values of the sub_content_regular_call_to_action_landing_page column in the IN_MAIL_CONTENT_HISTORY table always were NULL. We have renamed the column to sub_content_regular_call_to_action_landing_page_url, and it should now be populated by the actual values moving forward.

Mailchimp

We have changed our sync strategy for the UNSUBSCRIBE table. We now perform a daily incremental sync of the table.

Monday.com

The Monday.com connector now supports table-level re-syncs.

NetSuite SuiteAnalytics

We have changed the sync strategy of the TRANSACTION_TRACKING_NUMBERS table. We now re-import the TRANSACTION_TRACKING_NUMBERS table based on the table import duration:

• If the duration is less than or equal to 20 seconds, we re-import the table in every sync.
• If the duration is more than 20 seconds, we re-import the table twice daily.

Okta

The Okta connector now supports table-level re-syncs.

Open Data DC

We now incrementally sync the CRASH_IN_DC table and its child tables to capture new records. We also re-import these tables once a week to capture updates and deletes.

Outreach

We now capture deletes for all child tables in Outreach. To capture deletes of historical data in these tables, re-sync their parent tables. We are gradually rolling out this change to all existing connectors.

PostgreSQL

We now support syncing the LTREE data type. See all supported data types in our PostgreSQL documentation.

Recharge

We now capture soft deletes for the PAYMENT_METHOD table. We create a new _fivetran_deleted column in the destination table, and mark the deleted rows as TRUE.

We have released pre-built, dbt Core-compatible data models for Recharge. Find the models in Fivetran's dbt hub or data models documentation. Learn more about our dbt Core integration in our Transformations for dbt Core documentation*.

* dbt Core is a trademark of dbt Labs, Inc. All rights therein are reserved to dbt Labs, Inc. Fivetran Transformations is not a product or service of or endorsed by dbt Labs, Inc.

Reddit Ads

We have extended the reporting data retrieval period from the Reddit Ads API to 12 months to align with the recent restrictions imposed by Reddit.

Salesforce Commerce Cloud

The Salesforce Commerce Cloud connector now supports priority-first sync for the COUPON_REDEMPTION table. Priority-first syncs fetch your most recent data first so that it's quickly ready for you to use. Salesforce Commerce Cloud's priority-first syncs fetch the most recent 15 days' worth of data from the supported tables. We are gradually rolling out this change to all existing connectors. To learn more, see our priority-first sync documentation.

Samsara

The Samsara connector now supports table-level re-syncs.

SFTP

We now support multi-character delimiters for CSV files. You can specify a multi-character delimiter in the Delimiter field of the connector setup form. For more information, see our setup instructions.

We now support custom line separators for CSV files. You can specify a custom line separator in the Line Separator field of the connector setup form. For more information, see our setup instructions.

SharePoint

Our SharePoint connector can now sync packed JSON files in Merge Mode. We have added a new drop-down menu, JSON Delivery Mode, to the connector setup form. The drop-down menu provides you the option to choose how Fivetran should handle your JSON data. For more information, see our setup instructions.

Shopify

We have upgraded our Shopify connector from version 2022-07 to 2023-04 of the Shopify API. As part of the upgrade, we have deprecated the following columns:

• fulfillment_service column of the ABANDONED_CHECKOUT_LINE table
• fulfillment_service column of the FULFILLMENT_ORDER_LINE table
• fulfillment_service column of the ORDER_LINE table
• can_delete and lifetime_duration columns of the CUSTOMER table
• total_price_usd and processing_method columns of the ORDER table
• total_price_usd column of the DRAFT_ORDER table
• destination_location_*, fulfillment_service, and origin_location_* columns of the ORDER_LINE table

We are gradually rolling out these changes to all existing connectors.

TikTok Ads

You can now opt to configure the attribution window to capture conversions. In the connector setup form, set the Show advanced options toggle to ON, and set the attribution window using the Rollback sync (optional) drop-down. By default, the window is 7 days. For more information, see our setup guide and REST API Config.

We have added a new option, Add custom report, to the setup form. This option enables you to configure and sync your custom reports using our TikTok Ads connector. For more information, see our setup guide. We are gradually rolling out this feature to all existing connectors.

WooCommerce

The WooCommerce connector now supports table-level re-syncs.

YouTube Analytics

We have added support for multi-reporting for new connectors created after June 25, 2023. Once you create the connector, it will sync all available reports by default.

We now sync metadata for the following:

• channels
• playlists
• videos

REST API Improvements

AWS Lambda

Now, when you configure the AWS Lambda connector using our REST API, you can:

• Connect using AWS PrivateLink if your Lambda function and destination are in the same AWS Region. This feature is available only on the Business Critical plan.
• Select your sync method. You can opt to sync directly or through an S3 bucket.

Calendly

You can now configure your Calendly connector using the Fivetran REST API. This feature is available only for Free, Standard, Enterprise, and Business Critical accounts.

Qualtrics

You can now configure the API usage limit of the Qualtrics connector using our REST API. For more information, see our Qualtrics API Configuration documentation.

Webhook event improvement

We now support the edit_connector webhook event for manual or automatic connector status changes. Namely, this webhook event is raised for setup-state and schema-status property changes, which are reflected in editType.

The following edit_connector webhook events are shown below as being relevant for the How to Set Up a Schema Before the Initial Sync use case.

{
    event: 'edit_connector',
    // ...
    editType: 'UPDATE_SETUP_STATE',
    properties: { setupState: 'connected' },
    oldProperties: { setupState: 'incomplete' }
}
{
  event: 'edit_connector',
  // ...
  editType: 'UPDATE_SETUP_STATE',
  properties: { setupState: 'broken' },
  oldProperties: { setupState: 'connected' }
}
{
    event: 'edit_connector',
    // ...
    editType: 'UPDATE_SCHEMA_STATUS',
    properties: { schemaStatus: 'blocked_on_capture' },
    oldProperties: { schemaStatus: 'ready' }
}
{
    event: 'edit_connector',
    // ...
    editType: 'UPDATE_SCHEMA_STATUS',
    properties: { schemaStatus: 'blocked_on_customer' },
    oldProperties: { schemaStatus: 'blocked_on_capture' }
}
{
    event: 'edit_connector',
    // ...
    editType: 'UPDATE_SCHEMA_STATUS',
    properties: { schemaStatus: 'ready' },
    oldProperties: { schemaStatus: 'blocked_on_customer' }
}