January 2020

Account Security Improvements

We have changed the default access granted to new users added to Fivetran accounts which have SAML enabled. When you add a new user to your Fivetran account, the new user is automatically granted Read Only access for the account and no destination access. Fivetran then notifies the Account Administrator(s) about the new user via email. The Account Administrator can adjust the new user's permissions.

Previously, new users added to accounts with SAML enabled received access to every destination in their account, and an Account Administrator had to remove new user's superfluous access manually.

New connectors

AWS CloudTrail

AWS CloudTrail is a service that enables governance, compliance, operational auditing, and risk auditing of your AWS account. With AWS CloudTrail, you can log, continuously monitor, and retain account activity related to actions across your AWS infrastructure.

Read our AWS CloudTrail connector documentation.

Schema changes

Iterable

We have added a new table, IN_APP_TEMPLATE_HISTORY. This table supports In-App templates.

Kustomer

We have added a lookup table, MESSAGE_SHORTCUT, with the message_id and shortcut_id fields.

Lightspeed Retail

We have added five new history tables to replace the following tables:

ATTRIBUTE_VALUE_HISTORY will replace ATTRIBUTE_VALUE.
ITEM_MATRIX_PRICE_HISTORY will replace ITEM_MATRIX_PRICE.
ITEM_MATRIX_TAG_HISTORY will replace ITEM_MATRIX_TAG.
ITEM_PRICE_HISTORY will replace ITEM_PRICE.
ITEM_TAG_HISTORY will replace ITEM_TAG.

We will update all ten tables for three months, then deprecate the replaced tables. Be sure to update your queries with the new history table names before we remove the old tables.

Optimizely

We have added a new table, EXPERIMENT_RESULT_HISTORY.

To improve data security, we will deprecate the aws_access_key and aws_secret_key columns from the LIST_ATTRIBUTE table, on March 1, 2020. Be sure to update your queries.

Outbrain

We have made a number of schema changes to support Outbrain's API documentation.

We have added the following columns to all _REPORT tables:

sum_value
total_sum_value
view_sum_value
total_average_value
view_average_value
view_conversion_rate
total_conversions
views_conversions
total_cpa

We have added the approval_status and approval_status_is_editable columns to the PROMOTED_LINK_HISTORY table.

We will deprecate the status column in the PROMOTED_LINK_HISTORY table on May 1, 2020. This data will then appear in the approval_status column.

We will deprecate the total_value column in all _REPORT tables on May 1, 2020. This data will then appear in the sum_value column.

To learn more, see our Outbrain schema information.

Pinterest Ads

We have deprecated the optimization_goal field in the CAMPAIGN_HISTORY table. This change is required to migrate our Pinterest connector to the Pinterest Ads API V3. For more details, see Pinterest Ads' API documentation.

Recharge

We have added the following columns to the CHARGE table:

email
has_uncommited_changes
processor_name
total_refunds
transaction_id

Recurly

We have added three new history tables to replace the following tables:

COUPON_PLAN_HISTORY will replace COUPON_PLAN.
INVOICE_SUBSCRIPTION_HISTORY will replace INVOICE_SUBSCRIPTION.
SUBSCRIPTION_COUPON_REDEMPTION_HISTORY will replace SUBSCRIPTION_COUPON_REDEMPTION.

We will update all six tables with for three months, then deprecate the replaced tables. Be sure to update your queries with the new history table names before we remove the old tables.

Salesforce Marketing Cloud

We now store timestamp data using the TIMESTAMP WITHOUT TIME ZONE type. Previously we used TIMESTAMP WITH TIME ZONE and when Fivetran received timestamp data that did not have timezone information from Salesforce Marketing Cloud, we incorrectly stored the timestamp data as UTC. Using TIMESTAMP WITHOUT TIME ZONE fixes this issue.

The following table lists the columns whose type we have converted from TIMESTAMP WITH TIME ZONE to TIMESTAMP WITHOUT TIME ZONE:

Table Name	Modified Columns
EMAIL	created_date, modified_date
EVENT	created_date, event_date, modified_date
JOURNEY	created_date, modified_date
LINK	last_clicked
LIST	created_date, modified_date
LIST_SUBSCRIBER	created_date, modified_date
SEND	created_date, modified_date, send_date
SUBSCRIBER	created_date, unsubscribed_date
TRIGGERED_SEND	created_date, modified_date

Newly created connectors will use TIMESTAMP WITHOUT TIME ZONE in these columns by default. To update existing connectors to use TIMESTAMP WITHOUT TIME ZONE, drop your existing tables and trigger a re-sync. If you want to continue storing timestamps using TIMESTAMP WITH TIME ZONE, take no action.

Stripe

We have added a new column, unique_id, to the INVOICE_LINE_ITEM table. This fix resolves the pagination break issue for invoice line items, which was introduced by the Stripe API update.

Twitter Ads

We have deprecated the include_sentiment field from the LINE_ITEM_HISTORY table because the Twitter Ads API no longer supports it. To learn more, see Twitter's API documentation.

Xero

We have added a new column, option, to the following tables:

BANK_TRANSACTION_LINE_ITEM_HAS_TRACKING_CATEGORY
MANUAL_JOURNAL_LINE_ITEM_HAS_TRACKING_CATEGORY
INVOICE_LINE_ITEM_HAS_TRACKING_CATEGORY
OVERPAYMENT_LINE_ITEM_HAS_TRACKING_CATEGORY
PREPAYMENT_LINE_ITEM_HAS_TRACKING_CATEGORY
REPEATING_INVOICE_LINE_ITEM_HAS_TRACKING_CATEGORY
CREDIT_NOTE_LINE_ITEM_HAS_TRACKING_CATEGORY
JOURNAL_LINE_ITEM_HAS_TRACKING_CATEGORY
RECEIPT_LINE_ITEM_HAS_TRACKING_CATEGORY

We have added a new column, tracking_category_option_id, to the RECEIPT_LINE_ITEM_HAS_TRACKING_CATEGORY and CREDIT_NOTE_LINE_ITEM_HAS_TRACKING_CATEGORY tables.

We have added the following tables:

PURCHASE_ORDER
PURCHASE_ORDER_LINE_ITEM
PURCHASE_ORDER_LINE_ITEM_HAS_TRACKING_CATEGORY

We have added a new primary key column, index, to the ALLOCATION table.

Improvements

Adobe Analytics Data Feed

You can now configure your Adobe Analytics Data Feed connector through the Fivetran REST API.

Amazon DynamoDB

We have introduced packed mode, a new sync mode for Amazon DynamoDB tables with more than 1000 first-level fields. Previously, we only supported unpacked mode and used it regardless of the number of columns in a table. We have chosen 1000 columns as the reasonable upper limit for unpacked mode. This limit is compatible with all of our supported data warehouses, which vary in the upper limit of columns they accept. For more details, see our Amazon DynamoDB documentation.

Apple App Store

When setting up your Apple App Store connector, you can now choose which apps, sales accounts, and finance accounts you'd like to sync. Previously, Fivetran synced all apps, sales accounts, and finance accounts by default.

We have upgraded the following subscription reports to version 1_2:

SALES_SUBSCRIPTION_SUMMARY
SALES_SUBSCRIPTION_EVENT_SUMMARY
SALES_SUBSCRIBER_DETAILED

This version upgrade brings some changes in the fields of the above mentioned tables. Refer to the latest schema information: https://fivetran.com/docs/connectors/applications/itunes-connect#schemainformation

BigQuery

We have added support for BigQuery projects that are access-limited by a GCP VPC service perimeter. A customer-provided Google Cloud Storage bucket is required to ingest data into such a BigQuery project. For step-by-step instructions about how to setup a bucket with your BigQuery destination, follow our BigQuery destination setup instructions.

We are changing the way we access BigQuery destinations. The new design uses a unique service account for each destination instead of a fixed service account. The destinations that still use the fixed service account must migrate to a unique service account before March 1, 2020. These destinations will have a migration task in the Alerts section of the dashboard explaining how to perform the migration.

Freshdesk

We have added the functionality to identify and sync the tickets that are flagged as spam. We will also capture the change in state (non-spam to spam) of a previously synced ticket.

Jira

When the value of a field was null, our Jira connector didn't always update array values. That meant that, for example, if the value was removed from a field in the source, sometimes the destination still showed the previous value. Now when a field value is null, our Jira connector will write a default empty value according to the type of the field.

Our Jira connector can now sync issues from January 1, 1970 forward. Previously, our connector only synced issues from January 1, 2002 forward. If you have issues from 1970-2002, perform a re-sync to ensure that we capture all issues.

Optimizely

You can now exclude the VISITOR_ACTION table from your syncs by deselecting it on the Schema tab in your Fivetran dashboard.

Oracle

We are rolling out support for TRUNCATE statements. When we encounter a TRUNCATE statement for one of your selected tables, we set the _fivetran_deleted column value to TRUE for every row in that table that predates the TRUNCATE statement.

SQL Server

We now support Change Data Capture as an incremental update mechanism. To learn more, see our SQL Server setup guides.

Square

We can now delete records with no OrdersFulfillments from the ORDER_FULFILLMENT table.

Xero

We can now delete all of the following records from the tables below:

Bank transaction line items with no tracking from the BANK_TRANSACTION_LINE_ITEM_HAS_TRACKING_CATEGORY table
Receipts with no tracking from the RECEIPT_LINE_ITEM_HAS_TRACKING_CATEGORY table
Invoices with no line items from the INVOICE_LINE_ITEM table
Invoice line items with no tracking from the INVOICE_LINE_ITEM_HAS_TRACKING_CATEGORY table
Manual journal lines with no tracking from the MANUAL_JOURNAL_LINE_HAS_TRACKING_CATEGORY table
Overpayment line items with no tracking from the OVERPAYMENT_LINE_ITEM_HAS_TRACKING_CATEGORY table
Overpayments with no overpayment line items from the OVERPAYMENT_LINE_ITEM table
Pre-payments with no pre-payment line items from the PREPAYMENT_LINE_ITEM table
Repeating invoices with no repeating invoice line items from the REPEATING_INVOICE_LINE_ITEM table
Repeating invoice line items with no tracking from the REPEATING_INVOICE_LINE_ITEM_HAS_TRACKING_CATEGORY table
Credit note line items with no tracking from the CREDIT_NOTE_LINE_ITEM_HAS_TRACKING_CATEGORY table
Credit notes with no credit note line items from the CREDIT_NOTE_LINE_ITEM table
Contact addresses with no addresses from the CONTACT_ADDRESS table
Purchase orders with no purchase line items from the PURCHASE_ORDER_LINE_ITEM table
Purchase orders with no tracking from the PURCHASE_ORDER_LINE_ITEM_HAS_TRACKING_CATEGORY table
Journals with no tracking or tracking categories from the JOURNAL_LINE_HAS_TRACKING_CATEGORY table

Previously, we were not able to remove these records because of the way we set up the primary key.

Zendesk Sunshine

We now support custom relationship objects. For more information, see our Zendesk Sunshine documentation.

Zendesk Support

We have renamed our Zendesk connector to Zendesk Support. This name change applies to the Fivetran dashboard and our documentation.

REST API Improvements

We have expanded the current public API Modify a Connector endpoint. Now you can enable or disable automatic connector sync by changing the 'schedule_type' parameter. Read our REST API docs to learn how to do that.

Square

You can now configure your Square connector through the Fivetran REST API. This feature is in BETA and available only for Standard and Enterprise accounts.