September 2019

Sync logs

We have added a sync logs section to the Connector Details page in the Fivetran dashboard. It displays recent connector events.

Sync History

Schema changes

Apple Search Ads

Apple has deprecated the following fields from the CAMPAIGN_HISTORY table, so we have removed them:

referenceLabel
storefront
billingContactEmail (part of the loc_invoice_details field)
billingContactName (part of the loc_invoice_details field)

GitHub

We now support Teams in our GitHub connector. We have added three new tables:

TEAM to collect information about organization teams
TEAM_MEMBERSHIP to store team membership data
REPO_TEAM to keep the relationship between team and repo, including team permission level

Kantata

We have added two new tables, FIXED_FEE_ITEMS and INVOICE_FIXED_FEE_ITEMS.

Shopify

We have added the FULFILLMENT_ORDER_LINE table to support the relationship between FULFILLMENT and ORDER_LINE tables. You will now be able to get fulfillment information for each order line, which is necessary in cases where there is more than one fulfilment per order.

Stripe

We have added a new column, customer_id, to the INVOICE_ITEM table.

Xero

We have added two new tables, INVOICE_LINKED_TRANSACTIONS and BANK_LINKED_TRANSACTIONS, to support the linked transactions endpoint.

Improvements

Amazon DynamoDB

If you have a logging service connected to your Fivetran account, the import progress of your tables in the event logs will be visible in your logs as an import_progress event. Table names will be in either the COMPLETE or IN_PROGRESS state, depending on their sync status. Tables that are not reported in the event have not started their import.

Apple Search Ads

We have upgraded the connector to Apple Search Ads API version 2, since version 1 of the API has been deprecated. For more information, see Apple Search Ads' API documentation.

With this API update, we now support negative keywords fetched by campaigns and ad groups.

BigQuery

Our BigQuery destination now supports tables partitioned by ingestion time. If you want to use tables partitioned by ingestion time, learn how in our partitioned table documentation.

Eloqua

You can now exclude custom objects from your syncs by deselecting them on the Schema tab in your Fivetran dashboard.

Freshdesk

We can now delete all of the following records from the tables below:

Records with empty group_ids from the AGENT_GROUP table
Records with empty role_ids from the AGENT_ROLE table
Records with empty cc_mails from the TICKET_CC_EMAIL table
Records with empty fwd_emails from the TICKET_FWD_EMAIL table
Records with empty reply_cc_emails from the TICKET_REPLY_CC_EMAIL table
Records with empty to_emails from the TICKET_TO_EMAIL table
Records with empty tags from the TICKET_TAG table
Records with empty to_emails fields from the CONVERSATION_TO_EMAIL table
Records with empty bcc_emails from the CONVERSATION_BCC_EMAIL table

Google Ads

We are removing the AveragePosition field from all of our predefined reports, because Google will sunset the average position metric on September 30, 2019. See Ads Developer Blog: Sunset of the Average Position Metric.

Your own reports will remain intact, but starting on September 29, 2019, we will ignore the AveragePosition field and log a warning. We recommend that you remove the AveragePosition field from any of your custom reports and use top and absolute top metrics instead. See Ads Developer Blog: Top and Absolute Top metrics.

Any old data will remain as is and the average_position column will stay in your tables. If you want to remove it completely, you'll have to drop the corresponding tables and run a historical re-sync.

Jira

We now track deletes in the WORKLOG table. This means that any deleted work logs will be accurately reflected in the destination.

We can now sync empty array type fields, such as labels. This means that any removed values will be accurately reflected in the destination. Previously, we could not sync array fields with null values, which meant that some values deleted in the source were not deleted in the destination.

Kustomer

We have added the following new fields to the CONVERSATION table:

first_message_in_createdAt
first_message_in_id
first_message_in_sentAt
first_response_createdAt
first_response_id
first_response_responseTime
first_response_sentAt
first_response_time

Microsoft Advertising

We have added the following connection tests to the Microsoft Advertising setup form:

Query All Metadata: checks if all selected accounts have access to Bulk Service.
Query All Reports: checks if all selected accounts have access to Report Service.

MongoDB

MySQL

We have begun rolling out support for the following spatial types in version 4 of our MySQL connector:

GEOMETRY
GEOMETRYCOLLECTION
LINESTRING
MULTILINESTRING
POINT
MULTIPOINT
POLYGON
MULTIPOLYGON

Oracle

We have reduced our minimum archive log retention period to 3 hours. We still recommend setting a retention time of more than 24 hours.

We've fixed a few bugs that slowed down the import process for partitioned tables. To leverage the new speed improvements, you must enable SELECT access to the DBA_EXTENTS, DBA_TABLESPACES, and DBA_SEGMENTS system views. A new warning message on the Alerts page of your Fivetran dashboard will prompt you to enable access.

PostgreSQL

Salesforce Marketing Cloud

We have added support to authenticate Salesforce Marketing Cloud connectors using OAuth 2.0. Salesforce Marketing Cloud creates new packages with OAuth 2.0 as the only authentication option. Thus, all connectors that use these new packages must use OAuth 2.0 to authenticate with Fivetran.

When you create a new connector, you must now enter your subdomain in the setup form. You must also select New Enhanced Package to indicate that your package uses OAuth 2.0.

SQL Server

REST API Improvements

We have disabled HTTP protocol support for our REST API module for both public and partner APIs. To access the Fivetran REST API, you must now use HTTPS protocol, which is much more secure.