October 2018link
Destination primary key changeslink
For the following destinations, Fivetran now drops the primary_key
field from existing tables in your destination if there is a primary_key
coming in from your data source:
- MySQL
- PostgreSQL
- SQL Server
- Redshift
Faster BigQuery loadinglink
We’ve made two major updates to the way we load data into BigQuery:
We are now using the interactive priority method for our queries instead of the batch priority method. Despite being designed for Fivetran’s batch update use-case, the BigQuery team has informed us that the batch priority method simply queued up our jobs without any actual improvement in the write performance.
We have also re-enabled BigQuery's SQL tool to use the ‘MERGE’ function. This reduces the data manipulation language (DML) statements per load from two to one.
As a result of these updates, the data-writing process is more than five times faster than it was previously.
Logslink
Bug fixlink
We have fixed a bug that caused incomplete failure records in our user-facing logs. Previously, when connection tests failed because of a failed sync, our logs only showed the connection test failure. Now both failures are accurately recorded.
Format changeslink
Our user-facing logs now include a UUID-format incident identifier, exception_id
, that gives our support and developer teams access to more detailed information about connector issues.
Log eventslink
Fivetran now supports flushing dashboard activity event data to supported log services.
Each log service has a unique path for this data:
Log Service | Path Pattern |
---|---|
AWS CloudWatch | {ACCOUNT_NAME}/{GROUP_NAME}/dashboard/all |
Azure Monitor | {ACCOUNT_NAME}_{GROUP_NAME}_dashboard_all_CL |
Google Cloud Logging | {ACCOUNT_NAME}-{GROUP_NAME}-dashboard-all |
New connectorslink
Adobe Analyticslink
Adobe Analytics lets you analyze data about customer engagement with your website or application.
Read our Adobe Analytics connector documentation.
Iterablelink
Iterable is a growth marketing and customer engagement platform. It enables brands to create, execute and optimize campaigns across email, push, SMS, in-app and more with unparalleled data flexibility.
Read our Iterable connector documentation.
Schema changeslink
Greenhouselink
The timestamp
column now supports two new datetime formats:
- UTC (For example, '2018-06-27 23:51:40 UTC')
- Java TimeZone (For example, '2018-07-03T22:49:22.689Z')
We have renamed the id
column to tag_id
in the CANDIDATE_TAG
table.
We have changed the primary key columns for the following tables:
candidate_id
andindex
are the new primary keys for the following tables:ADDRESS
ATTACHMENT
EMAIL_ADDRESS
PHONE_NUMBER
SOCIAL_MEDIA_ADDRESS
WEBSITE_ADDRESS
scorecard_id
andindex
are the new primary keys for the following tables:SCORE_CARD_ATTRIBUTE
SCORE_CARD_QNA
application_id
andindex
are the new primary keys for theAPPLICATION_QNA
table.job_post_id
andindex
are the new primary keys for theJOB_POST_QUESTION
table.
Snapchat Adslink
We have added a new column, operation
, to the AD_SQUAD_SEGMENT_HISTORY
table.
Xerolink
We have added two new columns, edition
and class
, to the ORGANIZATION
table.
Improvementslink
Google Analyticslink
Fivetran has increased the stability of the Google Analytics connector.
We have upgraded the connector to the Google Analytics Reporting API version 4.
Our Google Analytics reports now support the segments
and filter_expression
fields.
LinkedIn Ad Analyticslink
You can now sync all campaigns for a specific ad account without having to select them manually in our setup form.
MongoDBlink
We have optimized the way we do incremental updates from the oplog of nested updates. Previously, when you updated deeply nested fields, we had to fetch the entire document using collection.find({_id:?})
. This process can be extremely slow and delay syncs for hours, especially when deeply nested fields are frequently updated. We can now fetch these documents in batches, making our MongoDB syncs ten times faster.
NetSuite SuiteAnalyticslink
We have released service version 3 of the NetSuite SuiteAnalytics connector, which includes the following new features:
- Tables that can't be incrementally updated can now be re-imported. The frequency with which we update these re-import tables is based on how fast we can ingest them from NetSuite. For more information, see our NetSuite SuiteAnalytics guide.
- The
ITEM_LOCATION_MAP
re-import table will update every sync. To see the duration of each table import, you can refer to the table warning field on the Schema tab of your Fivetran dashboard's Connector page. - We can now upload records for tables where the primary key column contains
null
values. For more information, see our NetSuite SuiteAnalytics guide.
Pinterest Adslink
We no longer process ITEM
reports for advertisers who are not running shopping campaigns, because Pinterest only allows advertisers who are running shopping campaigns to access these reports.
PostgreSQLlink
We have clarified the wording of the rescheduled sync notification message in your Sync History.
Segmentlink
The Segment connector now handles delete
event types. We create a new table, DELETE
, in your destination to track these events.