July 2020link
New connectorslink
Google Analytics (MCF)link
Google Analytics MCF shows user interactions with various traffic sources over multiple sessions prior to converting.
Read our Google Analytics (MCF) connector documentation.
YouTube Analyticslink
YouTube Analytics contains a comprehensive set of YouTube Analytics data for a channel or content owner.
Read our YouTube Analytics connector documentation.
Schema changeslink
Braintreelink
We have added a new column, trial_duration
, to replace the trail_duration
column in the SUBSCRIPTION
table. We will deprecate the replaced column after November 1, 2020. Be sure to update your queries with the new column name before we deprecate the old column.
Eloqualink
We now store custom timestamp fields in the TIMESTAMP WITH TIME ZONE format (for example, 2020-06-28T16:02:25.987Z
).
Facebook Ad Insightslink
We have added a new table, IMAGE_CROPS
. This table stores the image_crops
values that are part of the image_asset
breakdown response.
Note: Fivetran only syncs the
IMAGE_CROPS
table if you have selected theimage_asset
breakdown.
Fivetran Platform Connectorlink
We have removed the service_version
column from the CONNECTOR
table.
Frontlink
We have deprecated the custom_field_name
and custom_field_job_title
columns from the CONTACT
table. We now sync the custom_fields
from the Contacts API endpoint as custom_{field_name}
columns.
Google Analytics (MCF)link
We have added a new column, conversion_type
, to the CONVERSION_PATH_REPORT
table.
Google Analytics 360link
We now sync the INTRADAY
table as well as the FINAL
table. Since the INTRADAY
table contains data from the current day, you no longer need to wait a day to see your latest data. We sync the complete INTRADAY
table whenever it is updated because we can't de-duplicate the newly appended data. However, we do not sync the INTRADAY
table if it contains more than four gigabytes (4GB) of data to avoid long syncs that could distrupt your sync schedule. For more information, see Google Analytics' Help documentation.
Greenhouselink
If you have a Greenhouse Expert plan, we add the following tables when you grant Fivetran access to the demographics endpoint:
DEMOGRAPHIC_QUESTION_SET
DEMOGRAPHIC_QUESTIONS
DEMOGRAPHIC_ANSWER_OPTION
We have also added custom fields to the JOB_OPENING
table.
Leverlink
We have made some schema changes to support the new Opportunity endpoint. Once you connect Fivetran to the Opportunity endpoint, we will add new tables to replace the following tables:
OPPORTUNITY
will replaceCANDIDATE
CONTACT_EMAIL
will replaceCANDIDATE_EMAIL
CONTACT_LINK
will replaceCANDIDATE_LINK
CONTACT_PHONE
will replaceCANDIDATE_PHONE
OPPORTUNITY_POSITION
will replaceCANDIDATE_POSITION
OPPORTUNITY_SCHOOL
will replaceCANDIDATE_SCHOOL
OPPORTUNITY_SOURCE
will replaceCANDIDATE_SOURCE
OPPORTUNITY_STAGE_HISTORY
will replaceCANDIDATE_STAGE_HISTORY
OPPORTUNITY_TAG
will replaceCANDIDATE_TAG
LinkedIn Ad Analyticslink
We have added the following fields to the AD_ANALYTICS_BY_CAMPAIGN
and AD_ ANALYTICS_BY_CREATIVE
tables:
comment_likes
approximate_unique_impressions
viral_external_website_conversions
viral_external_website_post_view_conversions
viral_external_website_post_click_conversions
Microsoft Advertisinglink
We have added three new tables:
AD_EXTENSION_DETAIL_HOURLY_REPORT
AD_EXTENSION_DETAIL_DAILY_REPORT
SHARE_OF_VOICE_DAILY_REPORT
We have added a new column, title_part_3
, to the AD_HISTORY
table.
Pendolink
We now sync parameter data in the PAGE_EVENT
, FEATURE_EVENT
, and EVENT
tables. We sync each parameter to the destination in its own column and add the prefix param_
to the column name. For example, the decisionData
parameter is synced to the destination as the param_decisionData
column.
QuickBookslink
We have added two new columns, vendor_credit_id
and bill_id
, to the DEPOSIT_LINE
table.
Rechargelink
We have added the following columns to the CHARGE
table:
last_charge_attempt_date
retry_date
number_times_tried
error
error_type
shopify_variant_id_not_found
We have replaced the ANALYTICS_DATA
table with the UTM_TAG
table. We will deprecate the replaced table after November 1, 2020. Be sure to update your queries with the new table name before we remove the old table.
We have removed the _fivetran_deleted
column from the SUBSCRIPTION
table.
Shopifylink
We have deprecated the CHECKOUT
and GIFT_CARD
tables. We have also deprecated the following child tables of the CHECKOUT
table:
APPLIED_DISCOUNT
CHECKOUT_DISCOUNT_CODE
CHECKOUT_LINE
CHECKOUT_SHIPPING_LINE
CHECKOUT_TAX_LINE
CHECKOUT_URL_TAG
We have deprecated the featured
field from the COLLECT
table because the Shopify API no longer supports it. See Shopify's documentation for more information.
Snowplowlink
We have added two new fields, refr_domain_user_id
and refr_dvce_tstamp
, to the _EVENT
table to facilitate Cross-Domain Tracking. These two fields store the _sp
query parameter values.
Stripelink
We have added a new column, is_deleted
, to the PERSON
table.
We have added a new table, SETUP_INTENT
. Read Stripe's documentation for more information.
Improvementslink
Apple Search Adslink
The Apple Search Ads connector now supports priority-first sync. Priority-first syncs fetch your most recent data first so that it's quickly ready for you to use. Apple Search Ads's priority-first syncs fetch the most recent thirty days' worth of data from the reporting tables.
To learn more, see our priority-first sync documentation.
BigQuerylink
For partitioned tables, we ignore the partition and scan the whole table in the following scenarios:
- If you have selected a column that is not a primary key column
- If you have created an ingestion partitioned table
For example, we ignore the partition if the column is _fivetran_synced
or ingestion time partitioned
and scan the whole table while executing merge
or delete
queries to ensure data integrity.
Note: If there is no DATE or INSTANT column that is part of the primary key in your partitioned table, we scan the whole table in every sync in order to remove duplicate records.
We recommend you to select a partition column that is part of the primary key so we can scan only the corresponding partition.
We have added a task to your Fivetran dashboard that informs you that your API rate limit has exceeded, causing the syncs to fail. Contact BigQuery support to increase your quota limit.
Databrickslink
We now use our own Amazon S3 bucket as an intermediate storage for staging temporary data during a sync. Now, when setting up Databricks as your destination, you do not have to create a S3 bucket.
We will end support for clusters with Databricks Runtime 7.0 and below, on August 15, 2020. To prevent your integrations from failing or causing data loss, upgrade your Databricks Runtime to 7.1 before August 15, 2020.
Dropboxlink
We have improved the way we track which files we have already synced to make sure we only pull new or changed data from the source containers. Previously, we re-synced files that were created at the same time as the last observed cursor position. That ensured that we never missed any files that were created while we were syncing your data. That also meant that we sometimes synced the same files twice. Now, in addition to tracking the timestamp, we also track the names of the files we have already synced. We store up to 1,000 file names. We sync files created at the time of the last observed cursor position only if we don’t have the file in our list of synced files for that timestamp.
Facebook Ad Insightslink
We now support the conversion
value of the action_report_time
parameter. Previously, we retrieved your data using the default value impression
. In the connector setup form, set the Show Advanced Options toggle to ON and then select the value you want to use.
Freshdesklink
We have stopped inferring the data types of custom fields in the TICKET
table to improve data integrity. Previously, we used to downcast the STRING data type to other data types.
FTPlink
We have improved the way we track which files we have already synced to make sure we only pull new or changed data from the source containers. Previously, we re-synced files that were created at the same time as the last observed cursor position. That ensured that we never missed any files that were created while we were syncing your data. That also meant that we sometimes synced the same files twice. Now, in addition to tracking the timestamp, we also track the names of the files we have already synced. We store up to 1,000 file names. We sync files created at the time of the last observed cursor position only if we don’t have the file in our list of synced files for that timestamp.
FTPSlink
We have improved the way we track which files we have already synced to make sure we only pull new or changed data from the source containers. Previously, we re-synced files that were created at the same time as the last observed cursor position. That ensured that we never missed any files that were created while we were syncing your data. That also meant that we sometimes synced the same files twice. Now, in addition to tracking the timestamp, we also track the names of the files we have already synced. We store up to 1,000 file names. We sync files created at the time of the last observed cursor position only if we don’t have the file in our list of synced files for that timestamp.
Google Ad Managerlink
We have removed dimensions that are no longer supported in the Google Ad Manager reporting service. Read more about available dimensions in Google's Ad Manager API documentation.
Removed dimensions:
order_delivery_status
parent_ad_unit_id
parent_ad_unit_name
placement_status
video_break_type
video_break_type_name
is_first_look_deal
ad_unit_status
We have upgraded our Google Ad Manager connector from version 201911 to 202005 of the Ad Manager API and handled deprecations and updates. Read more about the deprecations and changes in Google's Ad Manager API documentation.
New dimensions:
custom_spot_id
custom_spot_name
nielsen_device_id
nielsen_device_name
New columns:
ad_server_cpm_and_cpc_revenue_gross
ad_server_all_revenue_gross
ad_exchange_video_auto_play
ad_exchange_video_click_to_play
ad_exchange_video_start
ad_exchange_video_midpoint
ad_exchange_video_complete
ad_exchange_video_completion_rate
ad_exchange_video_collapse
ad_exchange_video_expand
ad_exchange_video_full_screen
ad_exchange_video_mute
ad_exchange_video_pause
ad_exchange_video_resume
ad_exchange_video_rewind
ad_exchange_video_skipped
ad_exchange_video_unmute
ad_exchange_video_vast2_error_count
ad_exchange_video_vast_error_100_count
ad_exchange_video_vast_error_101_count
ad_exchange_video_vast_error_102_count
ad_exchange_video_vast_error_200_count
ad_exchange_video_vast_error_201_count
ad_exchange_video_vast_error_202_count
ad_exchange_video_vast_error_203_count
ad_exchange_video_vast_error_300_count
ad_exchange_video_vast_error_301_count
ad_exchange_video_vast_error_302_count
ad_exchange_video_vast_error_303_count
ad_exchange_video_vast_error_400_count
ad_exchange_video_vast_error_401_count
ad_exchange_video_vast_error_402_count
ad_exchange_video_vast_error_403_count
ad_exchange_video_vast_error_405_count
ad_exchange_video_vast_error_500_count
ad_exchange_video_vast_error_501_count
ad_exchange_video_vast_error_502_count
ad_exchange_video_vast_error_503_count
ad_exchange_video_vast_error_600_count
ad_exchange_video_vast_error_601_count
ad_exchange_video_vast_error_602_count
ad_exchange_video_vast_error_603_count
ad_exchange_video_vast_error_604_count
ad_exchange_video_vast_error_900_count
ad_exchange_video_vast_error_901_count
ad_exchange_video_total_vast_error_count
ad_exchange_video_total_error_rate
total_video_opportunities
total_video_capped_opportunities
dp_ecpm
Google Cloud Storagelink
We have improved the way we track which files we have already synced to make sure we only pull new or changed data from the source containers. Previously, we re-synced files that were created at the same time as the last observed cursor position. That ensured that we never missed any files that were created while we were syncing your data. That also meant that we sometimes synced the same files twice. Now, in addition to tracking the timestamp, we also track the names of the files we have already synced. We store up to 1,000 file names. We sync files created at the time of the last observed cursor position only if we don’t have the file in our list of synced files for that timestamp.
Google Playlink
We have improved the way we track which files we have already synced to make sure we only pull new or changed data from the source containers. Previously, we re-synced files that were created at the same time as the last observed cursor position. That ensured that we never missed any files that were created while we were syncing your data. That also meant that we sometimes synced the same files twice. Now, in addition to tracking the timestamp, we also track the names of the files we have already synced. We store up to 1,000 file names. We sync files created at the time of the last observed cursor position only if we don’t have the file in our list of synced files for that timestamp.
Google Sheetslink
We added a button to the setup form that allows you to check that the Service Account has access to the selected spreadsheet. Learn more about setting up in our Google Sheets connector documentation.
Greenhouselink
We now support syncing demographic data if you have a Greenhouse Expert plan. Fivetran requires access to the demographic endpoint to sync demographic data. To grant Fivetran access, create an API key with the demographic endpoint in Greenhouse, then enter the API key in your Fivetran connector setup form. Read Greenhouse's Harvest API Key Permissions documentation for more information.
HubSpotlink
We have added a warning to your Fivetran dashboard that asks you to increase your sync frequency if your COMPANY
and COMPANY_PROPERTY_HISTORY
tables have huge recurring updates. Previously, if these tables had more than 9900 updates, we initiated a complete re-sync to prevent data integrity issues. The re-sync caused high MAR (monthly active rows) consumption. You can prevent these re-syncs and avoid impact on your MAR by increasing your sync frequency.
Intercomlink
We now support Intercom REST API V2.0.
Jiralink
We have changed the styling of the connector name from "JIRA" to "Jira" in the Fivetran dashboard to match Atlassian's official styling.
Kustomerlink
We now support Kustomer applications that are hosted in the European Union (EU) region. When you create a new Kustomer connector, you can now designate EU as the region.
Leverlink
We now support Lever's Opportunity endpoint. The Opportunity endpoint replaces the deprecated Candidate endpoint. In addition to our existing access, grant Fivetran access to the Opportunity endpoint in your Lever dashboard, then add the regenerated API key to the connector setup form in your Fivetran dashboard. Fivetran automatically triggers a historical sync when we connect to this endpoint, so you do not need to manually re-sync your data.
LinkedIn Ad Analyticslink
The LinkedIn Ad Analytics connector now supports priority-first sync. Priority-first syncs fetch your most recent data first so that it's quickly ready for you to use. LinkedIn Ad Analytics priority-first syncs fetch the most recent thirty days' worth of data from the reporting tables.
To learn more, see our priority-first sync documentation.
Microsoft Advertisinglink
The Microsoft Advertising connector now supports priority-first sync. Priority-first syncs fetch your most recent data first so that it's quickly ready for you to use. Microsoft Advertising's priority-first syncs fetch the most recent thirty days' worth of data from the reporting tables.
To learn more, see our priority-first sync documentation.
NetSuite SuiteAnalyticslink
We have changed the sync strategy for the incremental tables that have the _fivetran_index
column as a primary key. We are gradually rolling out this change to all existing customers.
We have changed the sync strategy for the re-import tables that have the _fivetran_index
column as a primary key. We are gradually rolling out this change to all existing customers.
We have also changed the sync strategy of the _fivetran_index
column. Now, we use a monotonically increasing value for the _fivetran_index
column that is specific to each primary key. Previously, we used a monotonically increasing value that was specific to the full table. We have implemented the modified sync strategy for the following tables:
BILLING_SCHEDULE
ITEM_LOCATION_MAP
ITEM_PRICES
ITEM_VENDOR_PRICING
POSTING_ACCOUNT_ACTIVITY
REVRECSCHEDULELINES
TRANSACTION_SHIPPING_GROUPS
TRANSACTION_TRACKING_NUMBERS
We are gradually rolling out this change to all existing customers.
We now use the DELETED_RECORDS
table to capture deleted records for tables that have a single primary key and are updated incrementally. Previously, during the incremental update, we imported ranges, which caused high MAR (monthly active rows) consumption. We are gradually rolling out this change to all existing customers.
We have removed the Data Source field from the connector setup form because we only support NetSuite.com as a data source.
We have changed our sync strategy for tables that have a single primary key and are updated incrementally. We now use the data_last_modified
column to retrieve updates and new records, and we re-import ranges only to capture deletes. We are gradually rolling out this change to all existing customers.
Oraclelink
We have fixed a bug with very high precision NUMBER values from Oracle. Previously, we either truncated these values or did not sync them to the destination. Now, we sync these values to the destination without truncating them, but we change the column type to STRING to accomodate high precision values. Re-sync any tables where you have previously synced high precision NUMBER values to ensure data integrity.
Pinterest Adslink
The Pinterest Ads connector now supports priority-first sync. Priority-first syncs fetch your most recent data first so that it's quickly ready for you to use. Pinterest Ads's priority-first syncs fetch the most recent thirty days' worth of data from the reporting tables.
To learn more, see our priority-first sync documentation.
PostgreSQLlink
We now support logical replication on Aurora PostgreSQL versions 10.6 or later.
SendGridlink
We have updated our connector to support SendGrid accounts based on both New Marketing Campaigns and Legacy Marketing Campaigns. SendGrid released a new set of API endpoints under new marketing campaigns and will be deprecating their legacy marketing campaigns API.
We have integrated the new endpoints to support customers using the new SendGrid platform. We now use the latest SendGrid API v3 to sync data from SendGrid.
As part of the update, we now classify the connector based on the SendGrid account type you are using. We have adopted custom sync strategies for each connector type:
Legacy: If you are using a legacy SendGrid account and have configured your connector with a legacy API key, we classify the connector as Legacy. We upsert the legacy tables in to the destination. See the schema information for details on the legacy tables. This schema applies to all legacy SendGrid connectors created before July 20, 2020.
Important: We recommend you to switch to a new marketing campaign account, as SendGrid will deprecate the legacy APIs in the near future. We won't be able to support connectors configured with legacy API keys.
Upgraded: If you were using a legacy SendGrid account and had configured your connector with a legacy API key, but have migrated your account to a new marketing campaign account, and also configured the connector with the new API key, we classify the connector as Upgraded. We upsert data from both new marketing tables and mapped legacy tables into the destination to support your existing queries on the legacy tables.
Note: We recommend you to create new queries on the new tables to replace your existing queries on the legacy tables.
- New: If you have a new SendGrid account, and have created a new connector using a new API key, we classify the connector as New. We upsert the new marketing tables into the destination. See the schema information for details on the new marketing tables. This schema applies to all new SendGrid connectors created after July 20, 2020.
SFTPlink
We have improved the way we track which files we have already synced to make sure we only pull new or changed data from the source containers. Previously, we re-synced files that were created at the same time as the last observed cursor position. That ensured that we never missed any files that were created while we were syncing your data. That also meant that we sometimes synced the same files twice. Now, in addition to tracking the timestamp, we also track the names of the files we have already synced. We store up to 1,000 file names. We sync files created at the time of the last observed cursor position only if we don’t have the file in our list of synced files for that timestamp.
Snapchat Adslink
The Snapchat Ads connector now supports priority-first sync. Priority-first syncs fetch your most recent data first so that it's quickly ready for you to use. Snapchat Ads's priority-first syncs fetch the most recent thirty days' worth of data from the reporting tables.
To learn more, see our priority-first sync documentation.
Snowflakelink
We have reduced the sync duration of append-only tables. During internal performance testing, we observed a reduction in sync time for destinations with large table sizes. We are gradually rolling out this change to all our existing customers.
Snowplowlink
We now support Cross-Domain Tracking.
Stripelink
We now capture deletes for the INVOICE
, INVOICE_ITEM
, and PERSON
tables. Deletes from these tables are captured in the is_deleted
column.
You can now select the PAYMENT_METHOD
and CUSTOMER_BALANCE_TRANSACTION
tables to sync from the Schema tab on your connector details page.
Taboolalink
The Taboola connector now supports priority-first sync. Priority-first syncs fetch your most recent data first so that it's quickly ready for you to use. Taboola's priority-first syncs fetch the most recent thirty days' worth of data from the reporting tables.
To learn more, see our priority-first sync documentation.
Twitter Adslink
We can now sync the records that you have deleted from the funding_instruments
, campaigns
, line_items
, and promoted_tweets
entities.
Webhookslink
You can now configure your Webhooks connector using the Fivetran REST API. This feature is in BETA and is available only for Standard and Enterprise accounts.
REST API Improvementslink
Create a Connector Public API Endpoint Extensionlink
We have expanded the current public API endpoint Create a Connector. Now, you can create a new connector in a paused state by specifying the paused
parameter in payloads.
New Public API Endpointslink
We have implemented new Destination Management API endpoints for our REST API. You can now leverage the new Retrieve Destination Details feature.
Snowplowlink
You can now configure your Snowplow connector using the Fivetran REST API. This feature is in BETA and is available only for Standard and Enterprise accounts.