January 2021

Transformations for dbt Core

Email notifications

You can now enable email notifications for each of your dbt transformations on your Fivetran dashboard's Notifications tab. To learn more about email notifications, see our Notifications documentation.

New connectors

Instagram Business

Instagram Business provides metadata and insights for IG User (Instagram Business Account) and IG Media (Instagram Photo, Video, Story, or Album) objects.

Read our Instagram Business connector documentation.

Schema changes

Eloqua

We have added the following new columns to the CAMPAIGN table:

accessed_at
badge_id
clr_end_date
created_by
current_status
first_activation
is_read_only
member_count
run_as_user_id
scheduled_for
source_template_id
updated_by

Trigger a re-sync to backfill the history data for these columns.

We have added 3 new tables to support Eloqua's Email API:

EMAIL
EMAIL_FORM
EMAIL_HYPERLINK

Google Ad Manager

We deprecated the following columns on January 13, 2021 to support version 202008 of the Ad Manager API:

grp_population
grp_unique_audience
grp_unique_audience_share
grp_audience_impressions
grp_audience_impressions_share
grp_audience_reach
grp_audience_average_frequency
grp_gross_rating_points

Be sure to update your queries. Learn more about available report columns in Google Ad Manager's API documentation.

Google Ads

We now sync metadata tables that were previously available in the Google Ads Account connector:

ACCOUNT
AD_FINAL_APP_URL_HISTORY
AD_FINAL_MOBILE_URL_HISTORY
AD_FINAL_URL_HISTORY
AD_GROUP_CRITERION_HISTORY
AD_GROUP_CRITERION_LABEL_HISTORY
AD_GROUP_EXPLORER_AUTO_OPTIMIZER_SETTING_HISTORY
AD_GROUP_HISTORY
AD_GROUP_LABEL_HISTORY
AD_GROUP_LABEL_HISTORY
AD_GROUP_TARGETING_SETTING_HISTORY
AD_HISTORY
AD_LABEL_HISTORY
AD_POLICY_CONSTRAINT_HISTORY
AD_POLICY_EVIDENCE_HISTORY
AD_POLICY_TOPIC_HISTORY
BUDGET_HISTORY
CALL_ONLY_AD_HISTORY
CAMPAIGN_CONVERSION_OPTIMIZER_ELIGIBILITY_HISTORY
CAMPAIGN_CRITERION_HISTORY
CAMPAIGN_CRITERION_PRODUCT_SCOPE
CAMPAIGN_DYNAMIC_SEARCH_ADS_SETTING_HISTORY
CAMPAIGN_GEO_TARGET_TYPE_SETTING_HISTORY
CAMPAIGN_HISTORY
CAMPAIGN_LABEL_HISTORY
CAMPAIGN_REAL_TIME_BIDDING_SETTING_HISTORY
CAMPAIGN_SHOPPING_SETTING_HISTORY
CAMPAIGN_TARGETING_SETTING_HISTORY
CAMPAIGN_TRACKING_SETTING_HISTORY
CUSTOM_PARAMETER_HISTORY
DEPRECATED_AD_HISTORY
DISAPPROVAL_REASON_HISTORY
DYNAMIC_SEARCH_AD_HISTORY
EXPANDED_DYNAMIC_SEARCH_AD_HISTORY
EXPANDED_TEXT_AD_HISTORY
FINAL_APP_URL_HISTORY
FINAL_MOBILE_URL_HISTORY
FINAL_URL_HISTORY
GMAIL_AD_HISTORY
IMAGE_AD_HISTORY
LABEL
LOCATION
RESPONSIVE_DISPLAY_AD_HISTORY
SHOWCASE_AD_HISTORY
TEMPLATE_AD_HISTORY
TEXT_AD_HISTORY

We now sync tables related to prebuilt reports that were previously configured during the connector setup:

ACCOUNT_STATS
ACCOUNT_HOURLY_STATS
AD_STATS
AD_GROUP_STATS
AD_GROUP_HOURLY_STATS
AUDIENCE_STATS
CAMPAIGN_STATS
CAMPAIGN_HOURLY_STATS
KEYWORD_STATS

For more information, see the Google Ads ERD.

HubSpot

We have added a new table, DEAL_STAGE which stores the dealstage properties data. Previously, we synced the dealstage data to the DEAL_PROPERTY_HISTORY table. The HubSpot API returns only the date_entered value. Use the date_entered and date_exited values to calculate the time_in value.

Jira

We now support the name subfield in the Customer Request Type field of Jira Service Management (formerly Jira Service Desk).

Kustomer

We have added a new column, satisfaction_level_first_answer, to the CONVERSATION table.

Microsoft Advertising

We have added a new column, campaign_status, to the CAMPAIGN_PERFORMANCE_* and CAMPAIGN_IMPRESSION_PERFORMANCE_* tables.

We have added a new column, keyword_status, to the KEYWORD_PERFORMANCE_* tables.

Recurly

We have added a new table, GIFT_CARD. We are gradually rolling out this new feature to all existing connectors. If you'd like to enable the table on your connector, contact our support team.

Shopify

We now sync the properties field as JSON objects. We have added a new column, properties replacing the property_* column in the following tables:

ABANDONED_CHECKOUT_LINE
DRAFT_ORDER_LINE
ORDER_LINE

We now sync the note_attributes field as JSON objects. We have added a new column, note_attributes replacing the note_attribute__* column in the following tables:

ABANDONED_CHECKOUT
DRAFT_ORDER
ORDER

We will deprecate the properties_* and note_attribute__* columns from the Shopify schema after March 31, 2021. Be sure to update your queries.

We will deprecate the ORDER_NOTE_ATTRIBUTE and DRAFT_ORDER_NOTE_ATTRIBUTE tables after March 31, 2021. Be sure to update your queries.

We have removed the default_address_id column from the CUSTOMER table. You can use the is_default column in the CUSTOMER_ADDRESS table instead. First, re-sync the CUSTOMER_ADDRESS table to populate the is_default column, then update your queries.

We have removed the is_default column from the ADDRESS table.

We have added two new columns, delivery_expectation_range_min and delivery_expectation_range_max, to the ABANDONED_CHECKOUT_SHIPPING_LINE table.

We have added the following new columns to the CUSTOMER table:

can_delete
lifetime_duration
multipass_identifier

Square

We have added a new column, customer_id, to the ORDER table. See Square's Order object documentation for more information.

We have added two new columns, present_at_location_ids and absent_at_location_ids, to the CATALOG_OBJECT table.

Stripe

We have added a new field, metadata, to the SUBSCRIPTION_SCHEDULE table.

Improvements

Apache Kafka

We now support Protobuf messages.

Apple App Store

The Apple App Store connector now requires two-factor authentication for all connectors, including existing connectors. Learn more in our setup instructions.

AppsFlyer

The AppsFlyer connector now supports syncing Data Locker events from your custom AWS S3 buckets. Previously, we only supported Data Locker events from AppsFlyer-managed buckets. See our setup instructions for more information.

Confluent Cloud

We now support Protobuf messages.

Criteo

The Criteo connector now supports twenty-five global currencies. You can now opt to select one of the following currency codes when you configure the connector:

EUR
USD
GBP
CHF
JPY
BGN
CZK
DKK
HUF
LTL
PLN
RON
SEK
NOK
HRK
RUB
TRY
AUD
BRL
CAD
CNY
HKD
IDR
INR
KRW
NOTE: If you don't select the currency, Fivetran will fetch the reports with the default currency that the Criteo API returns.

Facebook Pages

The Facebook Pages connector now supports priority-first sync. Priority-first syncs fetch your most recent data first so that it's quickly ready for you to use. Facebook Pages priority-first syncs fetch the most recent thirty days' worth of data from the page insights tables. To learn more, see our priority-first sync documentation.

Google Ad Manager

We have upgraded our Google Ad Manager connector from version 202005 to 202008 of the Ad Manager API and handled deprecations and updates. Learn more about the deprecations and changes in Google Ad Manager's API documentation.

Google Ads

We now support metadata tables that were previously available in the Google Ads Account connector. You can select or deselect the metadata tables on the Schema tab of your dashboard or select or deselect them during the Schema configuration using our REST API.

You can now configure custom reports in the connector setup form and existing prebuilt reports from the Schema tab of your connector dashboard.

We have changed the configuration format. We have deprecated the Prebuilt configuration type and support it only for backward compatibility, and we will end support soon.

We have replaced the Prebuilt report type with the Custom report type that has a predefined set of selected fields. You can select the tables in all existing prebuilt reports during the Schema configuration.

For more information, see our Google Ads schema documentation and our Google Ads REST API documentation.

These changes only affect new Google Ads connectors created after January 18, 2021. We will migrate existing Google Ads and Google Ads Account connectors to Google Ads connectors in the new format in the future.

Intercom

We have released pre-built, dbt Core-compatible data models for Intercom. Find the models in Fivetran's dbt hub or data models documentation. Learn more about our dbt Core integration in our Transformations for dbt Core documentation*.

* dbt Core is a trademark of dbt Labs, Inc. All rights therein are reserved to dbt Labs, Inc. Fivetran Transformations is not a product or service of or endorsed by dbt Labs, Inc.

Iterable

You can now exclude the LIST_USER_HISTORY table from your syncs to reduce your MAR usage. On your Connector Details page, go to the Schema tab and deselect the table.

By default, we include the LIST_USER_HISTORY table in your syncs. You may observe data discrepancies if you exclude this table.

Jira

We now use the X-Force-Accept-Language header to get responses in English. This applies to all Jira connectors created after February 1, 2021.

Lever

We have released pre-built, dbt Core-compatible data models for Lever. Find the models in Fivetran's dbt hub or data models documentation. Learn more about our dbt Core integration in our Transformations for dbt Core documentation*.

* dbt Core is a trademark of dbt Labs, Inc. All rights therein are reserved to dbt Labs, Inc. Fivetran Transformations is not a product or service of or endorsed by dbt Labs, Inc.

Mixpanel

The Mixpanel connector now supports projects that are hosted in the European Union (EU) region. You can now sync event data from data stores located in the EU.

MySQL

We have fixed a bug where TIME values were sometimes synced to the destination incorrectly. Previously, we added a .0 to values with no decimals and dropped the hundredths place for values that did. We now sync TIME values to the destination as they appear in the source:

Source value	Destination value (with bug)	Destination value (fixed)
`23:59:59`	`23:59:59.0`	`23:59:59`
`23:59:59.00`	`23:59:59.0`	`23:59:59.00`
`23:59:59.10`	`23:59:59.1`	`23:59:59.10`

If any values were incorrectly synced prior to this change, you will need to initiate a historical re-sync or table re-sync to correct them.

Pardot

In your connector setup form, you can now limit the REST API calls that your Pardot connector makes per day. By default, the REST API call limit is 150,000 per day.

We have released pre-built, dbt Core-compatible data models for Pardot. Find the model in Fivetran's dbt hub. Learn more about our dbt Core integration in our Transformations for dbt Core documentation*.

* dbt Core is a trademark of dbt Labs, Inc. All rights therein are reserved to dbt Labs, Inc. Fivetran Transformations is not a product or service of or endorsed by dbt Labs, Inc.

The Pardot connector now supports priority-first sync. Priority-first syncs fetch your most recent data first so that it's quickly ready for you to use. Pardot's priority-first syncs fetch the most recent 14 days' worth of data from the EMAIL, VISIT, and VISITOR_ACTIVITY tables. To learn more, see our priority-first sync documentation.

PostgreSQL

We now support domains, which are user-defined data types with optional constraints. Learn more in PostgreSQL's domains documentation.

We now support logical replication on Azure Database for PostgreSQL version 9.5 and later.

QuickBooks

We have released pre-built, dbt Core-compatible data models for QuickBooks. Find the models in Fivetran's dbt hub or data models documentation. Learn more about our dbt Core integration in our Transformations for dbt Core documentation*.

* dbt Core is a trademark of dbt Labs, Inc. All rights therein are reserved to dbt Labs, Inc. Fivetran Transformations is not a product or service of or endorsed by dbt Labs, Inc.

Recurly

We have fixed a bug where the first name and last name values returned by the Recurly API were sometimes not synced to your destination. Trigger a re-sync to ensure data integrity.

Redshift

You can now opt to use AWS IAM to authenticate Fivetran requests to your Redshift cluster. See our setup instructions for more information.

Salesforce

We now support the following Salesforce objects:

Announcement
ContentDocumentLink
ContentFolderItem
ContentFolderMember
IdeaComment
Vote

If you'd like to enable the feature on your connector, contact our support team.

Twilio

You can now exclude the MESSAGE table from your syncs. On your Connector Details page, go to the Schema tab and deselect the table. By default, we include the table in the sync.

Zendesk Support

You can now exclude the TICKET_FIELD_HISTORY and TICKET_TAG_HISTORY tables from your syncs. On your Connector Details page, go to the Schema tab and deselect the tables. By default, we include the tables in the sync.

We now sync the TICKET_FIELD_HISTORY and TICKET_TAG_HISTORY tables on a priority-first basis.

REST API Improvements

New connector configuration return format in our API

We have introduced a new (v2) connector configuration return format for the following endpoints:

New REST API Endpoints

We have implemented a new Certificate Management API endpoint for our REST API. You can now leverage the new Approve a connector fingerprint feature.

You can also approve fingerprints automatically by specifying true as a value of the new trust_fingerprints parameter in the existing Connector Management API endpoints:

Support external OAuth credentials to authorize connectors in Public API

We have expanded the public API endpoints Create a Connector and Modify a Connector. Now you can authorize the following connectors by adding the auth section to payloads: