June 2019

Fivetran Status page

You can now check the status of the Fivetran Web APP, API, and Replication Servers by visiting the Status page.

Fivetran Status Page

You can also subscribe to our status page to receive updates on any incidents via email.

Invalid Date and Time values

Fivetran no longer supports invalid DATE, DATETIME, and TIMESTAMP values. For example, 2019-00-01 00:00:00 is invalid. This change immediately affects all new MySQL and MariaDB connectors, and we will roll it out gradually to existing connections.

The supported ranges of DATE, DATETIME, and TIMESTAMP in MySQL are:

Type	Supported Range
DATE	'1000-01-01' to '9999-12-31'
DATETIME	'1000-01-01 00:00:00' to '9999-12-31 23:59:59'
TIMESTAMP	'1970-01-01 00:00:01' UTC to '2038-01-19 03:14:07' UTC

We will load these invalid values as NULL into the destination and display a warning on your Fivetran dashboard. The warning will indicate the schemas, tables, and columns which contain NULL values.

New connectors

Heap

Sync records from the Heap platform. Heap is an automation tool for user analytics.

Read our Heap connector documentation.

Schema changes

GitHub

We now support Projects in our GitHub connector. We have added four new tables:

PROJECT table with the following columns:
- id
- repository_id
- creator_id
- body
- number
- state
- name
- created_at
- updated_at
- is_deleted
COLUMN table with the following columns:
- id
- project_id
- name
- created_at
- updated_at
- is_deleted
CARD table with the following columns:
- id
- creator_id
- column_id
- issue_id
- note
- archived
- created_at
- updated_at
- is_deleted
ISSUE_PROJECT_HISTORY table with the following columns:
- issue_id
- updated_at
- actor_id
- project_id
- card_id
- removed
- column_name
- previous_column_name

We have added the is_deleted column to the MILESTONE table to capture deletes.

Help Scout

Version 1.0 of the Help Scout API is now deprecated. Version 2.0 of the Help Scout API has caused major changes in the schema. The new schema contains the following tables:

USER_HISTORY
MAILBOX_HISTORY
MAILBOX_FOLDER_HISTORY
MAILBOX_WORKFLOW_HISTORY
TAG_HISTORY
CONVERSATION_HISTORY
CONVERSATION_TAG_HISTORY
CONVERSATION_THREAD_HISTORY
CONVERSATION_THREAD_BCC_HISTORY
CONVERSATION_THREAD_CC_HISTORY
CONVERSATION_THREAD_TO_HISTORY
CONVERSATION_THREAD_ATTACHMENT_HISTORY
CUSTOMER_HISTORY
CUSTOMER_SOCIAL_PROFILE_HISTORY
CUSTOMER_EMAIL_HISTORY
CUSTOMER_PHONE_HISTORY
CUSTOMER_CHAT_HISTORY
CUSTOMER_WEBSITE_HISTORY
HAPPINESS_RATING

Except the HAPPINESS_RATING table, all tables have become history tables and have the word HISTORY appended to indicate that.

Each history table has an updated_at field which is a part of the primary key. When a record is updated, its updated_at changes, and so does its primary key. Thus, the older record remains preserved as history.

We have temporarily removed two tables. Help Scout didn't include an endpoint for these two tables in version 2.0 of their API. Once they add the endpoint, we will sync these two tables again:

TEAM
TEAM_USER

Optimizely

Optimizely has introduced a new way to access your processed results data via an S3 export. We have added two tables to support access to that data:

RESULT table with the following columns:
- id
- campaign_id
- experiment_id
- project_id
- variation_id
- timestamp
- account_id
- visitor_id
- session_id
- event_id
- count
- revenue
- value
- received_timestamp
RESULT_SEGMENT table with the following columns:
- id
- result_id
- value

Pendo

We now provide segmentio metadata in ACCOUNT_HISTORY and VISITOR_HISTORY tables.

Quickbooks

We have added a new column, purchase_id, to the PURCHASE_ORDER_LINKED_TXN table to support Purchase transaction types.

We have added a new column, linked_bill_payment_id, to the BILL_PAYMENT_LINE table to support BillPaymentCheck transaction types.

Recharge

We have added three new columns, price, sku, and grams, to the ORDER_LINE_ITEM table.

Improvements

Amazon S3

We now require you to configure a Role ARN when you set up an Amazon S3 connector, even if your bucket is a public bucket.

Existing Amazon S3 connectors will not be affected. However, if you reconfigure an existing connector or create a new one, you will have to configure a Role ARN even if your bucket is a public bucket.

We have added a new connection test to the Amazon S3 setup form that helps you adopt security best practices. The test checks your bucket policy settings to make sure you have created your Role ARN using the given externalID.

If you mistakenly create your Role ARN without using the give externID, the configuration will now fail the test. You will have to correct your Role ARN.

AppsFlyer

We have released a new version of the AppsFlyer connector and have made the following schema changes:

We have split the EVENT table from the old version into two new tables:
- DATA_LOCKER_EVENTS
- WEBHOOK_EVENTS
The DATA_LOCKER_EVENTS table captures events from AppsFlyer Data Locker. The WEBHOOK_EVENTS table captures webhook events.
We have removed the following tables:
- WEBHOOK_EVENT_ATTRIBUTE
- S3_EVENT_ATTRIBUTE

The new version supports AppsFlyer Data Locker 2.0.

The new version resolves a data integrity issue with events and webhooks. Fivetran now better captures events from AppsFlyer Data Locker and webhooks.

GitHub

We now use the GitHub Webhook API. It allows us to make incremental updates faster and capture deletes for the following tables:

MILESTONE
PROJECT
COLUMN
CARD

MySQL

We have added a test to the MySQL setup forms which checks if you are using RDS. We added the test because if you select the wrong variant in the setup form, your connector may not work properly.

The test fails when there is a discrepancy between your selection and what Fivetran detected. If you are not confident in your selection, you may go back to the Add Connector view and select the connector type that matches your MySQL database variant.

Oracle

We can now sync changes to primary keys.

For each table where the primary key is expected to change, you need to run the following query:

ALTER TABLE "<schema>"."<table>" ADD SUPPLEMENTAL LOG DATA (ALL) COLUMNS

If you don't configure the logging data correctly, you will receive a warning when Fivetran encounters a primary key change. The warning will give you customized instructions.

We will gradually roll out this improvement to connectors over the next month.

Zendesk Support

We now exclude the ORGANIZATION_MEMBERSHIP table from the sync by default because it can be a large table and may degrade performance. To sync the ORGANIZATION_MEMBERSHIP table, go to the Schema tab of your Fivetran dashboard and select it.