Release Noteslink
March 2024link
Our MariaDB and MySQL connectors now support the TRUNCATE
command.
June 2022link
Fivetran Teleport Sync is now generally available. Learn more in our Fivetran Teleport Sync for MySQL documentation.
March 2022link
Fivetran Teleport Sync is now available in beta. Learn more in our Fivetran Teleport Sync for MySQL documentation.
May 2021link
To correct data integrity issues, we will now trigger a re-sync for a table when we detect a reduction in the size of one of its SET columns. Before the re-sync begins, you may notice a SET type change detected
error in your logs.
When we detect a reduction in the size of a table's ENUM columns, we now create an ENUM type change detected
error in your logs before we trigger a re-sync for that table. Previously, we added FIVETRAN_ERROR_ENUM_DELETED
values to your ENUM columns before we triggered a re-sync.
March 2021link
Data syncs no longer fail without explanation when we encounter prohibited DATETIME values (for example, 29 Feb 2021
). How we respond to prohibited values varies by sync type:
Initial syncs: The sync fails when it encounters a prohibited value. We create a Task in your Fivetran dashboard with instructions on how to resolve the issue.
Incremental updates: The sync does not fail when it encounters a prohibited value. We process the prohibited value and create a Warning in your Fivetran dashboard with instructions on how to resolve the issue. The way we process the prohibited value depends on your sync mode:
- Soft delete mode: We ignore the prohibited value.
- History mode: If the prohibited value is a primary key, we ignore it. If the prohibited value is not a primary key, we change it to NULL.
Learn more about Tasks and Warnings in our Alerts documentation.
April 2020link
We have increased the fractional second precision of our binary log parser from millisecond to microsecond for DATETIME, TIMESTAMP, and TIME types. To correct existing values of these types in your destination, trigger a table re-sync or a historical re-sync in the connector dashboard.
We now add a _fivetran_synced
column while syncing empty tables and columns from the source.
March 2020link
We have fixed a problem where unsigned BIGINT values larger than 9223372036854775807 were sometimes parsed incorrectly during incremental updates. If any values were incorrectly synced prior to this change, you will need to initiate a re-sync to correct them.
We now replicate empty tables in a MySQL source database as empty tables in the destination.
To correct data integrity issues, we will now trigger a re-sync for a table when we detect a reduction in the size of one of its ENUM columns. Before the re-sync begins, you may notice FIVETRAN_ERROR_ENUM_DELETED
values in these columns.
October 2019link
We are now rolling out support for BLOB, TINYBLOB, MEDIUMBLOB, LONGBLOB, and YEAR data types in version 4 of our MySQL connector.
We are now rolling out the ability to sync MySQL DATETIME columns to the destination as TIMESTAMP_NTZ columns for version 4 of our MySQL connector.
We have a fixed a bug that caused zeroed DATE and DATETIME values to sync incorrectly. MySQL allows
0000-00-00
and0000-00-00 00:00:00
as values for DATE and DATETIME primary key columns respectively. Previously, we converted these values to1970-01-01
and1970-01-01 00:00:00
. Therefore, zeroed times and the real values of1970-01-01
and1970-01-01 00:00:00
were mapped to the same value in the destination. Fivetran now converts these values as follows:- DATE: from
1970-01-01
to-999999999-01-01
- DATETIME: from
1970-01-01 00:00:00
to-999999999-01-01 00:00:00
- DATE: from
September 2019link
If you have a logging service connected to your Fivetran account, the import progress of your tables in the event logs will be visible in your logs as an
import_progress
event. Table names will be in either theCOMPLETE
orIN_PROGRESS
state, depending on their sync status. Tables that are not reported in the event have not started their import.We have begun rolling out support for the following spatial types in version 4 of our MySQL connector:
- GEOMETRY
- GEOMETRYCOLLECTION
- LINESTRING
- MULTILINESTRING
- POINT
- MULTIPOINT
- POLYGON
- MULTIPOLYGON
August 2019link
Fivetran has released version 4 of our MySQL connector, which offers improved sync reliability. We will add a number of upgrades to this version in the coming months:
We will sync DATETIME values as TIMEZONE_NTZ values in your destination.
We will support parsing for TRUNCATE DDL events.
We will support the following new data types:
- BINARY
- VARBINARY
- YEAR
- BLOB
- SMALLBLOB
- MEDIUMBLOB
- LONGBLOB
- GEOMETRY
- GEOMETRYCOLLECTION
- LINESTRING
- MULTILINESTRING
- POINT
- MULTIPOINT
- POLYGON
- MULTIPOLYGON
This month, we have begun rolling out support for BINARY and VARBINARY types.
For more information about data types, see our MySQL documentation.
June 2019link
We have added a test to the MySQL setup forms which checks if you are using RDS. We added the test because if you select the wrong variant in the setup form, your connector may not work properly.
The test fails when there is a discrepancy between your selection and what Fivetran detected. If you are not confident in your selection, you may go back to the Add Connector view and select the connector type that matches your MySQL database variant.
Fivetran no longer supports invalid Date
, Datetime
and Timestamp
values. For example, 2019-00-01 00:00:00
is invalid. This change immediately affects all new MySQL connectors, and we will roll it out gradually to existing connectors.
The supported ranges of Date
, Datetime
and Timestamp
in MySQL are:
Type | Supported Range |
---|---|
Date | '1000-01-01' to '9999-12-31' |
Datetime | '1000-01-01 00:00:00' to '9999-12-31 23:59:59' |
Timestamp | '1970-01-01 00:00:01' UTC to '2038-01-19 03:14:07' UTC |
We will load these invalid values as NULL
into the destination and display a warning on your Fivetran dashboard. The warning will indicate the schemas, tables, and columns which contain NULL
values.
March 2019link
Fivetran has added core support for a TIMESTAMP WITHOUT TIMEZONE type. Up until now, Fivetran assumed that timestamps without timezone were in UTC. This addition means databases with a TIMESTAMP WITHOUT TIMEZONE type - MySQL, Oracle, PostgreSQL, SQL Server - will replicate to a more correct destination type. TIMESTAMP WITHOUT TIMEZONE support is immediately available for Oracle, PostgreSQL, and SQL Server. MySQL / MariaDB support will be released soon.
Source | Applicable Type |
---|---|
Oracle | TIMESTAMP |
PostgreSQL | TIMESTAMP |
SQL Server | DATETIME, DATETIME2, SMALLDATETIME |
MySQL / MariaDB | DATETIME |
This change is backwards compatible. Fivetran's core will continue assuming UTC time zone if the destination type is TIMESTAMP WITH TIMEZONE. In order to get the more specific type, drop the table in the destination and re-sync the table in your Fivetran dashboard.
We are changing our architecture to allow for much higher data throughput. Starting on April 30, 2019, all Fivetran connectors will originate from a new set of fixed IP addresses. For the US region, these IPs are:
- 35.227.135.0/29
- 35.234.176.144/29
- 52.0.2.4/32.
Update your IP safelists for the following databases and warehouses before April 30. Not doing so will result in a connection failure.
Databases:
- MariaDB
- MongoDB
- MySQL
- Oracle
- PostgreSQL
- SQL Server
Warehouses:
- Redshift
- Azure Synapse
- Snowflake
If you don't use any of the above databases or warehouses with Fivetran, you don't need to take any action.
November 2018link
You can now trigger re-syncs of individual tables from your Fivetran dashboard:
The re-sync button appears when you hover your cursor over the row that contains the table name.
This feature is available for the following connectors:
- DynamoDB
- MongoDB
- MySQL
- NetSuite SuiteAnalytics
- Oracle
- PostgreSQL
- Salesforce
- SQL Server
- Zuora
We have identified and fixed an extremely rare bug that can cause data to be missed during binary log replication. The bug meant that, in some cases, our system believed that all data had been successfully synced, when the binary log replication process had actually stopped prematurely.
October 2018link
For the following warehouses, Fivetran now drops the primary_key
field from existing tables in your warehouse if there is a primary_key
coming in from your data source:
- MySQL
- PostgreSQL
- SQL Server
- Redshift
September 2018link
The MySQL connector is now able to handle unescaped control characters in textual data as if they had been properly escaped.
July 2018link
You can now customize the replica ID used to access your database's binary logs. By default, the replica ID is a random integer greater than 1000.
May 2018link
We now fully support the SET
data type and consistently sync it as a JSON array. Previously, we synced SET
as a string during the initial import and as an integer when processed from the binary logs. If any values were incorrectly synced prior to this change, you will need to initiate a re-sync to correct them.