We can now sync tables that do not have all of their columns included in their CDC instances. For these tables, we only sync the columns that are present in the CDC instance. To include excluded columns, delete your old CDC instance and create a new one that includes all columns.
We have optimized the queries that we use to retrieve table and column metadata from your database during our data syncs. Our new queries improve your sync speed, especially if you have large schemas.
We have changed our sync strategy for users who chose to exclude new tables from their data syncs by default. Our new sync method improves your sync speed, especially if you have a large number of tables.
We now add a fivetran_synced column while syncing empty tables and columns from the source.
Fivetran has released an updated version of our SQL Server connector, which offers improved sync reliability. We will add a number of upgrades to this version in the coming weeks:
We will support Change Data Capture as an incremental update mechanism for all customers.
We will support the following data types:
For more information, see our SQL Server documentation.
We now convert INSTANT data type to DATETIMEOFFSET data type in the destination. Previously, we converted INSTANT data type to DATETIME2 data type.
We are making this change because the DATETIMEOFFSET datatype contains time zone information, which allows you to view the time zone in the destination if the source data type is INSTANT.
This functionality is enabled by default for new connectors created after March 25, 2020. Contact our Support Team to upgrade the INSTANT data type mapping to DATETIMEOFFSET for your existing connector.
We now support Change Data Capture as an incremental update mechanism. To learn more, see our SQL Server setup guides.
Our SQL Server connector can now sync empty tables. You do not need to take any action to take advantage of this improvement. All the empty tables you previously selected to sync will now sync.
If you have a logging service connected to your Fivetran account, the import progress of your tables in the event logs will be visible in your logs as an
import_progress event. Table names will be in either the
IN_PROGRESS state, depending on their sync status. Tables that are not reported in the event have not started their import.
Tables with clustered indices are now queried using smaller page sizes. This reduces the contiguous length of time that a Fivetran connector has a lock on a table from which it's reading, without adversely affecting sync speeds.
For the following warehouses, Fivetran now drops the
primary_key field from existing tables in your warehouse if there is a
primary_key coming in from your data source:
- SQL Server
We can now sync columns with the data type
DATETIMEOFFSET. The values are converted to UTC using the timezone information present in the data.