Release Notes
June 2024
Partitioned tables
| Date | Event/Change | Details |
|---|---|---|
| June 27, 2024 | Change in partitioned tables | The Fivetran dashboard displays only the partitioned table for existing connections. Data syncs do not change, unless your connection meets one of four scenarios*. Previously, the dashboard displayed both the partitioned table and child tables. |
| June 27, 2024 | 3-month buffer period starts | For connections that fall within the four scenarios*, we provide a three-month buffer period so you can make the recommended changes. (We have reached out to users whose connections fall within the four scenarios.) |
| September 27, 2024 | 3-month buffer period ends | For all connections with partitioned tables, the Fivetran dashboard displays only the partitioned table. We then sync data into the partitioned table in destination, unless you previously reached out to Fivetran requesting data to be synced into partitions. In that case, there will be no change in sync behavior or dashboard display. |
* The four scenarios are as follows:
- If your connection uses XMIN or Teleport and the Fivetran user does not have select access to all partitions of a partitioned table, we can no longer select and sync that partitioned table after the three-month buffer period ends. We recommend you grant select access to all partitions of the partitioned table you want to sync.
- If your connection uses logical replication, your PostgreSQL version is below 13, and the Fivetran user did not include all partitions of the partitioned table in the publication, we can no longer select and sync that partitioned table after the three-month buffer period ends. We recommend you include all partitions of the partitioned table in your publication.
- If the partitioned table or some of its partitions are not selected in the Fivetran dashboard, we can no longer select and sync that partitioned table after the three-month buffer period ends. We recommend you select the partitioned table in the Fivetran dashboard.
- If your connection uses logical replication, the partitioned table is selected in the Fivetran dashboard, and none of the partitions is selected in the Fivetran dashboard, we do not currently sync data into the partitioned table. However, we can now sync into the partitioned table after the three-month buffer period ends. If you do not want data to be synced into the partitioned table, deselect the partitioned table in the Fivetran dashboard.
If you want to only select partial partitions of a partitioned table in the Fivetran dashboard and have data sync into the partitions, reach out to Fivetran Support and ask that the data be synced into partitions.
Inherited tables
| Date | Event/Change | Details |
|---|---|---|
| June 27, 2024 | 3-month buffer period starts | For all existing connections with inherited tables, we provide a three-month buffer period so you can make the recommended changes. Until the buffer period ends, the base table still shows the result of SELECT * FROM base_table for inherited tables. Learn more in our Partitioning using inheritance documentation. (We have reached out to users whose connections have inherited tables.) |
| July 27, 2024 | Change in inherited tables for newly created connections | For newly created connections, we treat inherited tables as separate entities. The base table only shows data that physically lives in the table (SELECT * FROM ONLY base_table). Learn how we sync child table data and base table data in our Partitioning using inheritance documentation. |
| September 27, 2024 | 3-month buffer period ends | Fivetran supports syncing inherited tables and treats them as separate entities. For all connections with inherited tables, the base table only shows data that physically lives in the table (SELECT * FROM ONLY base_table). Learn how we sync child table data and base table data in our Partitioning using inheritance documentation. |
If you have any questions or concerns, please reach out to Fivetran Support.
May 2024
Starting on June 15, 2024, newly created connections only display the root partitioned table in the Fivetran dashboard because we sync data from partitioned tables into the base table. Previously, we displayed both the root partitioned table and its child tables.
March 2024
Fivetran Teleport Sync for PostgreSQL now utilizes a hybrid of XMIN updates and Teleport deletes to reduce incremental update failures:
- Inserts and updates are captured using XMIN updates.
- The deletes are captured using a simplified Teleport query.
With this change, Teleport now processes some DDL events differently:
| DDL Statement | Before | After | Notes |
|---|---|---|---|
| CREATE TABLE | Yes | Yes | |
| CREATE SCHEMA | Yes | Yes | |
| ADD COLUMN | Yes | Yes | Rows synced before the ADD COLUMN operation now have null values in the newly-added destination column. Re-sync the table to update the existing rows with the default value. |
| ADD COLUMN ... DEFAULT | Yes | Partially | Rows synced before the ADD COLUMN operation now have null values in the newly-added destination column. Re-sync the table to update the existing rows with the default value. |
| RENAME TO new_table_name | No | No | |
| RENAME TO new_schema_name | No | No | |
| RENAME COLUMN | Yes | Yes | Rows synced before the ADD COLUMN operation now have null values in the newly-added destination column. Re-sync the table to update the existing rows with the default value. |
| DROP COLUMN | No | No | |
| ALTER COLUMN ... TYPE ... | Yes | Yes | We now support all non-primary key data type changes. For primary keys, we only support data type changes that don't affect the value. Unsupported primary key data type changes cause the subsequent sync to re-import the table. |
| ALTER COLUMN ... TYPE ... USING | Yes | Yes | We now support all non-primary key data type changes. For primary keys, we only support data type changes that don't affect the value. Unsupported primary key data type changes cause the subsequent sync to re-import the table. |
| ALTER COLUMN ... SET NOT NULL | No | No | |
| DROP SCHEMA | No | No | |
| DROP TABLE | No | No | |
| TRUNCATE TABLE | Yes | Yes |
Impact on MAR
- XMIN wraparounds result in a MAR overage. To avoid an overage, enable the
pageinspectandpg_visibilityXMIN extensions. Learn how in our XMIN wraparound causing excess MAR documentation. - Changes in excluded columns contribute towards MAR.
- Changes in a row's
xmincolumn are captured as a change and contribute to MAR. Learn more in our XMIN system column documentation. - MAR should decrease for tables with composite primary keys, because the previous Teleport update query synced more rows than the new query.
All connectors with Teleport enabled run setup tests to verify that XMIN extensions are enabled. You can see a list of your PostgreSQL connector's setup tests in the setup guide. If there is a risk of a MAR overage, you receive a warning to enable XMIN extensions.
History Mode
We detect any change to the XMIN value of a row and replicated that change in the destination.
February 2024
We now support PostgreSQL version 16. For more information, see our PostgreSQL documentation.
We now sync unknown data types to the destination as STRING.
November 2023
We have made the following changes to the Fivetran Teleport Sync update method:
- We no longer read for changes from columns that were selected and later deselected or update their
fivetran_synceddates - We no longer require a historical re-sync to bring in new data when you select new columns from the schema
August 2023
We can now sync tables with TEXT primary keys using Fivetran Teleport Sync. For more information, see our PostgreSQL documentation.
July 2023
We have added support for XML data type transformation. For more information about the supported data types, see our PostgreSQL documentation.
June 2023
We now support syncing the LTREE data type. See all supported data types in our PostgreSQL documentation.
May 2023
We can now sync tables with DATE primary keys using Fivetran Teleport Sync. For more information, see our PostgreSQL documentation.
March 2023
We now support PostgreSQL version 15. Read our PostgreSQL documentation for more information.
We are deprecating the logical replication with the test_decoding plugin update method. We are gradually migrating all users who use this method to logical replication with the pgoutput plugin. If you use the test_decoding method, we generate a warning message in your Fivetran dashboard that contains instructions for migrating to the pgoutput method. Follow the steps in that message to complete your migration. To learn about the differences between the pgoutput method and the test_decoding method, see our logical replication documentation.
We now support the TRUNCATE command for the pgoutput update method. If a table is truncated, the records prior to the TRUNCATE timestamp appear as _fivetran_deleted=true in the destination. If you are using history mode, the truncated table is deleted in your destination, just like a regular delete.
We now sync bounds information for DATERANGE, TSRANGE, and TSTZRANGE data types to the destination, where the values EXCLUSIVE and INCLUSIVE indicate the bounds.
February 2023
We added an additional layer of validation to proactively catch mismatches between PostgreSQL sources and Snowflake destinations, such as missing rows. We are gradually rolling out this new feature to all PostgreSQL connectors sending data to Snowflake.
January 2023
We now support Amazon Aurora Serverless PostgreSQL. Read our PostgreSQL documentation for more information.
We can now sync tables with constrained NUMERIC primary keys using Fivetran Teleport Sync. Learn more in our Fivetran Teleport Sync for PostgreSQL documentation.
December 2022
We no longer support PostgreSQL version 8. We only support PostgreSQL versions 9 - 14.x.
November 2022
We now sync incremental data from partitioned tables into the parent table. This change applies to new connectors created on or after November 8, 2022. For connectors created before November 8, 2022, we sync incremental data from partitioned tables into partitions. If you want to sync incremental data into the parent table, contact Fivetran Support.
October 2022
Fivetran Teleport Sync is now generally available. Learn more in our Fivetran Teleport Sync for PostgreSQL documentation.
June 2022
Fivetran Teleport Sync is now available in beta. Learn more in our Fivetran Teleport Sync for PostgreSQL documentation.
April 2022
The PostgreSQL connectors now only use soft deletes in the Soft Delete sync mode with all incremental update mechanisms. History mode doesn't use soft deletes regardless of your incremental update mechanism.
We fixed a bug that made our PostgreSQL connectors fail to parse arrays of complex types like POINT, TSRANGE, and TSTZRANGE for tables without primary keys. For these types, we now sync the raw values to the destination.
October 2021
We now support logical replication using the pgoutput plugin. The pgoutput plugin allows you to use PostgreSQL's publications feature. This feature only replicates change events from tables in your publication, so you control which changes are replicated to Fivetran. Learn more in our pgoutput plugin documentation.
July 2021
We now support logical replication in Google Cloud PostgreSQL. Learn more in about logical replication in our Updating data documentation.
January 2021
We now support domains, which are user-defined data types with optional constraints. Learn more in PostgreSQL's domains documentation.
We now support logical replication on Azure Database for PostgreSQL version 9.5 and later.
December 2020
We now enforce stricter checks on the replication slot's configuration during the setup tests. Ensure that a replication slot exists within the database you specify and turn on the "test_decoding" plugin.
We now support logical replication on Azure Database for PostgreSQL version 10 and later.
We have now made the GEOMETRY and GEOGRAPHY data types GeoJSON compliant.
October 2020
We now sync unparsable DATERANGE values (such as 10000-01-01) to the destination as null values.
August 2020
We have added support for PostgreSQL's email domain. We treat this domain as an extension of CITEXT data type. Learn how we map CITEXT data types to your destination in our PostgreSQL documentation.
July 2020
We now support logical replication on Aurora PostgreSQL versions 10.6 or later.
June 2020
We now replicate empty tables in a PostgreSQL source database as empty tables in the destination.
May 2020
We now support syncing the BYTEA data type from your source. In your destination, the BYTEA data type will appear as BINARY data type.
February 2020
We now support partitioned tables. You will be able to see both partitioned tables and their partitions in your Fivetran dashboard.
December 2019
We now create a task on your Alerts page when we encounter concurrent usage errors for your replication slot. Only one process can access the same replication slot at a time. For more details, see PostgreSQL's logical decoding documentation.
We can now use unique indexes to import tables without a primary key, making imports more efficient and reliable. This fix will be gradually rolled out over the next few weeks.
November 2019
We can now use unique key constraints to import tables without a primary key, making imports more efficient and reliable. This fix will be gradually rolled out over the next few weeks.
We now require that the PostgreSQL database configuration parameter statement_timeout is either 0 (the default value to disable the timeout) or is greater than 5 minute. Fivetran validates this value during the setup test.
October 2019
We can now sync single and multidimensional JSON, JSONB, and HSTORE arrays.
We now require that the PostgreSQL database configuration parameter 'wal_sender_timeout' is set to 0 or greater than 10 minutes. The value is validated during the setup test.
September 2019
If you have a logging service connected to your Fivetran account, the import progress of your tables in the event logs will be visible in your logs as an import_progress event. Table names will be in either the COMPLETE or IN_PROGRESS state, depending on their sync status. Tables that are not reported in the event have not started their import.
August 2019
We fixed a bug that caused connectors to break when syncing Postgis Geometry or Geography objects.
July 2019
We have fixed a bug in our PostgreSQL connectors which was preventing some syncs from making progress.
Previously, PostgreSQL connectors using WAL syncs could not make progress through the WAL if all of the log events were related to tables Fivetran did not sync. For example, if you selected to sync the table PUBLIC.MY_TABLE, and all log events were on PUBLIC.YOUR_TABLE, Fivetran could not make progress.
Now Fivetran clears the WAL slot to the most recent LSN we examined if no rows were added, deleted, or updated in Fivetran-delivered tables.
March 2019
Fivetran has added core support for the TIMESTAMP WITHOUT TIMEZONE data type. Up until now, Fivetran assumed that timestamps without timezone were in UTC. This addition means that for the following databases with a TIMESTAMP WITHOUT TIMEZONE type will replicate to a more correct destination type:
- MySQL
- Oracle
- PostgreSQL
- SQL Server
TIMESTAMP WITHOUT TIMEZONE support is immediately available for Oracle, PostgreSQL, and SQL Server. MySQL and MariaDB support will be released soon.
| Source | Applicable Type |
|---|---|
| Oracle | TIMESTAMP |
| PostgreSQL | TIMESTAMP |
| SQL Server | DATETIME, DATETIME2, SMALLDATETIME |
| MySQL and MariaDB | DATETIME |
This change is backwards compatible. Fivetran will continue assuming UTC time zone if the destination type is TIMESTAMP WITH TIMEZONE. In order to get the more specific type, drop the table in the destination and re-sync the table in your Fivetran dashboard.
We are changing our architecture to allow for much higher data throughput. Starting on April 30, 2019, all Fivetran connectors will originate from a new set of fixed IP addresses. For the US region, these IPs are:
- 35.227.135.0/29
- 35.234.176.144/29
- 52.0.2.4/32.
Update your IP safelists for the following databases and destinations before April 30. Not doing so will result in a connection failure.
Databases:
- MariaDB
- MongoDB
- MySQL
- Oracle
- PostgreSQL
- SQL Server
Destinations:
- Redshift
- Azure Synapse
- Snowflake
If you don't use any of the above databases or destinations with Fivetran, you don't need to take any action.
February 2019
TIMESTAMP column types will now be represented as TIMESTAMP WITHOUT TIME ZONE in the destination.
TSRANGE columns will be represented as JSON, with TIME WITHOUT TIME ZONE encoded as STRING.
November 2018
You can now trigger re-syncs of individual tables from your Fivetran dashboard:
The re-sync button appears when you hover your cursor over the row that contains the table name.
This feature is available for the following connectors:
- Amazon DynamoDB
- MongoDB
- MySQL
- NetSuite SuiteAnalytics
- Oracle
- PostgreSQL
- Salesforce
- SQL Server
- Zuora
October 2018
We have clarified the wording of the rescheduled sync notification message in your Sync History.
May 2018
For logical replication mode, we have switched to a different logical decoding plugin that consumes significantly less disk and memory resources on the database instance.
We can now correctly parse PostgreSQL array data types with escaped characters.
PostgreSQL on Azure Database is a managed relational cloud database service that can handle mission-critical workloads with predictable performance, security, high availability, and dynamic scalability.
See our Azure Database for PostgreSQL setup instructions.
PostgreSQL on Google Cloud SQL Database is a fully-managed database service that makes it easy to set up, maintain, manage, and administer your PostgreSQL relational databases on Google Cloud Platform.