Schema Migrations
Whenever we are introducing potentially breaking schema changes to an application connector, we will publish release notes with a table that breaks down the changes being made.
We may include the following types of schema changes in a migration of this type:
Schema change type | Description |
---|---|
New table | Adding a new table. |
New column | Adding a new column in a table. |
Rename table | Renaming a table. |
Rename column | Renaming a column. |
Deprecated table | Deprecating a table. |
Deprecated column | Deprecating a column. |
Type change | Changing a column’s data type. |
Primary key change | Changing the primary key columns of a table. |
Foreign key relationship change | Changing the foreign key relationships of a table. |
We highlight schema change types differently in the changelog table to help you identify potentially breaking schema changes:
Red - We highlight in red breaking schema changes, such as column and table renaming, primary key changes, and type changes.
Yellow - We highlight in yellow potentially breaking schema changes, such as column and table deprecations.
Green - We highlight in green non-breaking schema changes, such as the addition of new columns and tables.
Before applying the new schema, Fivetran does the following:
- Publishes in-depth release notes with a table that explain how the schema is changing.
- Creates an updated ERD.
- Sends an email notification to you indicating when Fivetran will apply the schema changes.
Schema migration approaches
To implement the changes, schema migrations follow one or a combination of the following approaches:
Schema migration approach | Description |
---|---|
Addition | Used to add new tables and columns. Additions do not introduce breaking schema changes. |
Rename | Used for renaming tables and columns to ensure old data is preserved. Fivetran does not delete the old data in the destination. Once the new schema is applied, we stop adding data into the old table or column. The newly named table or column starts receiving data. |
Deprecation | Used when a column or table is to no longer receive updates. When the new schema is applied, we stop adding new data into the deprecated table or column. |
Table rebuild | Used for changes to the primary key of a table and/or narrowing changes to the data type of a column. With this approach you may see a temporary table in your destination or notice a table is removed for a brief time before it is rebuilt during a re-sync. |
NOTE: Re-syncs are always free if they are a part of a schema migration.
Recommendations
If you receive a notification that a connector's schema is changing, there are several actions that you should take before Fivetran applies the new schema:
- Review the associated changelog and ERD. Consider whether the breaking changes affect your downstream queries, reports, or models.
- If necessary, update your workflows, scripts, and queries to prepare for the schema migration.