Row Filtering Private Preview

Row filtering enables you to only sync rows that meet user-defined criteria based on a selected table, column, and operator.

Use cases

The primary use cases for row filtering are as follows:

Filtering historical data - For tables where you only want to sync recent data, you can define a filter to sync rows from a specific time period.
Syncing or excluding a specific ID value - If your tables are multitenant or you have a specific ID value you explicitly want to or don’t want to sync, you can use filters to select only relevant rows.
Syncing or excluding records containing a matching string - Similar to filtering by IDs, you can include or exclude rows where a column's string value matches a specified string.

How row filtering works

When a filter is set up, we modify the SELECT statements that run on the source database by appending a WHERE statement based on your filter. This ensures any SELECT queries filter the data based on your filter predicates.

Supported data types

Row filtering is supported for columns with the following data types:

Integer
Serial
Date and Datetime/Timestamp
String

Supported comparison operators

Row filtering supports the following comparison operators:

Equal To
Not Equal To
Greater Than (for Integers, Serial and Date and Timestamp only)
Less Than (for Integers, Serial and Date and Timestamp only)
Greater Than Or Equal To (for Integers, Serial and Date and Timestamp only)
Less Than Or Equal To (for Integers, Serial and Date and Timestamp only)
Contains (for String only)
Starts With (for String only)

Supported connectors

Fivetran supports row filtering for the following connectors:

Limitations

We support row filtering only for initial sync and re-syncs. For incremental syncs, we do not apply row filtering.
You can only create one filter per table.

Enabling row filter

For existing connectors

To create and apply a row filter, do the following:

On the Schema tab of the Connector Details page, hover over the table you want apply the row filter to.
NOTE: If a table you want to apply a filter to does not have columns of a supported type, No filterable data is displayed for the table.
Click Filter Data. The Create a new filter sidebar opens on the right side of the Connector Details page.
NOTE: If the table you want to apply the filter to does not have columns of supported types, the message No filterable data will be displayed in the table's row.
Select the Column you want to filter by.
Select the Operator. See the list of supported comparison operators.
Specify the Value as integer, date, or string depending on the column type.
Click Save and Apply. The Apply Row Filter popup opens, notifying you that the table re-sync is required to apply the filter.
Click Apply filter and re-sync. A table re-sync starts.

TIP: A filtered table has a Filtered badge.

Filtered-table

For newly created connectors

To create and apply a row filter, do the following:

On the Schema tab of the Connector Details page, hover over the table you want apply the row filter to.
NOTE: If a table you want to apply a filter to does not have columns of supported type, No filterable data is displayed for the table.
Click Filter Data. The Create a new filter sidebar opens on the right side of the Connector Details page.
NOTE: If the table you want to apply the filter to does not have columns of supported types, the message No filterable data will be displayed in the table's row.
Select the Column you want to filter by.
Select the Operator. See the list of supported comparison operators.
Specify the Value as integer, date(time), or string depending on the column type.
Click Save and Apply.
Click Save and Continue. You are navigated to the Schema Change Handling Strategy step of the setup guide. The filter is applied during the initial sync.

TIP: A filtered table has a Filtered badge.

Filtered-table

Editing row filter

To edit a row filter, do the following:

On the Schema tab of the Connector Details page, click Filtered in the relevant table's row. The Filter applied to this table box opens.
Click View filter. The Edit filter sidebar opens on the right side of the Connector Details page.
Click View or edit filter.
Select the Column you want to filter by.
Select the Operator. See the list of supported comparison operators.
Specify the Value as integer or date, depending on the column type.
Click Save and Apply. The Apply Row Filter popup opens, notifying you that the table re-sync is required to apply the filter.
Click Apply filter and re-sync. The table re-sync starts.

Deleting row filter

To delete a row filter, do the following:

On the Schema tab of the Connector Details page, click Filtered in the relevant table's row. The Filter applied to this table box opens.
Click View filter. The Edit filter sidebar opens on the right side of the Connector Details page.
Click Delete filter in the sidebar.
In the Delete data filter and re-sync popup, type DELETE and click DELETE. The table re-sync starts.