Edit on GitHub
Replication of PostgreSQL to BiqQuery Taking a Long Time To Load to the Destinationlink
Issuelink
Why is the replication of PostgreSQL to Google BiqQuery is taking a long time to load to the destination regardless of the dataset size?
Environmentlink
- Connector: Google Cloud PostgreSQL
- Destination: BigQuery
Resolutionlink
To avoid delays, follow the two guides below:
- Split up the connector into multiple smaller connectors.
- Deselect tables from the sync that are not a priority
Causelink
Google Cloud PostgreSQL does not support the logical replication of the WAL. Fivetran, therefore, uses the XMIN sync strategy. This results in Fivetran performing full table scans each sync to identify new or updated records.