Error: Resolve Character Encoding Error in Source Data
Issue
Syncs fail with the following error:
Resolve Character Encoding Error in Source Data
This Fivetran error corresponds to the native PostgreSQL errors SQLSTATE 22021, character_not_in_repertoire, and SQLSTATE 22P05, untranslatable_character.
Environment
- Connector: PostgreSQL
- Source database encoding: Non-UTF-8, such as
EUC_JPorWIN1252
Resolution
To resolve this issue, contact your database administrator to identify and remove or replace the unmappable characters from the affected tables, then re-trigger the sync in Fivetran.
If you can't immediately update the source data, temporarily exclude the affected tables from your connection schema:
- In Fivetran, go to your PostgreSQL connection page.
- Select the Schema tab.
- Deselect the affected tables.
- Click Save changes.
Excluding the affected tables is a temporary workaround. Fix the underlying source data to restore full data replication.
Cause
Fivetran requests PostgreSQL source data in UTF-8 and relies on PostgreSQL to convert it from the source database encoding. This issue occurs when the source database uses a non-UTF-8 encoding, such as EUC_JP or WIN1252, and the source data contains characters that PostgreSQL can't convert to UTF-8. When this happens, PostgreSQL raises an error before returning the data.