How Does Fivetran Handle Character Conversion From Shift-JIS to UTF-8?

Question

How does Fivetran handle character conversion from Shift-JIS to UTF-8 when syncing data from SQL Server?

Environment

Connector: SQL Server

Answer

Fivetran uses the Microsoft ODBC/JDBC driver for SQL Server to perform character encoding conversion. Characters in the standard Shift-JIS to Unicode mapping are converted as expected. Fivetran does not perform proprietary or extra mapping.

User-defined characters, such as gaiji or EUDC characters, that do not have a Unicode equivalent are replaced with the Unicode replacement character (U+FFFD, '�'). This replacement is performed by the driver, not by Fivetran.

All successfully converted characters are stored as UTF-8 text in the destination. To preserve custom characters that are not representable in Unicode, you must convert or preserve them before Fivetran syncs the data. This may require custom ETL or application-level preprocessing.