Logging

Logging enables observability and debugging for connector execution.

Required import

# For enabling Logs in your connector code
from fivetran_connector_sdk import Logging as log

Logging levels

The SDK supports Python-native log levels. Java-style log levels are backward compatibility with existing connectors.

Python log levels (recommended)

Level	Method	Purpose
DEBUG	`log.debug()`	Detailed debugging. Only emitted during `fivetran debug`; silent in production.
INFO	`log.info()`	General status updates.
WARNING	`log.warning()`	Potential non-critical issues.
ERROR	`log.error()`	Recoverable errors that do not stop the sync.
CRITICAL	`log.critical()`	Severe errors requiring immediate attention.

Java log levels

log.fine() and log.severe() are deprecated in favor of their Python-style equivalents. All existing connectors that use these methods continue to work without any modification. However, new code should use log.debug() in place of log.fine(), and log.error() or log.critical() in place of log.severe().

log.warning() and log.error() write to sync logs only. They don't create a dashboard alert or affect the sync's outcome. To report a non-critical issue or fail a sync with a custom message on the dashboard, use op.warning() or op.error() instead. See Error Handling for guidance on choosing between them.

Example

# Python-native log levels (recommended)
log.debug("Debugging data transformation.")
log.info("Connector started successfully.")
log.warning("Response time slower than expected.")
log.error("Failed to process record, skipping.")
log.error("Failed to process record, skipping.", e)  # Error can be logged using this parameter in error method.
log.critical("Failed to connect to data source.", e) # Critical error can be logged using this parameter in critical method.

# Java-style log levels — log.fine() and log.severe() are deprecated
log.fine("Debugging data transformation.")     # DEPRECATED — use log.debug()
log.info("Connector started successfully.")
log.warning("Response time slower than expected.")
log.severe("Failed to connect to data source.") # DEPRECATED — use log.error() or log.critical()

Logging methods except error() and critical() accept only one argument. Errors can be logged using a second argument with the error() or critical() log level.

Exception handling

Python Runtime errors implemented in connector.py file, when raised after deployment, appear in the dashboard as an error and trigger an alert.

Do not use exit() to terminate Python code because connector can become stuck. Instead, raise error or catch exception and display error using log.error() or log.critical(), which are then displayed in sync logs.

To fail a sync by throwing an exception, use any RuntimeError. This triggers an error in the dashboard with the title Python code throwing error.
```
raise RuntimeError(f"Value not expected. Value: {value}")
```
To handle an error, use try-catch and log it with log.error("log_message", e). This handles the error and continues processing the remaining code. Use this only if the exception doesn't impact your sync in a critical manner.
```
try:
    # Some Code
    pass
except Exception as e:
    log.error("Value not expected.", e)
```

Notes

INFO logs are limited to 1500/min. Any logs that exceed this rate are dropped.
Avoid excessive logging. For example, avoid placing sync log event for each record, as it can increase log volume and cause logs to be discarded due to rate limits. You can check weather example, which uses info and fine-level logging in update() method for reference.
DEBUG logs (and the deprecated FINE logs) are only generated when running the fivetran debug command. Leaving these calls in code is safe; they are silently suppressed in production.
log.fine() and log.severe() are deprecated but remain fully supported for backward compatibility — existing connectors using them continue to work without any changes. New code should use log.debug() in place of log.fine(), and log.error() or log.critical() in place of log.severe().
Python log levels (DEBUG, INFO, WARNING, ERROR, CRITICAL) and Java log levels (FINE, INFO, WARNING, SEVERE) are both stored as-is in the level column of the CONNECTOR_SDK_LOG table. Existing queries filtering on Java level strings continue to work without modification.
Python log levels are propagated to external logging services with their native severity equivalents.
To intentionally fail a sync or report a non-critical issue with a custom message on the dashboard, use op.error() or op.warning(). See Error Handling for guidance on choosing between them.

Accessing Connector SDK logs

You can access your Connector SDK connections' log events by:

Navigating to the Connector SDK logs tab of your connection details page in the Fivetran dashboard.
Using the Fivetran Platform Connector. Your connections' logs are available in the CONNECTOR_SDK_LOG table within the associated Fivetran destination.
Using external log services, previously configured for your destination.

Connector SDK logs provide in-depth event data, including timestamps, log levels, and messages. We prefix all Fivetran process logs with the platform identifier, and logs from the fivetran-connector-sdk library are prefixed with the sdk identifier.

Local tester logs flow

When you run fivetran debug or fivetran run, your connector runs in the same process as the command. The logs of your connector.py code and the logs of the local tester appear in one stream, in the order they happen:

Logs from your connector code are prefixed with ⚡ connector.
Logs from the SDK and the local tester are prefixed with ⚡ sdk.

Log example

Expand to see log example

A successful fivetran debug run of a connector with two tables:

21:21:06.161 INFO     ⚡ sdk       ℹ️ reading configuration from configuration.json found in project folder
21:21:08.223 INFO     ⚡ sdk       › debugging connector at: /Users/you/my_connector
21:21:08.224 INFO     ⚡ sdk       › running the connector
21:21:08.226 INFO     ⚡ sdk       ℹ️ destination schema path: /Users/you/my_connector/files/warehouse.db (schema=tester)
21:21:08.226 INFO     ⚡ sdk       previous state:
{}
21:21:08.254 INFO     ⚡ sdk       calling schema()
21:21:08.255 INFO     ⚡ sdk       schema change detected: tester.company
21:21:08.255 INFO     ⚡ sdk       schema change detected: tester.department
21:21:08.263 INFO     ⚡ sdk       enforcing a 4 GB memory limit for local testing. Connectors may be subject to memory limits in production as well
21:21:08.263 INFO     ⚡ sdk       calling update()
21:21:08.263 WARNING  ⚡ connector Example: Common Patterns For Connectors - Cursors - Multiple Tables With Cursors
21:21:08.263 INFO     ⚡ connector Upserting company
21:21:08.277 INFO     ⚡ sdk       table created: tester.company
21:21:08.296 INFO     ⚡ sdk       checkpoint recorded: {"company_cursor": "2024-08-14T01:00:00Z"}
21:21:08.296 INFO     ⚡ connector Upserting department
21:21:08.300 INFO     ⚡ sdk       table created: tester.department
21:21:08.312 INFO     ⚡ sdk       checkpoint recorded: {"company_cursor": "2024-08-14T01:00:00Z", "department_cursor": {"1": "2024-08-14T01:00:00Z"}}
21:21:08.312 INFO     ⚡ sdk       Final checkpoint: committing any remaining operations
21:21:08.316 INFO     ⚡ sdk       SYNC SUCCEEDED — total elapsed 00:00:00.090
Operation       | Counts
----------------+------------
Upserts         | 2
Updates         | 0
Deletes         | 0
Truncates       | 0
Schema changes  | 2
Checkpoints     | 2
File uploads    | 0

21:21:08.316 INFO     ⚡ sdk       debug results in duckdb view using `duckdb -ui files/warehouse.db`
21:21:08.317 INFO     ⚡ sdk       connector process memory usage: average 0.70 MB, peak 0.70 MB

A failed run: the connector's code raises a KeyError in update().

21:24:16.719 INFO     ⚡ sdk       calling update()
21:24:16.719 INFO     ⚡ connector Fetching the company
21:24:16.745 SEVERE   ⚡ sdk       SYNC FAILED — elapsed 00:00:00.073
Operation       | Counts
----------------+------------
Upserts         | 0
Updates         | 0
Deletes         | 0
Truncates       | 0
Schema changes  | 0
Checkpoints     | 0
File uploads    | 0


Error: failed while executing update() error: 'name'
Traceback (most recent call last):
  File ".../site-packages/fivetran_connector_sdk_cli/connector.py", line 364, in run_update
    for _ in result:
             ^^^^^^
  File "/Users/you/my_connector/connector.py", line 6, in update
    yield op.upsert("company", {"id": 1, "name": company["name"]})
                                                 ~~~~~~~^^^^^^^^
KeyError: the message is above


See logs above for details.
21:24:16.745 INFO     ⚡ sdk       connector process memory usage: average 0.12 MB, peak 0.12 MB
crash report saved to /Users/you/my_connector/crash-reports/crash_20261023-212416.log