Working with Environment Variables

It's much more convenient to use environment variables for parameters you would otherwise have to enter repeatedly into the command line. We recommend using environment variables for the required deployment values so you do not have to repeat them every time you deploy your connector.

How to use environment variables

There are multiple ways to set environment variables:

• Using python-dotenv
• Through your terminal shell
• Creating an .env file (easy to get started with)

If you choose to use an .env file, create it in the root directory of your connector project with the following format:

FIVETRAN_API_KEY=<your 64 bit encoded Fivetran API key>

Load your .env file for use in your terminal by running the following command:

export $(grep -v '^#' .env | xargs)

Check that your environment variables are now available for use by running the following command:

echo $FIVETRAN_API_KEY

You should use configuration files for any configuration used within your connector.py code so that it will be available when the code is deployed and running in Fivetran and stored securely within Fivetran.

For fivetran deploy values, the CLI resolves each field in the following order:

1. Explicit flag value
2. Matching environment variable
3. The local options file
4. The package-level default in pyproject.toml, under the [tool.package-metadata] section, for python_version and proxy_host_config_key only
5. The field's SDK built-in default, if one exists; otherwise, if the field is required, the deploy fails with a clear error message

See Field resolution order for the full table.

Once the local environment variables are set, use the following command to deploy your connector.py:

fivetran deploy

You can also use environment variables and .env files to set defaults for other parameters used in fivetran debug, fivetran run, and fivetran deploy. This helps you test and deploy connectors and switch between connections more efficiently.

Supported environment variables for setting command parameter defaults are:

FIVETRAN_API_KEY=<your 64-bit encoded Fivetran API key>
FIVETRAN_DESTINATION_NAME=<your destination name>
FIVETRAN_CONNECTION_NAME=<your connection name>
FIVETRAN_CONFIGURATION=<your configuration file's absolute or relative path with its extension; for example, `config.json`, `~/Documents/config.json`>
FIVETRAN_STATE=<your state file's absolute or relative path with its extension; for example, `state.json`. Used by `fivetran debug` and `fivetran run`; the checkpoints update this file>
FIVETRAN_PYTHON_VERSION=<the Python version you want to use; for example, `3.11`>
FIVETRAN_HD_AGENT_ID=<your non default HD agent ID>
FIVETRAN_NAMING=<naming strategy for the connection schema; either `FIVETRAN` (default) or `SOURCE`. See naming conventions: https://fivetran.com/docs/core-concepts/syncoverview/naming-conventions>
FIVETRAN_OPTIONS_FILE=<absolute or relative path to your options.toml file>
DO_NOT_TRACK=<set to `1` or `true` to disable CLI usage telemetry (see https://donottrack.sh/). Overrides the `is_telemetry_enabled` setting in the CLI configuration file.>

Referencing environment variables in your code

You can reference the following environment variables in your connector.py code using the os library.

• FIVETRAN_CONNECTION_ID: This variable stores the unique identifier for the connection. It lets you interact with the Fivetran REST API during a sync. For example, you can retrieve or update the connector's configuration dynamically during runtime conditions. Learn more in Fivetran REST API documentation. The value of this variable is test_id when running the connector locally using fivetran debug.
• FIVETRAN_GROUP_ID: This variable stores the unique identifier for the group. The value of this variable is test_group_id when running the connector locally using fivetran debug.
• FIVETRAN_CONNECTION_NAME: This variable stores the name of the connection. The value of this variable is test_connection_name when running the connector locally using fivetran debug.
• FIVETRAN_DEPLOYMENT_MODEL: This variable stores the deployment model of the connector. It lets you implement environment-specific logic. For example, you can use mock data when running locally, while using optimized settings for production deployments. This approach improves both development efficiency and production reliability. The supported values are as follows:
  • managed_cloud: This value is returned for the SaaS Deployment model.
  • hybrid_cloud: This value is returned for the Hybrid Deployment model.
  • local_debug: This value is returned when running your connector.py locally using fivetran debug.

For example, to access the environment variable FIVETRAN_CONNECTION_ID, you can reference it in your code as follows:

import os
connection_id = os.getenv('FIVETRAN_CONNECTION_ID')
print(f"Connection ID: {connection_id}")