Working with Environment Variables

It's much more convenient to use environment variables for parameters you would otherwise have to enter repeatedly into the command line. We recommend using an environment variable called FIVETRAN_API_KEY for your Fivetran base 64 encoded api key to avoid having to find and copy it every time you deploy your connector.

How to use environment variables

There are multiple ways to set environment variables:

Using python-dotenv
Through your terminal shell
Creating an .env file (easy to get started with)

If you choose to use an .env file, create it in the root directory of your connector project with the following format:

FIVETRAN_API_KEY=<your 64 bit encoded Fivetran API key>

Load your .env file for use in your terminal by running the following command:

export $(grep -v '^#' .env | xargs)

Check that your environment variables are now available for use by running the following command:

echo $FIVETRAN_API_KEY

You should use configuration files for any configuration used within your connector.py code so that it will be available when the code is deployed and running in Fivetran and stored securely within Fivetran.

.env files are not deployed to Fivetran with the rest of your code.

Once the local environment variable for your FIVETRAN_API_KEY is set, use the following command to deploy your connector.py and your API key will be automatically detected and used:

fivetran deploy --destination <DESTINATION_NAME> --connection <CONNECTION_NAME>

You can also use environment variables and .env files to set defaults for other parameters used in fivetran debug and fivetran deploy commands, this can help you test and deploy your connectors quickly as well as efficiently switch between different connections.

Supported environment variables for setting command parameter defaults are:

FIVETRAN_API_KEY=<your 64-bit encoded Fivetran API key>
FIVETRAN_DESTINATION_NAME=<your destination name>
FIVETRAN_CONNECTION_NAME=<your connection name>
FIVETRAN_CONFIGURATION=<your configuration file's absolute or relative path with its extension; for example, `config.json`, `~/Documents/config.json`>
FIVETRAN_PYTHON_VERSION=<the python version you will to use; for example, `3.11`>
FIVETRAN_HD_AGENT_ID=<your non default HD agent ID>

If you use them, make sure to include your .env files in your .gitignore file to ensure you don't accidentally check your FIVETRAN_API_KEY into your code repository.
To use a default set using an environment variable, do not include the parameter in your initial fivetran deploy command. Instead, press Enter when prompted with the default value.
You can use $My_ENV_VAR as an argument in the fivetran deploy and fivetran debug commands, and in follow-up prompts.

Referencing environment variables in your code

You can reference the following environment variables in your connector.py code using the os library.

FIVETRAN_CONNECTION_ID: This variable stores the unique identifier for the connection. It lets you interact with the Fivetran REST API during a sync. For example, you can retrieve or update the connector's configuration dynamically during runtime conditions. Learn more in Fivetran REST API documentation. The value of this variable is test_id when running the connector locally using fivetran debug.
FIVETRAN_GROUP_ID: This variable stores the unique identifier for the group. The value of this variable is test_group_id when running the connector locally using fivetran debug.
FIVETRAN_CONNECTION_NAME: This variable stores the name of the connection. The value of this variable is test_connection_name when running the connector locally using fivetran debug.
FIVETRAN_DEPLOYMENT_MODEL: This variable stores the deployment model of the connector. It lets you implement environment-specific logic. For example, you can use mock data when running locally, while using optimized settings for production deployments. This approach improves both development efficiency and production reliability. The supported values are as follows:
- managed_cloud: This value is returned for the SaaS Deployment model.
- hybrid_cloud: This value is returned for the Hybrid Deployment model.
- local_debug: This value is returned when running your connector.py locally using fivetran debug.

For example, to access the environment variable FIVETRAN_CONNECTION_ID, you can reference it in your code as follows:

import os
connection_id = os.getenv('FIVETRAN_CONNECTION_ID')
print(f"Connection ID: {connection_id}")