Managing Project Dependencies

Usually, your connector's code will need to import additional Python libraries. To ensure the correct libraries are installed before we run your code in Fivetran, you must include a dependency file in the root directory of your connector project. The Connector SDK supports two formats:

• requirements.txt — a standard pip plain-text file listing packages one per line.
• pyproject.toml — a modern Python packaging standard (PEP 517/518) that declares dependencies in a structured TOML file.

You can use either format. If both files are present in the project root, pyproject.toml takes precedence. You can also install dependencies from a private repository or authenticated package index. Some authentication methods work with either format, while others require pyproject.toml.

Using requirements.txt

In your requirements.txt file, you must list all the Python libraries your connector uses. You can also define the required versions of the libraries used. The configuration example provides an example.

You can use PyPI and Git sources to install packages. For example:

pandas
git+https://github.com/oracle/python-oracledb.git@main#egg=oracledb

You can also quickly install all the needed libraries when setting up your environment using the following command:

pip install -r requirements.txt

Using pyproject.toml

pyproject.toml is the modern Python standard for declaring project metadata and dependencies. You can use it as an alternative to requirements.txt by defining your dependencies in the [project] section under the dependencies key.

A minimal pyproject.toml for a Connector SDK project looks like this:

[project]
name = "my-connector"
version = "0.1.0"
dependencies = [
    "pandas",
    "git+https://github.com/oracle/python-oracledb.git@main#egg=oracledb",
]

You can pin specific versions using standard version specifiers:

[project]
name = "my-connector"
version = "0.1.0"
dependencies = [
    "cryptography==44.0.2",
    "tenacity>=8.2",
]

Private repositories

If your connector depends on packages hosted in a private PyPI repository, corporate artifact registry, or authenticated package index, you can configure your connector to install such packages by setting specific environment variables in your configuration.json file. The Connector SDK automatically passes all configuration fields as environment variables to the package installation process.

Option 1: Public alternative PyPI

Use this approach for public PyPI mirrors or alternative indexes that don't require authentication. You can use either requirements.txt or pyproject.toml for this.

configuration.json:

{
  "UV_INDEX": "https://pypi.tuna.tsinghua.edu.cn/simple/ https://pypi.org/simple",
  "UV_INDEX_STRATEGY": "first-match"
}

requirements.txt:

pandas==2.2.3
numpy==2.2.1
httpx==0.28.1

How it works:

• UV_INDEX - Space-separated list of package indexes (checked in order)
• UV_INDEX_STRATEGY: "first-match" - Uses the first index where the package is found

Option 2: Private PyPI using inline credentials

Use this approach for private PyPI repositories that require username/password authentication. You can use either requirements.txt or pyproject.toml for this.

configuration.json:

{
  "UV_INDEX": "https://username:password@private-pypi.company.com/simple/ https://pypi.org/simple",
  "UV_INDEX_STRATEGY": "first-match"
}

Option 3: Private PyPI using named index credentials

Use this approach to keep credentials separate from the index URL. This method requires a pyproject.toml with a named index definition. The advantage over inline credentials is that the URL contains no embedded secrets.

1. Define the named index in pyproject.toml:

   [[tool.uv.index]]
   name = "my-private"
   url = "https://private-pypi.company.com/simple/"
   

2. Add the credentials to configuration.json using the index name in uppercase:

   {
     "UV_INDEX_MY_PRIVATE_USERNAME": "myuser",
     "UV_INDEX_MY_PRIVATE_PASSWORD": "mypassword",
     "UV_INDEX_STRATEGY": "first-match"
   }
   

   {NAME} in UV_INDEX_{NAME}_USERNAME and UV_INDEX_{NAME}_PASSWORD is the uppercase version of the name field in [[tool.uv.index]]. In this example, name = "my private" becomes MY_PRIVATE.

Cloud-based artifact registries

Most cloud providers follow the same pattern. Here's an example for AWS CodeArtifact:

{
  "UV_INDEX": "https://aws:${CODEARTIFACT_TOKEN}@domain-123.d.codeartifact.us-east-1.amazonaws.com/pypi/repo/simple/ https://pypi.org/simple",
  "UV_INDEX_STRATEGY": "first-match"
}

Other supported providers are:

• Azure Artifacts
• JFrog Artifactory
• GitHub Packages
• Nexus Repository
• Any PyPI-compatible package index

Follow the same pattern with your provider's URL and authentication method.

Supported environment variables

Choose one authentication method and use only its variables. Do not mix inline URL credentials with named index credential variables.

Inline credentials method

Named index credentials method

Verification

To verify if packages install correctly from your private PyPI, check your connector logs during sync:

platform Using UV_INDEX: https://****@private-pypi.company.com/simple/
DEBUG Sending fresh GET request for: https://private-pypi.company.com/simple/package-name/

Credentials are automatically masked with **** for security.