Deploying Connectors

Deploy your Connector SDK custom connector to Fivetran to create a connection or update an existing one. You can deploy from the Connector SDK command-line interface (CLI), automate deployments in a CI/CD pipeline, or create and upload a connector package using the Fivetran REST API.

The following sections describe the full connector deployment lifecycle:

• What to do before deploying a connector
• How to deploy a Connector SDK connector
• What to do after deploying a connector

Prerequisites

Make sure you have the following:

• A Fivetran account and a destination where the connection will deliver data.
• A Fivetran API key with permission to create or update the connection. The CLI expects a Base64-encoded value in the format {api_key}:{api_secret}.
• A supported Python version. If you do not explicitly select a version during deployment, the deployment uses the default version.

Production system resources

Once you deploy your custom connector to our production infrastructure, we run it on a 64-bit Linux (amd64) platform with 4 GB of RAM.

Our production infrastructure allows for some variation in resources available to your connections at any given moment, load balancing across multiple connections. Stream or batch source data instead of accumulating the complete dataset in memory.

Before you deploy your connector

Choose deployment model

Connector SDK supports both deployment models Fivetran offers:

• SaaS Deployment: Fivetran runs the connector code in an isolated, Fivetran-managed runtime in the cloud and region associated with the destination.
• Hybrid Deployment: Fivetran runs the connector using a Hybrid Deployment Agent in your environment. Hybrid Deployment requires an Enterprise or Business Critical plan.

When you deploy to a destination configured for Hybrid Deployment, the connection uses that destination's default agent. To use another available agent, pass its ID with --hybrid-deployment-agent-id.

The deployed Connector SDK runtime must be able to reach the source. Connector SDK supports the following network connection options:

• Direct access by safelisting Fivetran IP addresses
• SSH tunnel
• Reverse SSH tunnel
• VPN tunnel
• AWS PrivateLink
• Azure Private Link
• Google Cloud Private Service Connect
• Proxy Agent Beta

Some networking options require a specific Fivetran plan or coordination with Fivetran. Configure and validate networking before starting the first sync in production.

Prepare the connector project

See our Recommended Project Structure documentation for more information on the typical structure for your project and the recommended guidelines. You must run the deployment command from the connector project's root directory.

Prepare configuration values

Use a JSON configuration file to provide source credentials and runtime settings during the initial deployment. The file is commonly named configuration.json, but you can use another name.

Example:

{
  "api_key": "replace_with_source_api_key",
  "base_url": "https://api.example.com",
  "page_size": "100"
}

For configuration requirements, behavior, and how the configuration is resolved, see our configuration.json documentation.

Validate the connector before deployment

Do not use deployment as the first end-to-end test. Test your connector locally:

fivetran debug --configuration configuration.json

The local tester creates the following files in the project's files directory:

• warehouse.db: A DuckDB database that represents the data the connector sends to a destination.
• state.json: The state saved by the connector's checkpoint operations.

Before deployment, verify that:

• The connector can authenticate and retrieve data from the source.
• Tables, columns, data types, primary keys, and system-column behavior are correct. See our Data Extraction and Handling documentation for more information.
• The initial sync works when the connector receives an empty state object.
• A second run retrieves only the expected new or changed records.
• Deletes or full-refresh behavior work as intended.
• State advances only after data is safe to commit. See our State Management documentation for more information.
• The connector checkpoints at suitable batch or page boundaries.
• Source pagination has deterministic ordering and does not skip records with duplicate cursor values.
• Memory usage remains below the production limit. See our Connector Memory Management documentation for more information.
• Logs are useful and do not expose credentials or sensitive source data. See our Connector SDK Logs documentation for more information.

To test another initial sync, reset local state and destination data:

fivetran reset
fivetran debug --configuration configuration.json

Configure deployment settings

Before deploying your connector, gather the following deployment information.

Once you have these values, choose how to provide them to fivetran deploy:

• CLI flags: Pass values directly on the command line. Use this for one-off deployments or when values vary per run. This requires no prior setup.
• Environment variables: Export values in your shell or CI/CD system. Use this to avoid repeating values across runs and to keep secrets out of files.
• Deployment options file: Starting with fivetran-connector-sdk v3.0.0, you can also store values in a local file and organize settings for multiple connections into named profiles. Use this when you regularly deploy to more than one connection or environment. This file requires Python 3.11 or later.

Choose a naming strategy

The --naming option determines how source identifiers become destination identifiers:

• FIVETRAN: Applies Fivetran naming transformations. This is the default.
• SOURCE: Preserves source table and column names, including supported UTF-8 characters.

The naming strategy is applied only when the connection is created. A later redeployment cannot change it; the CLI ignores a new value and displays a warning.

Test locally with the same naming strategy you will use in production:

fivetran debug --naming SOURCE --configuration configuration.json
fivetran deploy --naming SOURCE <other_options>

The local tester stores the naming strategy in warehouse.db. Run fivetran reset before testing another strategy.

How to deploy a connector

Option 1: Deploy using the CLI

From the project root, run:

fivetran deploy --api-key <YOUR_BASE64_API_KEY> --destination <YOUR_DESTINATION_NAME> --connection <YOUR_CONNECTION_NAME> --configuration <PATH_TO_CONFIGURATION_JSON>

Example:

fivetran deploy --api-key $FIVETRAN_API_KEY --destination production_snowflake --connection custom_orders --configuration configuration.json --python 3.13

When deploying a new connection, fivetran deploy:

1. Packages the connector project.
2. Excludes configuration.json and destination.json (which fivetran run reads) from the code package.
3. Uploads the package and configuration values separately.
4. Creates the Connector SDK connection in the selected destination.
5. Configures the selected Python version, naming strategy, and deployment model.
6. Returns the connection ID and a link to the connection in the Fivetran dashboard.

For the full list of flags and examples, see the fivetran deploy reference.

If environment variables provide all required settings, run fivetran deploy with no additional flags. If you use a deployment options file, provide the API key separately through FIVETRAN_API_KEY or --api-key, and select a named profile with --profile <profile_name> unless your settings are in [default].

Option 2: Automate deployment with CI/CD

We recommend deploying from a centralized pipeline for production connectors.

A deployment pipeline should:

1. Check out a reviewed connector revision.
2. Set up a supported Python version.
3. Install Connector SDK and project dependencies.
4. Run automated tests and fivetran debug against controlled test data when practical.
5. Create a temporary configuration file from protected secrets, or omit configuration for code-only redeployments.
6. Deploy with --yes so an intentional update does not stop at the overwrite prompt.
7. Remove temporary secret files after deployment.
8. Record the deployed commit, SDK version, Python version, target connection, and deployment result.

Example deployment step:

- name: Deploy Connector SDK connection
  env:
    FIVETRAN_API_KEY: ${{ secrets.FIVETRAN_API_KEY }}
    FIVETRAN_DESTINATION_NAME: ${{ secrets.FIVETRAN_DESTINATION_NAME }}
    FIVETRAN_CONNECTION_NAME: ${{ secrets.FIVETRAN_CONNECTION_NAME }}
  run: fivetran deploy --python 3.13 --yes

If the pipeline creates configuration.json, ensure that it is temporary, excluded from build artifacts, and populated only from protected secrets. Remember that deploying non-empty configuration replaces the stored connection configuration.

See our Create Git CI/CD Deployment Pipeline documentation for an example implementation.

Option 3: Deploy using a package and the REST API

Use the REST API deployment route for programmatic workflows, multi-stage release pipelines, or Powered by Fivetran integrations where you need explicit control over package artifacts and connection creation as separate steps.

To create a Connector SDK connection through the REST API:

1. Create the connector ZIP with fivetran package. The command creates files/<project_name>.zip, validates the dependency file, respects .gitignore, and excludes configuration.json and destination.json.
2. Upload the ZIP using the Create Connector SDK Package endpoint. Each package can only be associated with one connection at a time.
3. Record the returned package_id.
4. Call the standard Create a Connection endpoint with the package_id in the connector configuration and the required credentials in secrets_list.
5. Configure and start the connection using the normal Connection resource endpoints.

After you deploy a connector

The new connection is paused by default. Deployment uploads the code but does not prove that the production runtime can retrieve and sync the complete source data. Your next step is to start and verify the initial sync in production.

Start and verify the initial sync

After deployment, start and verify the initial sync:

1. Open the connection using the URL returned by the CLI, or find it on the Connections page in the Fivetran dashboard.
2. Review the connection's configuration and schema settings.
3. Start the initial sync by clicking Start Initial Sync, clicking Sync Now, or enabling the connection.
4. Alternatively, use the Fivetran REST API to unpause or trigger the connection.
5. Monitor the Connector SDK logs and the connection status until the sync succeeds.
6. Validate destination tables, row counts, primary-key behavior, deletes, timestamps, and cursor boundaries against the source.
7. Run at least one subsequent incremental sync and confirm that the connector does not repeat or skip data.

Check production logs

You can access production logs in one of the following ways:

• The Connector SDK logs tab on the connection details page
• The CONNECTOR_SDK_LOG table delivered by the Fivetran Platform Connector
• A configured external logging service

Connector code logs are prefixed with sdk; Fivetran runtime and delivery logs are prefixed with platform.

Performance profiling

Performance profiling is automatically enabled for all connections, apart from connections hosted on Hybrid Deployment. You can access generated flamegraphs in your Fivetran dashboard. For more information, see our Production performance profiling documentation.

Next steps

• Redeploy an existing connection
• Troubleshoot deployment issues