Enterprise Patterns

This guide covers patterns for operating the Fivetran Terraform provider at scale: reusable modules, workspace organization, mixed Terraform/UI ownership, and state import workflows.

Reusable connection module

Defining a group, destination, connection, schema config, and pause state as a module lets teams instantiate a full pipeline with consistent defaults:

# modules/fivetran-connection/main.tf

variable "group_name"          { type = string }
variable "service"             { type = string }
variable "connection_config"   { type = map(string) }
variable "sync_frequency" {
  type    = number
  default = 360
}
variable "destination_service" { type = string }
variable "destination_config"  { type = map(string) }

resource "fivetran_group" "group" {
  name = var.group_name
}

resource "fivetran_destination" "destination" {
  group_id         = fivetran_group.group.id
  service          = var.destination_service
  time_zone_offset = "0"
  run_setup_tests  = true

  config {
    # These fields vary by destination service.
    project_id        = var.destination_config["project_id"]
    data_set_location = var.destination_config["data_set_location"]
  }
}

resource "fivetran_connection_v2" "connection" {
  group_id                 = fivetran_group.group.id
  service                  = var.service
  destination_schema_names = "FIVETRAN_NAMING"
  sync_frequency           = var.sync_frequency
  run_setup_tests          = true
  config                   = var.connection_config

  depends_on = [fivetran_destination.destination]
}

resource "fivetran_connector_schema_config" "schema_config" {
  connector_id           = fivetran_connection_v2.connection.id
  schema_change_handling = "ALLOW_ALL"
}

resource "fivetran_connection_v2_pause_state" "pause_state" {
  connection_id = fivetran_connection_v2.connection.id
  paused        = false

  depends_on = [fivetran_connector_schema_config.schema_config]
}

output "connection_id"  { value = fivetran_connection_v2.connection.id }
output "group_id"       { value = fivetran_group.group.id }
output "destination_id" { value = fivetran_destination.destination.id }

Call the module from your root configuration:

module "salesforce_pipeline" {
  source = "./modules/fivetran-connection"

  group_name         = "marketing-production"
  service            = "salesforce"
  connection_config  = {
    user     = var.salesforce_user
    password = var.salesforce_password
  }
  destination_service = "big_query"
  destination_config  = {
    project_id        = var.bq_project_id
    data_set_location = "US"
  }
}

Workspace structure

Organize Terraform workspaces to match team ownership boundaries. A common pattern separates infrastructure tiers:

terraform/
├── platform/           # shared infrastructure: groups, destinations, networking
│   ├── main.tf
│   └── outputs.tf
├── teams/
│   ├── marketing/      # marketing team's connections
│   ├── finance/
│   └── engineering/
└── access/             # teams, users, and RBAC grants (separate lifecycle)
    ├── teams.tf
    └── memberships.tf

Why separate access from pipeline resources? Access changes (adding a team member, adjusting a role) happen much more frequently than infrastructure changes and should go through a lightweight review cycle. Keeping them in a separate workspace prevents unrelated plan outputs from appearing in access-only PRs.

Why separate platform from teams? Platform resources (groups, destinations) are owned by a central team and should not be modified by individual squads. Using remote state references lets teams consume group IDs and destination IDs as read-only inputs:

# teams/marketing/main.tf

data "terraform_remote_state" "platform" {
  backend = "s3"
  config = {
    bucket = "fivetran-terraform-state"
    key    = "platform/terraform.tfstate"
    region = "us-east-1"
  }
}

resource "fivetran_connection_v2" "hubspot" {
  group_id                 = data.terraform_remote_state.platform.outputs.marketing_group_id
  service                  = "hubspot"
  destination_schema_names = "FIVETRAN_NAMING"
  sync_frequency           = 360
  # ...
}

Partial ownership: Terraform alongside UI-managed resources

Not every resource needs to be Terraform-managed. A common pattern is to manage infrastructure (groups, destinations) in Terraform while allowing individual connection setup via the Fivetran dashboard.

What Terraform controls:

• Groups and destinations
• Connection sync frequency and schedule
• Schema change handling policy
• Certificates and networking

What stays in the UI:

• Per-connection credential setup (especially OAuth flows requiring browser interaction)
• One-off schema table inclusions/exclusions
• Emergency pause/resume operations

To reference a UI-managed connection in Terraform without taking full ownership, use the fivetran_connector data source with the connection's ID:

data "fivetran_connector" "ui_managed" {
  id = "your_connection_id"
}

Import a dashboard-created connection

If you later want Terraform to manage a connection that was created in the dashboard, import that existing connection into Terraform state instead of recreating it.

1. Add a matching resource definition to your Terraform configuration:

   resource "fivetran_connection_v2" "imported_connection" {
     group_id                 = fivetran_group.group.id
     service                  = "salesforce"
     destination_schema_names = "FIVETRAN_NAMING"
     sync_frequency           = 360
   
     config = {
       # Add the same config values that exist in the dashboard.
     }
   }
   

2. Import the existing connection with its Fivetran connection ID:

   terraform import fivetran_connection_v2.imported_connection <connection_id>
   

3. Import the pause-state resource if you also want Terraform to manage pause and resume:

   terraform import fivetran_connection_v2_pause_state.imported_connection <connection_id>
   

   Import fivetran_connection_v2 first. Importing fivetran_connection_v2_pause_state before the connection exists in Terraform state forces a replacement plan.

4. Reconcile the plan until terraform plan shows only the changes you expect.

What cannot be fully automated

Some Fivetran operations require manual steps and cannot be driven entirely by Terraform:

Importing existing infrastructure

If your Fivetran account has resources created before Terraform adoption, import them rather than recreating them.

Write the resource definition

Write the resource definition matching what you expect the existing resource to look like.

Import the resource

Import the resource using its Fivetran ID:

# Get the group ID from the Fivetran dashboard URL
terraform import fivetran_group.my_group <group_id>

# Get the destination group ID (same as group ID for destinations)
terraform import fivetran_destination.my_destination <group_id>

# Import an existing connection. This ID is different from the group ID.
terraform import fivetran_connection_v2.my_connection <connection_id>

Reconcile the plan

Run terraform plan and adjust your resource definition until there are no unexpected differences. Pay particular attention to:

• Fields that Terraform tracks but the import didn't capture (for example, run_setup_tests - the API doesn't return this, so Terraform will show it as a diff)
• Computed fields that Terraform sets as null initially
• auth secrets, which the API does not return - you must add these to your configuration manually after import

Commit the reconciled Terraform configuration

Commit the reconciled Terraform configuration once terraform plan shows no changes.

CI/CD integration

A typical GitHub Actions CI/CD pipeline for Fivetran Terraform changes:

# .github/workflows/terraform.yml (GitHub Actions example)
on:
  pull_request:
    paths: ['terraform/**']

jobs:
  plan:
    steps:
      - uses: hashicorp/setup-terraform@v3
      - run: terraform init
        working-directory: terraform/
      - run: terraform plan -out=tfplan
        working-directory: terraform/
        env:
          FIVETRAN_APIKEY: ${{ secrets.FIVETRAN_APIKEY }}
          FIVETRAN_APISECRET: ${{ secrets.FIVETRAN_APISECRET }}
      - name: Post plan to PR
        uses: actions/github-script@v7
        # ... post plan output as PR comment

  apply:
    if: github.ref == 'refs/heads/main'
    needs: plan
    steps:
      - run: terraform apply tfplan
        working-directory: terraform/

Store API credentials as CI/CD secrets, never in .tf files or committed terraform.tfvars files.

Drift management

Fivetran state can drift from Terraform when changes are made in the dashboard or via the API outside of Terraform. To detect drift:

terraform plan -refresh-only

This shows what has changed in Fivetran without proposing any infrastructure changes. Review the output and either:

• Accept the drift by running terraform apply -refresh-only to update state
• Revert the drift by running terraform apply to restore the Terraform-defined state

For automated drift detection, run terraform plan -refresh-only on a schedule in CI/CD and alert on non-empty output.

Related guides