Managed Data Lake Service API Configuration

Request

POST https://api.fivetran.com/v1/destinations

{
  "group_id": "group_id",
  "service": "managed_data_lake",
  "region": "GCP_US_WEST1",
  "time_zone_offset": "+3",
  "trust_certificates": true,
  "trust_fingerprints": true,
  "run_setup_tests": true,
  "daylight_saving_time_enabled": true,
  "hybrid_deployment_agent_id": "hybrid_deployment_agent_id",
  "private_link_id": "private_link_id",
  "networking_method": "Directly",
  "proxy_agent_id": "proxy_agent_id",
  "config": {
    "tenant_id": "service_principal_tenant_id",
    "auth_type": "PERSONAL_ACCESS_TOKEN | OAUTH2",
    "should_maintain_tables_in_glue": true,
    "connection_type": "Directly",
    "storage_account_name": "adls_storage_account_name",
    "catalog": "string",
    "should_maintain_tables_in_databricks": true,
    "http_path": "string",
    "oauth2_secret": "string",
    "fivetran_role_arn": "iam_role_arn",
    "snapshot_retention_period": "ONE_WEEK",
    "server_host_name": "string",
    "client_id": "service_principal_client_id",
    "prefix_path": "s3_bucket_path_prefix",
    "bucket": "s3_bucket_name",
    "container_name": "adls_container_name",
    "port": 0,
    "databricks_connection_type": "Directly | PrivateLink | SshTunnel | ProxyAgent",
    "secret_value": "service_principal_secret_value",
    "storage_provider": "AWS",
    "region": "s3_bucket_region",
    "oauth2_client_id": "string",
    "personal_access_token": "string"
  }
}

Config parameters

Name	Description
`tenant_id`	Tenant ID of the service principal you created for Fivetran. Use this parameter only if you want to deploy your data lake on ADLS.
`auth_type`	Specifies the authentication types for Fivetran to connect to Databricks. Use this parameter only if you want to use Unity Catalog to manage your Delta Lake tables.
`should_maintain_tables_in_glue`	Specifies whether you want to manage your Iceberg tables in AWS Glue. Use this parameter only if you want to deploy your data lake on AWS.
`connection_type`	connection type
`storage_account_name`	(Immutable) Name of your Azure Data Lake Storage Gen2 (ADLS Gen2) storage account. Use this parameter only if you want to deploy your data lake on ADLS.
`catalog`	Name of the specific catalog in Unity Catalog that you want to use to manage your Delta Lake tables. Use this parameter only if you want to use Unity Catalog with your data lake.
`should_maintain_tables_in_databricks`	Specifies whether you want to manage your Delta Lake tables in Databricks Unity Catalog.
`http_path`	HTTP path of the Databricks account associated with Unity Catalog. Use this parameter only if you want to use Unity Catalog to manage your Delta Lake tables.
`oauth2_secret`	OAuth 2.0 secret you created for authenticating Fivetran. Use this parameter only if you want to use OAuth 2.0 as the authentication type for Fivetran to connect to Databricks.
`fivetran_role_arn`	(Immutable) ARN of the IAM role you created for the IAM policy associated with your S3 bucket or Glue catalog. Use this parameter only if you want to deploy your data lake on AWS.
`snapshot_retention_period`	Specifies how long you want us to retain your table snapshots. We delete the snapshots that are older than the retention period during our table maintenance operations. Default value: `ONE_WEEK`.
`server_host_name`	Server host name of the Databricks account associated with Unity Catalog. Use this parameter only if you want to use Unity Catalog to manage your Delta Lake tables.
`client_id`	Client ID of the service principal you created for Fivetran. Use this parameter only if you want to deploy your data lake on ADLS.
`prefix_path`	(Immutable) Prefix path or virtual folder path within your bucket or container where you want Fivetran to store your data. Skip this parameter for AWS-based data lakes if the IAM policy configured for your S3 bucket grants access to the entire bucket.
`bucket`	(Immutable) Name of the S3 or Google Cloud Storage (GCS) bucket you want to use to store your data. Use this parameter only if you want to deploy your data lake on Amazon Web Services (AWS) or GCS.
`container_name`	(Immutable) Name of the ADLS container you want to use to store your data. Use this parameter only if you want to deploy your data lake on ADLS.
`port`	Server port number of the Databricks account associated with Unity Catalog. Use this parameter only if you want to use Unity Catalog to manage your Delta Lake tables.
`databricks_connection_type`	Specifies how you want Fivetran to connect to the Databricks account associated with Unity Catalog. Default value: `Directly`. Use this parameter only if you want to use Unity Catalog with your data lake.
`secret_value`	Secret value of the service principal you created for Fivetran. Use this parameter only if you want to deploy your data lake on ADLS.
`storage_provider`	The cloud storage provider you want to use for your data lake.
`region`	(Immutable) AWS Region of your S3 bucket. Use this parameter only if you want to deploy your data lake on AWS.
`oauth2_client_id`	The OAuth 2.0 client ID you created for authenticating Fivetran. Use this parameter only if you want to use OAuth 2.0 as the authentication type for Fivetran to connect to Databricks.
`personal_access_token`	The personal access token you created for authenticating Fivetran. Use this parameter only if you want to use a personal access token as the authentication type for Fivetran to connect to Databricks.