Connector Management
REST API supports the following connector management actions:
- Create a connector
- Retrieve connector details
- Modify a connector
- Sync connector data
- Re-sync connector table data
- Run connector setup tests
- Delete a connector
- Retrieve a connector schema config
- Retrieve source table columns config
- Reload a connector schema config
- Modify a connector schema config
- Modify a connector database schema config
- Modify a connector table config
- Modify a connector column config
- Use Case: Create a new connector based on an existing connector's parameters
- Use Case: How to enable sync only for columns that you need
Create a Connector
Creates a new connector within a specified group in your Fivetran account. Runs setup tests and returns testing results.
Request
POST api.fivetran.com/v1/connectors
{
"service": "criteo",
"group_id": "projected_sickle",
"trust_certificates": true,
"run_setup_tests": true,
"paused": true,
"config": {
"schema": "criteo",
"username": "myuser",
"password": "mypassword",
"app_token": "myapptoken"
},
"auth": {
"client_access": {
"client_id": "my_client_id",
"client_secret": "my_client_secret"
},
"refresh_token": "my_refresh_token"
}
}
Payload parameters
| Name | Description |
|---|---|
group_id (required) |
The unique identifier for the group within the Fivetran system. Find you group_id by fetching your account's group list. |
service (required) |
The name for the connector type within the Fivetran system. |
config (required) |
The connector setup configuration. The format is specific for each connector. |
auth |
The connector authorization settings. Can be used to authorize a connector using your external client credentials. The format is specific for each connector. |
paused |
The boolean specifying whether the connector should be paused. |
trust_certificates |
Specifies whether we should trust the certificate automatically. Applicable only for database connectors. The default value is FALSE. If a certificate is not trusted automatically, it has to be approved with Certificates Management API. |
run_setup_tests |
Specifies whether the setup tests should be run automatically. The default value is TRUE. |
Response
HTTP 201 Created
{
"code": "Success",
"message": "Connector has been created",
"data": {
"id": "speak_inexpensive",
"group_id": "projected_sickle",
"service": "criteo",
"service_version": 0,
"schema": "criteo",
"connected_by": "interment_burdensome",
"created_at": "2018-12-01T15:43:29.013729Z",
"succeeded_at": null,
"failed_at": null,
"sync_frequency":60,
"status": {
"setup_state": "incomplete",
"sync_state": "scheduled",
"update_state": "on_schedule",
"is_historical_sync": true,
"tasks": [],
"warnings": []
},
"setup_tests": [{
"title": "Validate Login",
"status": "FAILED",
"message": "Invalid login credentials"
}],
"config": {
"username": "myuser",
"password": "******",
"api_token": "******",
"service_version": "0"
}
}
}
If run_setup_tests is set to FALSE in the request, the response will not contain the setup_tests field.
See how to create a new connector based on an existing connector's parameters.
Retrieve Connector Details
Returns a connector object if a valid identifier was provided.
Request
GET api.fivetran.com/v1/connectors/{connector_id}
Path parameters
| Name | Description |
|---|---|
connector_id (required) |
The unique identifier for the connector within the Fivetran system. |
Response
HTTP 200 OK
{
"code": "Success",
"data": {
"id": "speak_inexpensive",
"group_id": "projected_sickle",
"service": "criteo",
"service_version": 0,
"schema": "criteo",
"connected_by": "interment_burdensome",
"created_at": "2018-12-01T15:43:29.013729Z",
"succeeded_at": null,
"failed_at": null,
"sync_frequency": 60,
"status": {
"setup_state": "incomplete",
"sync_state": "scheduled",
"update_state": "delayed",
"is_historical_sync": true,
"tasks": [],
"warnings": []
},
"config": {
"username": "myuser",
"password": "******",
"api_token": "******",
"service_version": "0"
}
}
}
Fields
| Name | Description |
|---|---|
id |
The unique identifier for the connector within the Fivetran system. |
group_id |
The unique identifier for the group within the Fivetran system. |
service |
The connector type name within the Fivetran system. |
service_version |
The connector type version within the Fivetran system. |
schema |
The name used both as the connector's name within the Fivetran system and as the source schema's name within your destination. |
connected_by |
The unique identifier of the user who has created the connector in your account. |
created_at |
The timestamp of the time the connector was created in your account. |
succeeded_at |
The timestamp of the time the connector sync succeeded last time. |
failed_at |
The timestamp of the time the connector sync failed last time. |
sync_frequency |
The connector sync frequency in minutes. |
status.setup_state |
The current setup state of the connector. The available values are:incomplete - the setup config is incomplete, the setup tests never succeeded;connected - the connector is properly set up;broken - the connector setup config is broken. |
status.sync_state |
The current sync state of the connector. The available values are:scheduled - the sync is waiting to be run;syncing - the sync is currently running;paused - the sync is currently paused;rescheduled - the sync is waiting until more API calls are available in the source service. |
update_state |
The current data update state of the connector. The available values are:on_schedule - the sync is running smoothly, no delays;delayed - the data is delayed for a longer time than expected for the update. |
is_historical_sync |
The boolean value specifying whether the connector is at the historical data sync stage (also true for resyncing connectors). |
tasks |
The collection of tasks for the connector. |
warnings |
The collection of warnings for the connector. |
config |
The connector setup configuration. The format is specific for each connector. |
Modify a Connector
Updates the information for an existing connector within your Fivetran account.
Request
PATCH api.fivetran.com/v1/connectors/{connector_id}
{
"paused": false,
"sync_frequency": 60,
"trust_certificates": true,
"run_setup_tests": true,
"config": {
"username": "newuser",
"password": "newpassword"
},
"auth": {
"client_access": {
"client_id": "my_client_id",
"client_secret": "my_client_secret"
},
"refresh_token": "my_refresh_token"
},
"schedule_type": "manual"
}
Path parameters
| Name | Description |
|---|---|
connector_id (required) |
The unique identifier for the connector within the Fivetran system. |
Payload parameters
| Name | Description |
|---|---|
paused |
The boolean specifying whether the connector should be paused |
sync_frequency |
The connector sync frequency in minutes. The supported values are: 5, 15, 30, 60, 120, 180, 360, 480, 720, 1440. |
config |
The connector setup configuration. The format is specific for each connector. |
auth |
The connector authorization settings. Can be used to authorize a connector using your external client credentials. The format is specific for each connector. |
trust_certificates |
Specifies whether we should trust the certificate automatically. Applicable only for database connectors and when the config section is present and defined. The default value is FALSE. If a certificate is not trusted automatically, it has to be approved using the Certificates Management API. |
is_historical_sync |
The boolean specifying whether the connector should be triggered to re-sync all historical data. |
schedule_type |
The connector schedule config type. Supported values: auto, manual. Lets you disable or enable an automatic data sync on a schedule. If you set this parameter to manual, the automatic data sync will be disabled, but you will be able to trigger the data sync using the Sync Connector Data endpoint. |
run_setup_tests |
Specifies whether the setup tests should be run automatically. The default value is TRUE. |
None of the paused, sync_frequency, config, auth, is_historical_sync, schedule_type parameters is required, but at least one of them has to be present and defined for a request of this type to succeed.
Response
HTTP 200 OK
{
"code": "Success",
"message": "Connector has been updated",
"data": {
"id": "speak_inexpensive",
"group_id": "projected_sickle",
"service": "criteo",
"service_version": 0,
"schema": "criteo",
"connected_by": "interment_burdensome",
"created_at": "2018-12-01T15:43:29.013729Z",
"succeeded_at": null,
"failed_at": null,
"sync_frequency":60,
"status": {
"setup_state": "incomplete",
"sync_state": "scheduled",
"update_state": "on_schedule",
"is_historical_sync": true,
"tasks": [],
"warnings": []
},
"setup_tests": [{
"title": "Validate Login",
"status": "FAILED",
"message": "Invalid login credentials"
}],
"config": {
"username": "newuser",
"password": "******",
"api_token": "******",
"service_version": "0"
}
}
}
If the run_setup_tests parameter is set to FALSE in the request, then the response will not contain the setup_tests field.
Sync Connector Data
Triggers a data sync for an existing connector within your Fivetran account without waiting for the next scheduled sync. This action does not override the standard sync frequency you defined in the Fivetran dashboard.
If there is a data sync already in progress, that sync is immediately stopped and restarted. If the connector is paused, the data sync will be scheduled to be performed when the connector is re-enabled.
Request
POST api.fivetran.com/v1/connectors/{connector_id}/force
Path parameters
| Name | Description |
|---|---|
connector_id (required) |
The unique identifier for the connector within the Fivetran system. |
Payload
The request body must be empty.
Response
HTTP 200 OK
{
"code": "Success",
"message": "Sync has been successfully triggered for connector with id 'connector_id1'"
}
Re-sync Connector Table Data
Triggers a historical sync of all data for a single table within a connector. This action does not override the standard sync frequency you defined in the Fivetran dashboard.
If the connector is paused, the table sync will be scheduled to be performed when the connector is re-enabled. If there is a data sync already in progress, we will try to complete it. If it fails, the request will be declined and the HTTP 409 Conflict error will be returned.
Request
POST api.fivetran.com/v1/connectors/{connector_id}/schemas/{schema}/tables/{table}/resync
Path parameters
| Name | Description |
|---|---|
connector_id (required) |
The unique identifier for the connector within the Fivetran system. |
schema (required) |
The database schema name within your destination. NOTE: The database schema is a different entity from the connector schema. |
table (required) |
The table name within your database schema. |
In order for the request to succeed, you must specify its parameters correctly. You can access the exact name of your database schema and table in the Schema tab of your connector. See the following example:
Payload
The request body must be empty.
Response
HTTP 200 OK
{
"code": "Success",
"message": "Re-sync has been successfully triggered for 'schema1.table1'"
}
Run connector setup tests
Runs the setup tests for an existing connector within your Fivetran account.
Request
POST api.fivetran.com/v1/connectors/{connector_id}/test
{
"trust_certificates": true
}
Path parameters
| Name | Description |
|---|---|
connector_id (required) |
The unique identifier for the connector within the Fivetran system. |
Payload parameters
| Name | Description |
|---|---|
trust_certificates |
Specifies whether we should trust the certificate automatically. Applicable only for database connectors. The default value is FALSE. If a certificate is not trusted automatically, it has to be approved using the Certificates Management API. |
Response
HTTP 200 OK
{
"code": "Success",
"message": "Setup tests were completed",
"data": {
"id": "speak_inexpensive",
"group_id": "projected_sickle",
"service": "criteo",
"service_version": 0,
"schema": "criteo",
"connected_by": "interment_burdensome",
"created_at": "2018-12-01T15:43:29.013729Z",
"succeeded_at": null,
"failed_at": null,
"sync_frequency":60,
"status": {
"setup_state": "incomplete",
"sync_state": "scheduled",
"update_state": "on_schedule",
"is_historical_sync": true,
"tasks": [],
"warnings": []
},
"setup_tests": [{
"title": "Validate Login",
"status": "FAILED",
"message": "Invalid login credentials"
}],
"config": {
"username": "newuser",
"password": "******",
"api_token": "******",
"service_version": "0"
}
}
}
Delete a Connector
Deletes a connector from your Fivetran account.
Request
DELETE api.fivetran.com/v1/connectors/{connector_id}
Path parameters
| Name | Description |
|---|---|
connector_id (required) |
The unique identifier for the connector within the Fivetran system. |
Response
HTTP 200 OK
{
"code": "Success",
"message": "Connector with id 'speak_inexpensive' has been deleted"
}
Retrieve a Connector Schema Config
Returns the connector schema config for an existing connector within your Fivetran account.
Request
GET api.fivetran.com/v1/connectors/{connector_id}/schemas
Path parameters
| Name | Description |
|---|---|
connector_id (required) |
The unique identifier for the connector within the Fivetran system. |
Response
HTTP 200 OK
{
"code": "Success",
"data": {
"enable_new_by_default": false,
"schemas": {
"schema_1": {
"enabled": true,
"tables": {
"table_1": {
"enabled": true,
"enabled_patch_settings": {
"allowed": true
},
"columns": {
"column_1": {
"enabled": true,
"hashed": false,
"enabled_patch_settings": {
"allowed": false,
"reason_code": "SYSTEM_COLUMN",
"reason": "The column does not support exclusion as it is a Primary Key"
}
},
"column_2": {
"enabled": true,
"hashed": false,
"enabled_patch_settings": {
"allowed": true
}
},
"column_3": {
"enabled": true,
"hashed": true,
"enabled_patch_settings": {
"allowed": true
}
}
}
},
"table_2": {
"enabled": true,
"enabled_patch_settings": {
"allowed": false,
"reason_code": "SYSTEM_TABLE"
}
}
}
},
"schema_2": {
"enabled": true,
"tables": {
"table_1": {
"enabled": true,
"enabled_patch_settings": {
"allowed": true
},
"columns": {
"column_1": {
"enabled": false
}
}
},
"table_2": {
"enabled": false,
"enabled_patch_settings": {
"allowed": false,
"reason_code": "OTHER",
"reason": "Permission denied"
}
}
}
}
}
}
}
Fields
| Name | Description |
|---|---|
enable_new_by_default |
The boolean value specifying whether to enable new schemas and tables by default. |
schema.enabled |
The boolean value specifying whether the sync for the schema into the destination is enabled. |
table.enabled |
The boolean value specifying whether the sync of table into the destination is enabled. |
table.enabled_patch_settings.allowed |
The boolean value specifying whether the table.enabled property can be modified. |
table.enabled_patch_settings.reason_code |
The reason code indicating the reason why the table.enabled property cannot be modified: SYSTEM_TABLE - the table is a system table; DELETED - the table was deleted in the source; OTHER - the table was excluded by the system for some reason. This field appears in the response when the table.enabled_patch_settings.allowed property value is FALSE. |
table.enabled_patch_settings.reason |
The additional message indicating the reason why the table.enabled property cannot be modified. This field appears in the response only when the table.enabled_patch_settings.reason_code value is OTHER. |
column.enabled |
The boolean value specifying whether the sync of the column into the destination is enabled. |
column.enabled_patch_settings.allowed |
The boolean value specifying whether the column.enabled property can be modified. |
column.enabled_patch_settings.reason_code |
The reason code indicating the reason why the column.enabled property cannot be modified: SYSTEM_COLUMN - the column is a system column; DELETED - the column was deleted in the source; OTHER - the column was excluded by the system for some reason. |
column.enabled_patch_settings.reason |
The additional message indicating the reason why the column.enabled property cannot be modified. This field only appears in the response when the column.enabled_patch_settings.reason_code property value is SYSTEM_COLUMN or OTHER. |
column.hashed |
The boolean value specifying whether a column is hashed or not. |
The response contains only the difference between your selections and the schema config. The schema config includes schemas, tables, and columns. If you have not specified any schema configurations that differ from the default, the response will consist of only the top schema level.
Retrieve Source Table Columns Config
Returns the source table columns config for an existing connector within your Fivetran account.
Request
GET api.fivetran.com/v1/connectors/{connector_id}/schemas/{schema}/tables/{table}/columns
Path parameters
| Name | Description |
|---|---|
connector_id (required) |
The unique identifier for the connector within the Fivetran system. |
schema (required) |
The database schema name within your source. NOTE: The database schema is a different entity from the connector schema. |
table (required) |
The table name within your source schema. |
Response
HTTP 200 OK
{
"code": "Success",
"columns": {
"column_1": {
"enabled": true,
"hashed": false,
"enabled_patch_settings": {
"allowed": false,
"reason_code": "SYSTEM_COLUMN",
"reason": "The column does not support exclusion as it is a Primary Key"
}
},
"column_2": {
"enabled": true,
"hashed": false,
"enabled_patch_settings": {
"allowed": true
}
},
"column_3": {
"enabled": false,
"hashed": false,
"enabled_patch_settings": {
"allowed": true
}
}
}
}
The response contains an array of columns with the Fivetran sync settings for each column.
Fields
| Name | Description |
|---|---|
enabled |
The boolean value specifying whether the sync of the column into the destination is enabled. |
hashed |
The boolean value specifying whether a column should be hashed. |
column.enabled |
The boolean value specifying whether the sync of the column into the destination is enabled. |
enabled_patch_settings.allowed |
The boolean value specifying whether the column.enabled property can be modified. |
enabled_patch_settings.reason_code |
The reason code indicating the reason why the column.enabled property cannot be modified: SYSTEM_COLUMN - the column is a system column; DELETED - the column was deleted in the source; OTHER - the column was excluded by the system for some reason. This field appears in the response when the enabled_patch_settings.allowed field value is FALSE. |
Fetching the data may take long time because we interact with the source. The response time depends on the connection to the source.
Reload a Connector Schema Config
Reloads the connector schema config for an existing connector within your Fivetran account.
We advise that you use this method if the connector was recently created or its schema is outdated.
Request
POST api.fivetran.com/v1/connectors/{connector_id}/schemas/reload
{
"exclude_mode":"PRESERVE"
}
Path parameters
| Name | Description |
|---|---|
connector_id (required) |
The unique identifier for the connector within the Fivetran system. |
Payload
| Name | Description | Possible Values |
|---|---|---|
exclude_mode |
Specifies whether all schemas and tables will be enabled or disabled in the standard config. NOTE: This does not exclude columns by default. | EXCLUDE, INCLUDE and PRESERVE. The default value is PRESERVE. |
The request body can be empty because it contains only one optional parameter exclude_mode that has value PRESERVE set by default.
Response
HTTP 200 OK
{
"code": "Success",
"data": {
"enable_new_by_default": false,
"schemas": {
"schema_1": {
"enabled": true,
"tables": {
"table_1": {
"enabled": true,
"enabled_patch_settings": {
"allowed": true
},
"columns": {
"column_1": {
"enabled": true,
"hashed": false,
"enabled_patch_settings": {
"allowed": false,
"reason_code": "SYSTEM_COLUMN",
"reason": "The column does not support exclusion as it is a Primary Key"
}
},
"column_2": {
"enabled": true,
"hashed": false,
"enabled_patch_settings": {
"allowed": true
}
},
"column_3": {
"enabled": true,
"hashed": true,
"enabled_patch_settings": {
"allowed": true
}
}
}
},
"table_2": {
"enabled": true,
"enabled_patch_settings": {
"allowed": false,
"reason_code": "SYSTEM_TABLE"
}
}
}
},
"schema_2": {
"enabled": true,
"tables": {
"table_1": {
"enabled": true,
"enabled_patch_settings": {
"allowed": true
},
"columns": {
"column_1": {
"enabled": false
}
}
},
"table_2": {
"enabled": false,
"enabled_patch_settings": {
"allowed": false,
"reason_code": "OTHER",
"reason": "Permission denied"
}
}
}
}
}
}
}
Fields
| Name | Description |
|---|---|
enable_new_by_default |
The boolean value specifying whether to enable new schemas and tables by default. |
schemas |
The set of schemas within your connector schema config that will be synced into the destination. |
schema.enabled |
The boolean value specifying whether the sync of the schema into the destination is enabled. |
tables |
The set of tables within your database schema config that will be synced into the destination. |
table.enabled |
The boolean value specifying whether the sync of table into the destination is enabled. |
table.enabled_patch_settings.allowed |
The boolean value specifying whether the property table.enabled can be modified. |
table.enabled_patch_settings.reason_code |
The reason code indicating the reason why the property table.enabled cannot be modified: SYSTEM_TABLE - the table is a system table; DELETED - the table was deleted in the source; OTHER - the table was excluded by the system for some reason. This field only appears in the response when the table.enabled_patch_settings.allowed property value is FALSE. |
table.enabled_patch_settings.reason |
The additional message indicating the reason why the property table.enabled cannot be modified. This field only appears in the response when the table.enabled_patch_settings.reason_code property value is OTHER. |
columns |
The set of columns within your table schema config that will be synced into the destination. |
column.enabled |
The boolean value specifying whether the sync of the column into the destination is enabled. |
column.enabled_patch_settings.allowed |
The boolean value specifying whether the column.enabled property can be modified. |
column.enabled_patch_settings.reason_code |
The reason code indicating the reason why the column.enabled property cannot be modified: SYSTEM_COLUMN - the column is a system column; DELETED - the column was deleted in the source; OTHER - the column was excluded by the system for some reason. |
column.enabled_patch_settings.reason |
The additional message indicating the reason why the column.enabled property cannot be modified. This field only appears in the response when the column.enabled_patch_settings.reason_code property value is SYSTEM_COLUMN or OTHER. |
column.hashed |
The boolean value specifying whether the column is hashed or not. |
This method reloads the full schema from the connector's data source. It may take a long time to complete the request. The method execution speed depends on the schema size and the number of databases, tables and columns. To retrieve the connector schema config without reload, use the Retrieve a Connector Schema Config endpoint.
By specifying the exclude_mode parameter value, you can enable or disable syncing for all schemas and tables when reloading the connector schema config.
When the default PRESERVE value is set, all selected schemas and tables in the standard config preserve their state.
When the EXCLUDE value is set, the columns that haven't been modified by user yet are not excluded from the sync. Such columns may be present in the source and not displayed in the response, but they are considered to be included in the sync by default. If you enable syncing of their table, then all of these columns from the source will be synced in the next sync cycle. See also Use Case: How to enable only those table columns you need.
The response contains only the difference between your selections and the schema config. The schema config includes schemas, tables, and columns. If you have not specified any schema configurations that differ from the default, the response will consist of only the top schema level.
Modify a Connector Schema Config
Updates the schema config for an existing connector within your Fivetran account (for a single schema for a connector with multiple schemas).
Request
PATCH api.fivetran.com/v1/connectors/{connector_id}/schemas
{
"enable_new_by_default": true,
"schemas": {
"schema_1": {
"enabled": true,
"tables": {
"table_1": {
"enabled": true
},
"table_2": {
"enabled": false,
"columns": {
"column_2": {
"enabled": true,
"hashed": false
},
"column_3": {
"hashed": true
}
}
}
}
},
"schema_2": {
"enabled": false
}
}
}
Path parameters
| Name | Description |
|---|---|
connector_id (required) |
The unique identifier for the connector within the Fivetran system. |
Payload parameters
| Name | Description |
|---|---|
enable_new_by_default |
The boolean value specifying whether to enable new schemas, tables, and columns by default. |
schema.enabled |
The boolean value specifying whether the sync for the schema into the destination is enabled. |
table.enabled |
The boolean value specifying whether the sync of table into the destination is enabled. |
column.enabled |
The boolean value specifying whether the sync of the column into the destination is enabled. |
column.hashed |
The boolean value specifying whether a column should be hashed. |
Response
HTTP 200 OK
{
"code": "Success",
"data": {
"enable_new_by_default": true,
"schemas": {
"schema_1": {
"enabled": true,
"tables": {
"table_1": {
"enabled": true,
"enabled_patch_settings": {
"allowed": true
}
},
"table_2": {
"enabled": false,
"enabled_patch_settings": {
"allowed": true
},
"columns": {
"column_1": {
"enabled": true,
"hashed": false,
"enabled_patch_settings": {
"allowed": false,
"reason_code": "SYSTEM_COLUMN",
"reason": "The column does not support exclusion as it is a Primary Key"
}
},
"column_2": {
"enabled": true,
"hashed": false,
"enabled_patch_settings": {
"allowed": true
}
},
"column_3": {
"enabled": true,
"hashed": true,
"enabled_patch_settings": {
"allowed": true
}
}
}
}
}
},
"schema_2": {
"enabled": false,
"tables": {
"table_1": {
"enabled": true,
"enabled_patch_settings": {
"allowed": true
}
},
"table_2": {
"enabled": false,
"enabled_patch_settings": {
"allowed": false,
"reason_code": "SYSTEM_TABLE"
}
}
}
}
}
}
}
The response contains only the difference between your selections and the schema config. The schema config includes schemas, tables, and columns. If you have not specified any schema configurations that differ from the default, the response will consist of only the top schema level.
Modify a Connector Database Schema Config
Updates the database schema config for an existing connector within your Fivetran account (for a single schema within a connector with multiple schemas).
Request
PATCH api.fivetran.com/v1/connectors/{connector_id}/schemas/{schemaName}
{
"enabled": true,
"tables": {
"table_1": {
"enabled": true
},
"table_2": {
"enabled": false,
"columns": {
"column_2": {
"enabled": true,
"hashed": false
},
"column_3": {
"hashed": true
}
}
}
}
}
Path parameters
| Name | Description |
|---|---|
connector_id (required) |
The unique identifier for the connector within the Fivetran system. |
schema (required) |
The database schema name within your destination (different from the connector schema). |
Payload parameters
| Name | Description |
|---|---|
enabled |
The boolean value specifying whether the sync for the schema into the destination is enabled. |
table.enabled |
The boolean value specifying whether the sync of table into the destination is enabled. |
column.enabled |
The boolean value specifying whether the sync of the column into the destination is enabled. |
column.hashed |
The boolean value specifying whether the column should be hashed. |
Response
HTTP 200 OK
{
"code": "Success",
"data": {
"enable_new_by_default": true,
"schemas": {
"schema_1": {
"enabled": true,
"tables": {
"table_1": {
"enabled": true,
"enabled_patch_settings": {
"allowed": true
}
},
"table_2": {
"enabled": false,
"enabled_patch_settings": {
"allowed": true
},
"columns": {
"column_1": {
"enabled": true,
"hashed": false,
"enabled_patch_settings": {
"allowed": false,
"reason_code": "SYSTEM_COLUMN",
"reason": "The column does not support exclusion as it is a Primary Key"
}
},
"column_2": {
"enabled": true,
"hashed": false,
"enabled_patch_settings": {
"allowed": true
}
},
"column_3": {
"enabled": true,
"hashed": true,
"enabled_patch_settings": {
"allowed": true
}
}
}
}
}
},
"schema_2": {
"enabled": false,
"tables": {
"table_1": {
"enabled": true,
"enabled_patch_settings": {
"allowed": true
}
},
"table_2": {
"enabled": false,
"enabled_patch_settings": {
"allowed": false,
"reason_code": "SYSTEM_TABLE"
}
}
}
}
}
}
}
The response contains only the difference between your selections and the schema config. The schema config includes schemas, tables, and columns. If you have not specified any schema configurations that differ from the default, the response will consist of only the top schema level.
Modify a Connector Table Config
Updates the table config within your database schema for an existing connector within your Fivetran account.
Request
PATCH api.fivetran.com/v1/connectors/{connector_id}/schemas/{schema}/tables/{table}
{
"enabled": false,
"columns": {
"column_2": {
"enabled": true,
"hashed": false
},
"column_3": {
"hashed": true
}
}
}
Path parameters
| Name | Description |
|---|---|
connector_id (required) |
The unique identifier for the connector within the Fivetran system. |
schema (required) |
The database schema name within your destination (different from the connector schema). |
table (required) |
The table name within your database schema. |
Payload parameters
| Name | Description |
|---|---|
enabled |
The boolean value specifying whether the sync for the table into the destination is enabled. |
column.enabled |
The boolean value specifying whether the sync of the column into the destination is enabled. |
column.hashed |
The boolean value specifying whether the column should be hashed. |
Response
HTTP 200 OK
{
"code": "Success",
"data": {
"enable_new_by_default": true,
"schemas": {
"schema_1": {
"enabled": true,
"tables": {
"table_1": {
"enabled": true,
"enabled_patch_settings": {
"allowed": true
}
},
"table_2": {
"enabled": false,
"enabled_patch_settings": {
"allowed": true
},
"columns": {
"column_1": {
"enabled": true,
"hashed": false,
"enabled_patch_settings": {
"allowed": false,
"reason_code": "SYSTEM_COLUMN",
"reason": "The column does not support exclusion as it is a Primary Key"
}
},
"column_2": {
"enabled": true,
"hashed": false,
"enabled_patch_settings": {
"allowed": true
}
},
"column_3": {
"enabled": true,
"hashed": true,
"enabled_patch_settings": {
"allowed": true
}
}
}
}
}
},
"schema_2": {
"enabled": false,
"tables": {
"table_1": {
"enabled": true,
"enabled_patch_settings": {
"allowed": true
}
},
"table_2": {
"enabled": false,
"enabled_patch_settings": {
"allowed": false,
"reason_code": "SYSTEM_TABLE"
}
}
}
}
}
}
}
The response contains only the difference between your selections and the schema config. The schema config includes schemas, tables, and columns. If you have not specified any schema configurations that differ from the default, the response will consist of only the top schema level.
Modify a Connector Column Config
Updates the column config within your table for an existing connector within your Fivetran account.
Request
PATCH api.fivetran.com/v1/connectors/{connector_id}/schemas/{schema}/tables/{table}/columns/{column}
{
"enabled": true,
"hashed": false
}
Path parameters
| Name | Description |
|---|---|
connector_id (required) |
The unique identifier for the connector within the Fivetran system. |
schema (required) |
The database schema name within your destination (different from the connector schema). |
table (required) |
The table name within your database schema. |
column (required) |
The column name within your table. |
Payload parameters
| Name | Description |
|---|---|
enabled |
The boolean value specifying whether the sync for the column into the destination is enabled. |
hashed |
The boolean value specifying whether a column should be hashed. |
Response
HTTP 200 OK
{
"code": "Success",
"data": {
"enable_new_by_default": true,
"schemas": {
"schema_1": {
"enabled": true,
"tables": {
"table_1": {
"enabled": true,
"enabled_patch_settings": {
"allowed": true
}
},
"table_2": {
"enabled": false,
"enabled_patch_settings": {
"allowed": true
},
"columns": {
"column_1": {
"enabled": true,
"hashed": false,
"enabled_patch_settings": {
"allowed": false,
"reason_code": "SYSTEM_COLUMN",
"reason": "The column does not support exclusion as it is a Primary Key"
}
},
"column_2": {
"enabled": true,
"hashed": false,
"enabled_patch_settings": {
"allowed": true
}
},
"column_3": {
"enabled": true,
"hashed": true,
"enabled_patch_settings": {
"allowed": true
}
}
}
}
}
},
"schema_2": {
"enabled": false,
"tables": {
"table_1": {
"enabled": true,
"enabled_patch_settings": {
"allowed": true
}
},
"table_2": {
"enabled": false,
"enabled_patch_settings": {
"allowed": false,
"reason_code": "SYSTEM_TABLE"
}
}
}
}
}
}
}
The response contains only the difference between your selections and the schema config. The schema config includes schemas, tables, and columns. If you have not specified any schema configurations that differ from the default, the response will consist of only the top schema level.
Use Case: Create a new connector based on an existing connector's parameters
You can create new connectors using the parameters of an existing connector.
-
Retrieve the connector object details. See Retrieve Connector Details.
Request
GET api.fivetran.com/v1/connectors/{connector_id}Path parameters
Name Description connector_id(required)The unique identifier for the connector within the Fivetran system. Response
HTTP 200 OK -
Make a note of the following fields from the API response:
{ "group_id": "projected_sickle" "service": "criteo" "config": { "username": "myuser", "password": "******", "api_token": "******", "service_version": "0" } } -
Create the new connector using the fields from the previous step. See Create a connector.
Request
POST api.fivetran.com/v1/connectors{ "service": "criteo", "group_id": "projected_sickle", "trust_certificates": true, "run_setup_tests": true, "config": { "schema": "criteo", "username": "myuser", "password": "mypassword", "app_token": "myapptoken" } }Payload parameters
Name Description group_id(required)The unique identifier for the group within the Fivetran system. Find you group_idby fetching your account's group list.service(required)The name for the connector type within the Fivetran system. config(required)The connector setup configuration. The format is specific for each connector. trust_certificatesSpecifies whether we should trust the certificate automatically. Applicable only for database connectors. The default value is FALSE. If a certificate is not trusted automatically, it has to be approved using the Certificates Management API.run_setup_testsSpecifies whether setup tests should be run automatically. The default value is TRUE.Response
HTTP 201 Created
Use Case: How to enable sync only for columns that you need
The connector standard config only stores the information on the difference between the schema config in the source and the changes made to it by the user. To clarify this, let's have a look at this example:
Request
GET api.fivetran.com/v1/connectors/connector_id_1/schemas/schema_1/tables/table_1/columns
Response
HTTP 200 OK
{
"code": "Success",
"columns": {
"column_1": {
"enabled": false,
"hashed": false,
"enabled_patch_settings": {
"allowed": true
}
}
}
}
This example shows that we have only one disabled sync for column_1, but it does not necessarily mean we do not have other columns in the source.
For example, the source might have column_2 and column_3 in this table. Syncing is enabled for all table columns by default, they will be synced in the next connector sync cycle.
Assume we want to enable sync only for column column_3.
First of all, to exclude other unwanted columns you need to know all column names of your table:
-
We can use the Retrieve source table columns config endpoint for this purpose.
Request
GET api.fivetran.com/v1/connectors/connector_id_1/schemas/schema_1/tables/table_1/columnsResponse
HTTP 200 OK{ "code": "Success", "columns": { "column_1": { "enabled": false, "hashed": false, "enabled_patch_settings": { "allowed": true, } }, "column_2": { "enabled": true, "hashed": false, "enabled_patch_settings": { "allowed": true } }, "column_3": { "enabled": true, "hashed": false, "enabled_patch_settings": { "allowed": true } } } } -
Once we know column names, we can explicitly exclude from the sync the columns we do not want to be synced and include the ones we need through the Modify a connector table config endpoint.
Request
PATCH api.fivetran.com/v1/connectors/connector_id_1/schemas/schema_1/tables/table_1{ "enabled": true, "columns": { "column_1": { "enabled": false, "hashed": false }, "column_2": { "enabled": false, "hashed": false }, "column_3": { "enabled": true, "hashed": false } } }Response
HTTP 200 OK{ "code": "Success", "data": { "enable_new_by_default": false, "schemas": { "schema_1": { "enabled": true, "tables": { "table_1": { "enabled": true, "enabled_patch_settings": { "allowed": true }, "columns": { "column_1": { "enabled": false, "hashed": false, "enabled_patch_settings": { "allowed": true } }, "column_2": { "enabled": false, "hashed": false, "enabled_patch_settings": { "allowed": true } }, "column_3": { "enabled": true, "hashed": false, "enabled_patch_settings": { "allowed": true } } } } } } } } }