Metadata API Beta

IMPORTANT: We have deprecated the Metadata API and replaced it with Fivetran Platform Connector's capabilities to query source and destination metadata.

Overview

The Fivetran Metadata API provides metadata for data synced by Fivetran connectors, which helps you understand the mapping between the source and destination names of schemas, tables, and columns. Note that the table and column names in the source and destination may differ because we name them using our naming conventions. In some use cases, it is important to be aware of the schema, table, and column names in order to control your pipeline or build third-party integrations using this information. For example, the fully qualified name My_Schema_Name1.TableName123.Column1 in the source becomes my_schema_name_1.table_name_123.column_1 in the destination.

Supported features

Once you have access to the Fivetran API, you can call the following API endpoints:

• Retrieve schema metadata
• Retrieve table metadata
• Retrieve column metadata

Retrieve metadata

You must know the ID of the connector for which you want to fetch metadata. To explore what connectors are in your account group, you can use the List All Connectors within a Group endpoint.

We have the following API endpoints to separately fetch metadata by connector ID for a schema, table, or column, respectively:

 - GET /metadata/connectors/{connector_id}/schemas
 - GET /metadata/connectors/{connector_id}/tables
 - GET /metadata/connectors/{connector_id}/columns

The endpoints return all endpoint-specific entities that belong to the connector with the maximum limit of 10,000. For example, if the connector has 5 tables and each table has 10 columns, then the Retrieve column metadata endpoint returns 50 columns per one API call.

We recommend that you pull metadata in the following order:

1. Schemas
2. Tables
3. Columns

The id and parent_id fields reflect the relationship between the parent and child entities. You can then compose fully qualified names in the schema_name.table_name.column_name format in the request to find information about the entity relationships.

The Metadata API uses the schema captured from data passing through the Fivetran pipeline. The metadata only becomes available after a successful sync. This means that:

• if the connector has never successfully synced, then an empty response is returned
• only tables and columns that have successfully synced will be returned in the response

Retrieve schema metadata

Returns schema-level metadata for an existing connector within your Fivetran account.

Request

GET https://api.fivetran.com/v1/metadata/connectors/{connector_id}/schemas

Path parameters

Query parameters

Response

HTTP 200 OK

{
  "code": "Success",
  "data": {
    "items": [
      {
        "id": "bWFpbl9wdWJsaWM",
        "name_in_source": "Main Public",
        "name_in_destination": "main_public",
      },
      {
        "id": "dXNlcl9zZXR0aW5ncw",
        "name_in_source": "Local backup",
        "name_in_destination": "local_backup"
      }
    ],
    "next_cursor": "eyJza2lwIjoxfQ"
  }
}

Fields

Retrieve table metadata

Returns table-level metadata for an existing connector within your Fivetran account.

Request

GET https://api.fivetran.com/v1/metadata/connectors/{connector_id}/tables

Path parameters

Query parameters

Response

HTTP 200 OK

{
  "code": "Success",
  "data": {
    "items": [
      {
        "id": "NjUwMTU",
        "parent_id": "bWFpbl9wdWJsaWM",
        "name_in_source": "User Accounts",
        "name_in_destination": "user_accounts"
      },
      {
        "id": "NjUwMTY",
        "parent_id": "bWFpbl9wdWJsaWM",
        "name_in_source": "User Subscriptions",
        "name_in_destination": "user_subscriptions"
      },
      {
        "id": "NjUwMTW",
        "parent_id": "bWFpbl9wdWJsaWM",
        "name_in_source": "Account Details",
        "name_in_destination": "account_details"
      }
    ],
    "next_cursor": "YUWEudlwIjoxkK"
  }
}

Fields

Retrieve column metadata

Returns column-level metadata for an existing connector within your Fivetran account.

Request

GET https://api.fivetran.com/v1/metadata/connectors/{connector_id}/columns

Path parameters

Query parameters

Response

HTTP 200 OK

{
  "code": "Success",
  "data": {
    "items": [
      {
        "id": "NTY4ODgzNDI",
        "parent_id": "NjUwMTU",
        "name_in_source": "id",
        "name_in_destination": "id",
        "type_in_source": "Integer",
        "type_in_destination": "Integer",
        "is_primary_key": true,
        "is_foreign_key": false
      },
      {
        "id": "NTY4ODgzNDM",
        "parent_id": "NjUwMTU",
        "name_in_source": "FirstName",
        "name_in_destination": "first_name",
        "type_in_source": "String",
        "type_in_destination": "Text",
        "is_primary_key": false,
        "is_foreign_key": false
      },
      {
        "id": "NTY4ODgzNDQ",
        "parent_id": "NjUwMTU",
        "name_in_source": "LastName",
        "name_in_destination": "last_name",
        "type_in_source": "String",
        "type_in_destination": "Text",
        "is_primary_key": false,
        "is_foreign_key": false
      }
    ],
    "next_cursor": "YUWEudlwIjoxkK"
  }
}

Fields