Amazon Kinesis Firehose API Configuration

Request

POST https://api.fivetran.com/v1/connections

{
  "group_id": "group_id",
  "service": "kinesis",
  "trust_certificates": true,
  "trust_fingerprints": true,
  "run_setup_tests": true,
  "paused": false,
  "pause_after_trial": false,
  "sync_frequency": 1440,
  "data_delay_sensitivity": "NORMAL",
  "data_delay_threshold": 0,
  "daily_sync_time": "14:00",
  "schedule_type": "auto",
  "connect_card_config": {
    "redirect_uri": "https://your.site/path",
    "hide_setup_guide": true
  },
  "proxy_agent_id": "proxy_agent_id",
  "private_link_id": "private_link_id",
  "networking_method": "Directly",
  "hybrid_deployment_agent_id": "hybrid_deployment_agent_id",
  "destination_configuration": {
    "virtual_warehouse": "virtual_warehouse"
  },
  "destination_schema_names": "SOURCE_NAMING",
  "config": {
    "archive_pattern": "string",
    "bucket": "aws_region_code",
    "compression": "zip | gz | tar_gz | tar | infer | gzip | uncompressed | bz2 | tar_bz2",
    "connection_type": "Directly | PrivateLink | SshTunnel | ProxyAgent",
    "delimiter": "string",
    "empty_header": true,
    "escape_char": "string",
    "is_public": true,
    "json_delivery_mode": "Packed | Unpacked",
    "line_separator": "string",
    "list_strategy": "complete_listing | time_based_pattern_listing",
    "null_sequence": "string",
    "on_error": "fail | skip",
    "pattern": "string",
    "prefix": "folder_path",
    "role_arn": "arn::your_role_arn",
    "skip_after": 0,
    "skip_before": 0,
    "schema": "schema_name",
    "table": "table_name"
  }
}

Config parameters

Name	Description
`archive_pattern`	Optional. Files inside of compressed archives with filenames matching this regular expression will be synced.
`bucket`	The name of the Kinesis bucket.
`compression`	If your files are compressed, but do not have extensions indicating the compression method, you can force them to be uncompressed according to the selected compression algorithm. Leave the value as infer if your files are saved with the correct compression extensions.
`connection_type`	Connection method. Default value: `Directly`.
`delimiter`	Optional. You can specify your the delimiter that your CSVs use here. Fivetran generally tries to infer the delimiter, but in some cases this is impossible. If your files sync with the wrong number of columns, consider setting this value
`empty_header`	Optional. If your CSV generating software doesn't provide header line for the documents, Fivetran can generate the generic column names and sync data rows with them.
`escape_char`	Optional. If your CSV generator follows non-standard rules for escaping quotation marks, you can set the escape character here.
`is_public`	Is the bucket public? (you don't need an AWS account for syncing public buckets!)
`json_delivery_mode`	Control how your JSON data is delivered into your destination
`line_separator`	You can specify the custom line separator for your CSV files. The line separator is used in files to separate one row from the next.
`list_strategy`	Optional. If you have a file structure where new files are always named in lexicographically increasing order such as files being named in increasing order of time, you can select `time_based_pattern_listing`.
`null_sequence`	Optional. If your CSVs use a special value indicating null, you can specify it here.
`on_error`	If you know that your files contain some errors, you can choose to have poorly formatted lines skipped. We recommend leaving the value as fail unless you are certain that you have undesirable, malformed data.
`pattern`	Optional. All files in your search path matching this regular expression will be synced. This parameter is optional.
`prefix`	Folder path to the Kinesis files within the bucket.
`role_arn`	The Role ARN required for authentication.
`skip_after`	Enter 1 or greater
`skip_before`	Enter 1 or greater
`schema` (required)	Destination schema name. Schema name is permanent and cannot be changed after connection creation
`table` (required)	Destination table. Table is permanent and cannot be changed after connection creation

Authorization

There are two ways to authorize this connector type:

By specifying the values for the role_arn and bucket parameters in the request.

POST https://api.fivetran.com/v1/connections

{
  "service": "kinesis",
  "group_id": "group_id",
  "config": {
    "role_arn": "arn::your_role_arn",
    "bucket": "aws_region_code"
  }
}

By using the Connect Card or the Fivetran dashboard.