Wasabi Cloud Storage API Configuration Private Preview

Request

POST https://api.fivetran.com/v1/connections

{
  "group_id": "group_id",
  "service": "wasabi_cloud_storage",
  "trust_certificates": true,
  "trust_fingerprints": true,
  "run_setup_tests": true,
  "paused": false,
  "pause_after_trial": false,
  "sync_frequency": 1440,
  "data_delay_sensitivity": "NORMAL",
  "data_delay_threshold": 0,
  "daily_sync_time": "14:00",
  "schedule_type": "auto",
  "connect_card_config": {
    "redirect_uri": "https://your.site/path",
    "hide_setup_guide": true
  },
  "proxy_agent_id": "proxy_agent_id",
  "private_link_id": "private_link_id",
  "networking_method": "Directly",
  "hybrid_deployment_agent_id": "hybrid_deployment_agent_id",
  "config": {
    "access_key_id": "access_key_id_value",
    "prefix": "folder_path",
    "pattern": "file_pattern",
    "json_delivery_mode": "Packed | Unpacked",
    "skip_after": 0,
    "empty_header": true,
    "use_pgp_encryption_options": true,
    "skip_before": 0,
    "signer_public_key": "<signer-public-key-used>",
    "delimiter": "string",
    "file_type": "csv | log | json | jsonl | infer | tsv | avro | parquet | xml | spreadsheet | unstructured",
    "on_error": "fail | skip",
    "archive_pattern": "regex_pattern",
    "pgp_secret_key": "<secret-key-used>",
    "auth_type": "IAM_ROLE | ACCESS_KEY | PUBLIC_BUCKET",
    "append_file_option": "upsert_file | append_file | upsert_file_with_primary_keys",
    "escape_char": "string",
    "bucket": "bucket_name",
    "line_separator": ";",
    "null_sequence": "string",
    "access_key_secret": "access_key_secret_value",
    "region": "US_EAST_1 | US_EAST_2 | US_CENTRAL_1 | US_WEST_1 | CA_CENTRAL_1 | EU_CENTRAL_1 | EU_CENTRAL_2 | EU_WEST_1 | EU_WEST_2 | AP_NORTHEAST_1 | AP_NORTHEAST_2 | AP_SOUTHEAST_1 | AP_SOUTHEAST_2",
    "compression": "bz2 | gz | gzip | zip | tar | tar_bz2 | tar_gz | uncompressed | infer",
    "pgp_pass_phrase": "<pass-phrase-used>",
    "schema": "schema_name",
    "table": "table_name"
  }
}

Config parameters

Name	Description
`access_key_id`	Access Key ID
`prefix`	All files and folders under this folder path will be searched for files to sync.
`pattern`	All files in your search path matching this regular expression will be synced.
`json_delivery_mode`	Specifies how Fivetran should handle your JSON data. Default value: `Packed`.
`skip_after`	We will skip over the number of lines specified at the end to avoid introducing aberrant data into your destination.
`empty_header`	If your CSVs are headerless, set this is as `true`. When `true`, we will generate generic column names following the convention of `column_0`, `column_1`, ... `column_n` to map the rows. Default value: `false`.
`use_pgp_encryption_options`	Set to `true` if files are encrypted using PGP in the Wasabi Cloud Storage bucket. Default value: `false`.
`skip_before`	We will skip over the number of lines specified before syncing data.
`signer_public_key`	The contents of the signer's public key file. Must be populated if `use_pgp_encryption_options` is set to `true` and PGP encrypted files are signed.
`delimiter`	You can specify the delimiter that your CSVs use here. Fivetran generally tries to infer the delimiter, but in some cases this is impossible.
`file_type`	If your files are saved with improper extensions, you can force them to be synced as the selected file type.
`on_error`	If you know that your files contain some errors, you can choose to have poorly formatted lines skipped. We recommend leaving the value as `fail` unless you are certain that you have undesirable, malformed data.
`archive_pattern`	Files inside compressed archives with filenames matching this regular expression will be synced.
`pgp_secret_key`	The contents of your PGP secret key file. Must be populated if `use_pgp_encryption_options` is set to `true`.
`auth_type`	The Wasabi Cloud Storage Access approach. Required for connector creation. Default value: `ACCESS_KEY`.
`append_file_option`	If you know that the source completely over-writes the same file with new data, you can append the changes instead of upserting based on filename and line number.
`escape_char`	If your CSV generator follows non-standard rules for escaping quotation marks, you can set the escape character here.
`bucket`	The Wasabi Cloud Storage bucket name. Required for connector creation.
`line_separator`	You can specify the custom line separator for your CSV files. The line separator is used in files to separate one row from the next.
`null_sequence`	If your CSVs use a special value indicating null, you can specify it here.
`access_key_secret`	Access Key Secret
`region`	The Wasabi Cloud Storage bucket region. Required for connector creation. Default value: `US_EAST_1`.
`compression`	The compression format is used to let Fivetran know that even files without a compression extension should be decompressed using the selected compression format.
`pgp_pass_phrase`	The PGP passphrase used to create the key. Must be populated if `use_pgp_encryption_options` is set to `true`.
`schema`	Destination schema name. Schema name is permanent and cannot be changed after connection creation
`table`	Destination table. Table is permanent and cannot be changed after connection creation

Authorization

There are two ways to authorize this connector type:

Access key: By specifying the auth_type as "ACCESS_KEY" and the values for the access_key_id and access_key_secret parameters in the request.

POST https://api.fivetran.com/v1/connections

{
  "service": "wasabi_cloud_storage",
  "group_id": "group_id",
  "config": {
    "access_key_id": "access_key_id_value",
    "access_key_secret": "access_key_secret_value",
    "auth_type": "ACCESS_KEY"
  }
}

By using the Connect Card or the Fivetran dashboard.