Dynamic Table Mapping

Dynamic Table Mapping automatically creates and maps destination tables based on your file naming patterns. Instead of manually configuring each table, you provide a single pattern that extracts table names from your file paths. When new files matching your pattern appear, Fivetran automatically creates the corresponding tables, eliminating the need for manual intervention. Once you enter your table extraction pattern. Fivetran then automatically performs the following actions:

• Files that match the table extraction pattern during the initial sync: Fivetran identifies all files matching your pattern and creates tables based on the extracted names.

• New files added after the initial sync that match the table extraction pattern: When files with new table names arrive (for example, refunds_20250120.csv), Fivetran applies your pattern as follows:

  • If the Schema Change Handling setting for the connection is set to Allow all, Fivetran automatically creates the new table refunds and begins syncing the data.
  • Otherwise, the table is created but remains unselected until you review and select it.

Use Cases

Your source contains many entities represented by files that follow a consistent naming pattern. For example:

• sales_20250115.csv - sales
• inventory_20250115.csv - inventory
• orders_20250115.csv - orders

We recommend to use Define per table option instead in the following cases:

• One-off files with inconsistent naming.
• Scenarios requiring complex or custom table naming logic.
• Archive files.

How to Use Dynamic Table Mapping

Choose file mapping method

In the Fivetran dashboard, under Configure Files section, select Dynamic extract tables.

Enter table extraction pattern

Write a regex pattern with exactly one named capture group called table. This capture group defines which part of the filename becomes your table name.

• Pattern structure: (?<table>...)
• The ... represents the part of the filename you want to extract as the table name.

Validate table extraction pattern

Click Preview to see how your pattern will map files to tables. Fivetran will show up to 5 examples of discovered tables based on your current files.

File pattern examples

This section helps you understand the file pattern through examples, including the table extraction pattern and the key components of a table pattern.

Files with date suffixes

• Files: sales_2025_01.json, transactions_2025_01.json, inventory_2025_01.json
• Table extraction pattern: (?<table>\w+)_\d{4}_\d{2}\.json
• Extracted table names: sales, transactions, and inventory

Files in categorized folders

• Files: /exports/customers/data_20251016.csv, /exports/products/data_20251016.csv
• Table extraction pattern: /exports/(?<table>\w+)/data_\d{8}\.csv
• Extracted table names: customers and products

Multi-level folder extraction

• Files: /client/acme/invoices/file.csv, /client/acme/contracts/file.csv
• Table extraction pattern: /client/\w+/(?<table>\w+)/.*\.csv
• Extracted table names: invoices and contracts

Combining folder and file patterns

• Files: /data/us/sales_final.parquet, /data/eu/sales_final.parquet
• Table extraction pattern: /data/(?<table>\w+)/sales_final\.parquet
• Extracted table names: us and eu

Standard pattern components

• \w+ – Matches one or more alphanumeric characters or underscores. Commonly used to capture table or file name components.
• \d{8} – Matches exactly eight digits, often used to represent dates in YYYYMMDD format (for example, 20250115).
• \d{4} – Matches exactly four digits, often used to capture years (for example, 2025).
• \.csv – Matches the .csv file extension.
• .* – Matches any sequence of characters. Use carefully, as it can match more than intended and reduce precision.

Best practices and suggestions

• Use \w+ to match word characters (letters, numbers, underscores).
• Escape special characters with backslash: \.csv not .csv.
• Use \d{n} to match exactly n digits.
• Test your pattern on Regex expression and then click Preview.
• Keep patterns as simple as possible while being specific.
• Don't include multiple named groups. Only (?<table>...) is allowed.
• Don't use .* in your table capture group.