Is Fivetran MDLS Compatible With AWS Glue Compaction?

Question

My connections load data into a Managed Data Lake Service (MDLS) destination in S3. I enabled AWS Glue automatic compaction on the destination, but the compacted data is overwritten during the next sync.

Does Fivetran Managed Data Lake Service (MDLS) support AWS Glue compaction?

Environment

• Destination: Fivetran Managed Data Lake Service (MDLS)
• Feature: AWS Glue automatic compaction

Answer

No, Fivetran MDLS doesn't support AWS Glue compaction on Fivetran-managed Iceberg tables.

Fivetran-managed Iceberg tables are single-writer tables. We own and manage the table metadata, snapshots, and state. When an external service, such as AWS Glue, rewrites the table, including for compaction, we treat those changes as external modifications. During the next sync, we restore the managed metadata and table state. As a result, any compaction performed by AWS Glue is reverted.

This behavior is expected. It helps us to maintain data consistency, metadata integrity, and reliable sync behavior. For more information, see Managed Data Lake Service - Metadata Catalogs.