Edit on GitHub
Why Are Staging or Intermediate Tables and Schemas Created in the Destination?link
Questionlink
Why are staging or intermediate tables and schemas created in the destination?
Environmentlink
- Projects using Fivetran data models
- Projects using Quickstart data models
Answerlink
We create staging or intermediate tables in the destination when you configure your transformations. We do this for the following reasons:
- To load data efficiently: Loading data into staging or intermediate tables allows us to load data to the destination tables in bulk, which is more efficient than loading it row-by-row.
- To load data incrementally: We use incremental loading strategies to update only changed or new data since the last synchronization. To facilitate this process, we temporarily load incoming data in staging tables and compare it with existing data to determine what needs updating or adding.
- To map schemas consistently: Different source systems may have varying data structures. Adding staging tables allows us to normalize and transform data before loading it into the destination tables.