How Can I Optimize Historical Data Retrieval Speed?
Question
Why is historical data retrieval slow, and how can I optimize my connection for data retrieval speed?
Environment
Connector: Adobe Analytics Data Feed
Answer
Retrieval speed depends on data volume and Adobe's processing limits. However, you can optimize your connection's performance by doing the following:
- Create separate feeds, each covering a specific period such as a quarter or a year.
- Ensure the periods don't overlap, since Fivetran doesn't de-duplicate records.
- If overlaps are unavoidable, de-duplicate records in your destination system using timestamps or other unique identifiers.