Salesforce MAR Management
NOTE: We wrote this article before we changed our pricing model on February 1, 2022. If you signed up for a new Fivetran account or renewed your existing account after January 31, 2022, you have an unlimited number of free customer-triggered re-syncs per connector each month. For more information about MAR usage, see our pricing documentation.
Use Case
You want to understand how the functioning of Salesforce connectors affects Monthly Active Rows (MAR).
Environment
Connector Overview
Some Salesforce objects are only available with specific versions of the Salesforce REST API.
Upon setting up the connector, Fivetran automatically selects the tables that can be replicated into the destination. This includes custom tables, which are unique for each customer.
Fivetran may not be able to sync some tables into the destination because of the following reasons:
- the table is not queryable
- you do not have the permission to read the table
We need to re-sync some tables with various periodicity (up to at every sync) depending on the time it takes to re-import a given table. In particular cases, we also re-sync tables when we fail to fetch the deleted records:
- when we need to fetch deleted records within the lookback window greater than 15 days
- when we need to fetch deleted records within a given lookback window, but Salesforce returns the
INVALID_REPLICATION_DATE
error - when we need to fetch deleted records within the smallest (1 minute) lookback window
We recommended to exclude them from syncing.
You can activate history mode, which captures type 2 slowly changing dimensions.
You may be concerned with the number of API calls used by Fivetran. Fivetran automatically reschedules syncs for one hour later every time 90% of the Salesforce API call quota has been consumed. The system table
FIVETRAN_API_CALL
tracks information about API calls that are made to sync the data into the destination.
Contributing factors
There are five primary actions that contribute to MAR:
- Syncing tables that need to be re-imported each time they are synced.
- Custom Objects are unique for each customer and may significantly impact MAR.
- Enabling history mode will keep multiple versions of the same record.
- Adding new fields to objects/tables that have multiple records.
- Number of active people updating records in Salesforce.
Managing MAR
The number of objects you selected to be synced determines MAR. There may be custom tables in Salesforce as a result of an integration with Salesforce. Ensure the tables selected in the schema are necessary for analytical purposes.
Avoid historical re-syncs if possible. Table syncs are possible if changes are only made to one object/table. If significant changes are made to the data in Salesforce, it might be worth exploring the option of creating a new connector.
History mode impacts MAR. If you do not find history mode valuable, you should disable it.