How Can I Track Monthly Data Volume by Connection?

Question

How can I track the monthly data volume synced by a connection?

Environment

The following database connectors:

• MySQL
• Oracle
• PostgreSQL
• SQL Server

Answer

For supported connectors, the Fivetran Platform Connector delivers sync stats to your destination in the SYNC_STATS table, which records the volume of data extracted, processed, and loaded during each sync.

To calculate the monthly volume of data extracted, processed, and loaded during syncs by each connection (in GB), run a query similar to the example below. If your log schema differs from fivetran_log, replace the schema name accordingly.

SELECT
  integration_id,
  schema_name,
  DATE_FORMAT(sync_started_at, '%Y-%m-01') AS year_month,
  SUM(extract_volume_bytes * 9.313E-10) AS extract_vol_GB,
  SUM(process_volume_bytes * 9.313E-10) AS process_volume_GB,
  SUM(load_volume_bytes * 9.313E-10) AS load_volume_GB
FROM
  fivetran_log.SYNC_STATS
GROUP BY
  integration_id,
  schema_name,
  year_month
ORDER BY
  integration_id,
  year_month;