Updated 3 days ago
This section briefly describes the pricing plans for Fivetran HVR.
Our pricing model is based on consumption or usage. You are charged based on how much data is replicated or consumed by a target/destination. For more information see, pricing documentation.
Fivetran calculates your consumption/usage based on the number of Monthly Active Rows (MAR) across all source and target locations within your HVR Hub System(s). MAR is the number of distinct primary keys synced/replicated from a source location to a target location in a given calendar month. For more information about MAR, see Monthly Active Rows (MAR).
Fivetran offers the following two pricing plans for HVR:
Both pricing plans require reporting the consumption/usage data to Fivetran. In the Business Critical plan, Fivetran automatically handles the consumption data and license renewal. In the Private Deployment plan, user must manually send their consumption data to Fivetran while Fivetran sends a new license to the user via email for renewal. For more information about our pricing plans, see pricing.
This is a pricing plan where HVR can be deployed on-premise. The HVR Hub System should be connected to the Fivetran server for automatically sending the consumption/usage data and receiving the license. The consumption/usage data are sent in the form of two data files - MAR Data and Hub Snapshot.
Each month, your consumption/usage is calculated based on the number of MAR across all source (connector) and target (destination) locations in your account.
Once the HVR Hub System is registered with a Fivetran account, HVR securely begins transmitting consumption/usage data to Fivetran, which is encrypted and stored on AWS S3. The consumption/usage data does not include any personally identifiable information about the user.
HVR uploads MAR data file hourly, and Fivetran sends back a 7-day license key daily as part of this data exchange. HVR purges the MAR data after 24 hours.
HVR uploads the hub snapshot every 30 days.
The MAR data is stored on the disk of the hub server machine in hour granularity buckets per table prior to being sent to Fivetran. One week's worth of storage provisioning on hub machine is required, but the actual use is one or two days' worth of data because HVR automatically purges the MAR data a day after it is sent to Fivetran. Automatic purging of MAR data can also be disabled using the repository property Metering_Purge_Frequency. For more information about the MAR storage, see Storage Requirements for MAR.
If the Fivetran account registration is deleted from a hub system, HVR immediately stops sending usage data to Fivetran, and the automatic license renewal is stopped (the license expires after 7 days).
To view the hourly MAR data that is sent to Fivetran,
Execute the command hvrlicense in the CLI:
hvrlicense -m filename -B now-1h
Use a text editor to open the file and view the data.
For the Business Critical plan, you must:
- Enable outbound access from the web browser that you will use to register the HVR Hub System to the IP address 184.108.40.206 (fivetran.com).
- Enable outbound access from the system running the HVR Hub Server to the following IP addresses (on port 443). This is required for sending usage data to the Fivetran server:
- Register your HVR Hub System. You must create a Fivetran account or log in to fivetran.com with an account Administrator role.
- If you are using proxy server to connect HVR Hub System with Fivetran, then ensure to configure the proxy server setup.
If you have multiple HVR Hub Systems, you must register each hub system individually.
This is a usage based subscription plan where HVR can be deployed on-premises, in a virtual private cloud, or air-gapped in your secure network. There is no direct connection between the HVR Hub System and the Fivetran server.
Each quarter (91 days), your consumption/usage is calculated based on the number of MAR across all source (connector) and target (destination) locations in your account. You are required to manually send the consumption data ([MAR]) via email to Fivetran at least once every 91 days from the effective date of the contract.
After purchasing the software, Fivetran technical support team will send you the license file via email. In case you have not received the license file or have questions regarding the license file, contact Fivetran technical support. The license is valid for 6 months or until the end of your contract, whichever is sooner.
In this plan, there are two types of Monthly Active Rows (MAR) data generated:
Daily MAR: The daily MAR data is auto-generated for the hub systems. However, this data is never sent to the Fivetran server. This MAR data is automatically purged after 30 days, so the storage requirement is for one month` worth of data. Automatic purging of daily MAR data can also be disabled using the repository property Metering_Purge_Frequency.
Monthly Aggregate MAR: The monthly aggregate MAR data is also auto-generated for the hub systems. This MAR data is never purged automatically. The size of the monthly aggregate MAR data files is less than 5 MB per month.
For more information about the MAR storage, see Storage Requirements for MAR.
If you have multiple installation of HVR Hub System on a hub machine, each hub system must be individually registered with the Fivetran account.
Manually Send Consumption Data to Fivetranlink
For the Private Deployment plan, to manually send the consumption data (MAR) to Fivetran,
Execute the command hvrlicense in the CLI to generate MAR data file in CSV format. The following command generates MAR data file (CSV) containing consumption/usage details for the last 3 months:
hvrlicense -M .
Example 1 - Generate MAR file with custom name
hvrlicense -M filename.csv
Example 2 - Generate MAR file with data from a specified date and time (e.g., 2022-03-30 18:00:00)
hvrlicense -B '2022-03-20 18:00:00' -M .
Example 3 - Generate MAR file from remote machine
hvrlicense -R http://hubserverhost:4343/ -M .
Send the generated CSV file to the Fivetran email ID - firstname.lastname@example.org.
The consumption/usage data (metadata) that are sent to Fivetran is used for calculating the MAR for your HVR Hub System(s). Following are the two data files that contain consumption/usage details:
The data files that are sent to Fivetran and the frequency of sharing them depend on your pricing plan.
- For the Business Critical plan, the consumption/usage data (MAR) is automatically sent to Fivetran daily. The Hub Snapshot is sent to Fivetran every 30 days.
- For the Private Deployment plan, the user should manually send the consumption/usage data (only MAR) via email to Fivetran at least once a quarter.
The MAR data or snapshot file sent to Fivetran does not contain any data from replicated tables or any confidential or secret data.
The MAR data that HVR sends to Fivetran contains information such as Hub Name, Hub ID, Hub Description, Hub Server URL, Channel Name, Location Name, Location Description, Location Type, Table Name, Base Name, Refresh Events (including the timestamp and metadata), hashes of primary keys changed, and HLL records for Integrate and Refresh jobs.
|Column in MAR File
|Month and year for which the MAR data is applicable.
|Name of the hub.
|Unique identifier of the HVR Hub System.
|Name of the channel.
|Name of the source location.
|Name of the target location.
|Total sum of MAR for the month.
|List of incremental MAR values per day.
|Type (e.g. Oracle, SQL Server) of source location.
|Type (e.g. Oracle, SQL Server) of target location.
|Estimate of the number of tables seen per source-target connection in a channel. The estimate has the same error rate as regular MAR, so usually +- 2%.
|Serialized form of the raw internal HLL structure which is used to track MAR data.
|Serialized form of the raw internal HLL structure which is used to track MAR data.
Storage Requirements for MARlink
HVR uses a special data structure called HyperLogLog (HLL) to track MAR. These data structures are stored on disk in the HVR Hub System, before sending it to Fivetran, in hour granularity buckets per table. HVR uses a hash value from the primary keys to update the data structure. HVR stores these data structures for a maximum of 1 month.
Each data structure is approximately 2 KB. Every day the amount of data stored is 24 (hour) * 2 (KB) * number of active tables; so the disk space needed per day is 48 (KB) * number of active tables.
For Business Critical plan, we recommend allocating a week's worth of storage capacity on the HVR Hub System for storing MAR data. The actual storage allocation required for MAR data on the HVR Hub System is typically only one or two days' worth of data, as HVR automatically purges the MAR data one day after it is sent to Fivetran. Automatic purging of MAR data can also be disabled using the repository property Metering_Purge_Frequency.
For Private Deployment plan, two types of MAR data are generated - daily and monthly. For daily MAR data, we recommend allocating a month's worth of storage capacity on the HVR Hub System. The daily MAR data is automatically purged after 30 days. For monthly aggregate MAR data, we recommend allocating the storage capacity of approximately 5MB per month. The monthly aggregate MAR data is never purged automatically. To purge the monthly aggregate MAR data, you need to manually delete the folders/files available in the hvr_config/metering directory.
The original primary keys can not be retrieved from the HLL data structure. In addition, HVR does not store the primary keys directly, but instead uses their hashes to update this structure. The hashes are irreversible and can not be retrieved.
The hub snapshot contains information about the system configuration or usage such as Channel definitions, Events, Statistics, and Job state. On receiving a request from the Fivetran server, HVR uploads a usage snapshot per hub every 30 days. The hub snapshot is encrypted and stored on AWS S3. The hub snapshot file is stored on the disk of the hub machine prior to sending it to Fivetran. HVR automatically purges it a day after it is sent to Fivetran.
The hub snapshot file contains the following details:
- a week of events
- a week of statistics
- channel definitions
- job state
The snapshot does not contain log files, history, or transaction files.
The automatic generation of hub snapshot is similar to manually executing the command hvrsnapshotsave for saving the snapshot data into a zip file:
hvrsnapshotsave -r "Usage snapshot for account_id my_account_id" -e now-1w -s now-1w -l none -t none -V redact hub_name snapshot_filename.zip