How to Reduce BigQuery Integration Costs

Question

Running MERGE queries against large BigQuery tables results in high costs. How can I reduce integration costs?

Environment

HVR 6

Answer

You can reduce BigQuery MERGE costs by using key ranges, which limit MERGE operations to a subset of rows instead of scanning the entire table. To configure HVR MERGE queries to use key ranges, add the environment variable HVR_BURST_KEY_RANGE_KEYS with a value of 2.