Old Databricks SQL Warehouse Still Shows Activity After Switching Warehouses
Issue
After you update your HVR channel to use a new Databricks SQL warehouse, the old warehouse still receives queries, including metadata-related INSERT and SELECT statements.
Environment
- HVR 6
- Target: Databricks
Resolution
To resolve this issue:
- Suspend the Capture job.
- Wait for the Integrate job to finish processing all pending changes, then suspend it.
- In the Databricks location settings, update the HTTP PATH field to point to the new SQL warehouse. For more information, see Location Connection for Databricks.
- On the Channel Details page, run Activate Replication with the Jobs and Table Enrollment options selected.
- If applicable, recreate any Compare or Refresh jobs to use the updated settings.
- Use Databricks monitoring tools to confirm that the old SQL warehouse no longer receives queries.
Cause
This issue occurs when you update the Databricks location's HTTP path without reactivating replication with the Jobs and Table Enrollment components selected. When this happens, HVR continues to use the previously activated settings and sends queries to the old SQL warehouse.