How Can I Enable Deletion Vectors for My Databricks Tables?
Question
How can I enable deletion vectors for my Databricks tables?
Environment
Destination: Databricks
Answer
You must consider the following prerequisite and limitations before you enable deletion vectors for your tables:
- Databricks recommends that you use Databricks Runtime 14.3 LTS or above to write tables with deletion vectors to leverage all optimizations. For more information, see Databricks' documentation.
- You can enable deletion vectors for Materialized views, but to disable deletion vectors for a Materialized view, you must drop the Materialized view and recreate it.
- You cannot generate a manifest file for a table with deletion vectors. To generate a manifest, run the
REORG TABLE ... APPLY (PURGE)
command and ensure that no concurrent write operations are running. - You cannot incrementally generate manifest files for a table with deletion vectors.
To enable deletion vectors for your Databricks tables, do the following:
Pause your connector.
Run the following SQL query for the tables where you want to enable deletion vectors:
ALTER TABLE <table-name> SET TBLPROPERTIES ('delta.enableDeletionVectors' = true);
Unpause your connector.