Atlas now manages Databricks tags on schemas, tables, views, materialized views, and metric views, and on the columns of tables, views, and materialized views, declared with the tags attribute of each object.
Atlas now inspects, diffs, and plans Databricks tags. A tag is a key/value pair, declared in the tags map of the object it labels. These objects take tags:
- schema blocks
- table blocks and their columns
- view blocks and their columns
- materialized blocks and their columns
- metric views, on the view itself
A tag is not an object of its own, so there is no tag block. Atlas sets the tags of an object right after it creates the object.
Example
The project file uses catalog scope on both URLs:
env "local" { url = "databricks://...?catalog=atlas_tag_test" dev = "databricks://...?catalog=atlas_tag_test_dev" migration { dir = "file://migrations" tx_mode = "none" } schema { src = "file://schema.hcl" }}
The desired state tags the sales schema, a table, a view, a materialized view, a metric view, and the email column of the table, the view, and the materialized view:
schema "default" {} schema "sales" { tags = { owner = "crm" }} table "customers" { schema = schema.sales column "customer_id" { null = false type = BIGINT } column "email" { null = true type = STRING tags = { pii_type = "email" } } tags = { data_classification = "confidential" }} view "customer_contacts" { schema = schema.sales column "customer_id" { null = false type = BIGINT } column "email" { null = true type = STRING tags = { pii_type = "email" } } as = "SELECT customer_id, email FROM sales.customers" tags = { audience = "support" }} materialized "marketing_audience" { schema = schema.sales column "email" { null = true type = STRING tags = { pii_type = "email" } } as = "SELECT email FROM sales.customers" tags = { owner = "marketing" }} view "customer_metrics" { schema = schema.sales metrics = true as = <<-YAML version: 0.1 source: sales.customers measures: - name: customer_count expr: COUNT(1) YAML depends_on = [table.customers] tags = { domain = "reporting" }}
Run the diff:
atlas migrate diff --env localAtlas sets the tags of each object right after creating it:
-- Add new schema named "sales"CREATE SCHEMA `sales`;-- Set the tags of schema "sales"ALTER SCHEMA `sales` SET TAGS ('owner' = 'crm');-- Create "customers" tableCREATE TABLE IF NOT EXISTS `sales`.`customers` ( `customer_id` BIGINT NOT NULL, `email` STRING);-- Set the tags of "customers" tableALTER TABLE `sales`.`customers` SET TAGS ('data_classification' = 'confidential');-- Set the tags of column "email" in "customers" tableALTER TABLE `sales`.`customers` ALTER COLUMN `email` SET TAGS ('pii_type' = 'email');-- Create view named "customer_contacts"CREATE OR REPLACE VIEW `sales`.`customer_contacts` ( `customer_id`, `email`)WITH SCHEMA COMPENSATION AS SELECT customer_id, email FROM sales.customers;-- Set the tags of view "customer_contacts"ALTER VIEW `sales`.`customer_contacts` SET TAGS ('audience' = 'support');-- Set the tags of column "email" in view "customer_contacts"SET TAG ON COLUMN `sales`.`customer_contacts`.`email` `pii_type` = `email`;-- Create view named "customer_metrics"CREATE OR REPLACE VIEW `sales`.`customer_metrics`WITH METRICSLANGUAGE YAMLAS $$version: 0.1source: sales.customersmeasures: - name: customer_count expr: COUNT(1)$$;-- Set the tags of view "customer_metrics"ALTER VIEW `sales`.`customer_metrics` SET TAGS ('domain' = 'reporting');-- Create view named "marketing_audience"CREATE OR REPLACE MATERIALIZED VIEW `sales`.`marketing_audience` ( `email` STRING) AS SELECT email FROM sales.customers;-- Set the tags of view "marketing_audience"ALTER MATERIALIZED VIEW `sales`.`marketing_audience` SET TAGS ('owner' = 'marketing');-- Set the tags of column "email" in view "marketing_audience"ALTER MATERIALIZED VIEW `sales`.`marketing_audience` ALTER COLUMN `email` SET TAGS ('pii_type' = 'email');
Getting Started
The Databricks driver is part of Atlas Pro. Run atlas login to get started, and see the Databricks HCL reference.