Back to changelog
New
•2 minute read

Databricks: Tag Support

Atlas now manages Databricks tags on schemas, tables, views, materialized views, and metric views, and on the columns of tables, views, and materialized views, declared with the tags attribute of each object.

Atlas now inspects, diffs, and plans Databricks tags. A tag is a key/value pair, declared in the tags map of the object it labels. These objects take tags:

A tag is not an object of its own, so there is no tag block. Atlas sets the tags of an object right after it creates the object.

Example

The project file uses catalog scope on both URLs:

atlas.hcl
env "local" {  url = "databricks://...?catalog=atlas_tag_test"  dev = "databricks://...?catalog=atlas_tag_test_dev"  migration {    dir     = "file://migrations"    tx_mode = "none"  }  schema {    src = "file://schema.hcl"  }}

The desired state tags the sales schema, a table, a view, a materialized view, a metric view, and the email column of the table, the view, and the materialized view:

schema.hcl
schema "default" {}
schema "sales" {  tags = {    owner = "crm"  }}
table "customers" {  schema = schema.sales  column "customer_id" {    null = false    type = BIGINT  }  column "email" {    null = true    type = STRING    tags = {      pii_type = "email"    }  }  tags = {    data_classification = "confidential"  }}
view "customer_contacts" {  schema = schema.sales  column "customer_id" {    null = false    type = BIGINT  }  column "email" {    null = true    type = STRING    tags = {      pii_type = "email"    }  }  as = "SELECT customer_id, email FROM sales.customers"  tags = {    audience = "support"  }}
materialized "marketing_audience" {  schema = schema.sales  column "email" {    null = true    type = STRING    tags = {      pii_type = "email"    }  }  as = "SELECT email FROM sales.customers"  tags = {    owner = "marketing"  }}
view "customer_metrics" {  schema  = schema.sales  metrics = true  as      = <<-YAML  version: 0.1  source: sales.customers  measures:    - name: customer_count      expr: COUNT(1)  YAML  depends_on = [table.customers]  tags = {    domain = "reporting"  }}

Run the diff:

atlas migrate diff --env local

Atlas sets the tags of each object right after creating it:

migrations/20261111145744.sql
-- Add new schema named "sales"CREATE SCHEMA `sales`;-- Set the tags of schema "sales"ALTER SCHEMA `sales` SET TAGS ('owner' = 'crm');-- Create "customers" tableCREATE TABLE IF NOT EXISTS `sales`.`customers` (  `customer_id` BIGINT NOT NULL,  `email` STRING);-- Set the tags of "customers" tableALTER TABLE `sales`.`customers` SET TAGS ('data_classification' = 'confidential');-- Set the tags of column "email" in "customers" tableALTER TABLE `sales`.`customers` ALTER COLUMN `email` SET TAGS ('pii_type' = 'email');-- Create view named "customer_contacts"CREATE OR REPLACE VIEW `sales`.`customer_contacts` (  `customer_id`,  `email`)WITH SCHEMA COMPENSATION AS SELECT customer_id, email FROM sales.customers;-- Set the tags of view "customer_contacts"ALTER VIEW `sales`.`customer_contacts` SET TAGS ('audience' = 'support');-- Set the tags of column "email" in view "customer_contacts"SET TAG ON COLUMN `sales`.`customer_contacts`.`email` `pii_type` = `email`;-- Create view named "customer_metrics"CREATE OR REPLACE VIEW `sales`.`customer_metrics`WITH METRICSLANGUAGE YAMLAS $$version: 0.1source: sales.customersmeasures:  - name: customer_count    expr: COUNT(1)$$;-- Set the tags of view "customer_metrics"ALTER VIEW `sales`.`customer_metrics` SET TAGS ('domain' = 'reporting');-- Create view named "marketing_audience"CREATE OR REPLACE MATERIALIZED VIEW `sales`.`marketing_audience` (  `email` STRING) AS SELECT email FROM sales.customers;-- Set the tags of view "marketing_audience"ALTER MATERIALIZED VIEW `sales`.`marketing_audience` SET TAGS ('owner' = 'marketing');-- Set the tags of column "email" in view "marketing_audience"ALTER MATERIALIZED VIEW `sales`.`marketing_audience` ALTER COLUMN `email` SET TAGS ('pii_type' = 'email');
Note: atlas schema apply unsets tags that the desired state does not declare, including tags set outside Atlas. Declare the tags you want to keep in HCL first.

Getting Started

The Databricks driver is part of Atlas Pro. Run atlas login to get started, and see the Databricks HCL reference.

featuredatabrickstagsmetadataatlas pro