One post tagged with "schema linting"

Hey everyone!

Welcome to the second Atlas release of 2025, v0.31! We're excited to share the latest updates and improvements with you. In this release you will find:

• Custom schema rules: You can now define custom rules for your database schema and have Atlas enforce them for you during CI.
• pgvector support: We've added support for managing schemas for projects that use the LLM-based pgvector extension.
• Drift detection: It is now simpler to set up drift detection checks to alert you when a target database isn't in the state it's supposed to be in.
• Multi-project ER Diagrams: you can now create composite ER diagrams that stitch schema objects from multiple Atlas projects.

Enforce custom schema rules​

atlas login

Atlas now supports the definition of custom schema rules that can be enforced during local development or CI/CD pipelines. This feature is particularly useful for teams that want to ensure that their database schema adheres to specific conventions or best practices, such as:

• "Require columns to be not null or have a default value"
• "Require foreign keys to reference primary keys of the target table and use ON DELETE CASCADE"
• "Require an index on columns used by foreign keys"
• ... and many more

The linted schema can be defined in any supported Atlas schema format, such as HCL, SQL, ORM, a URL to a database, or a composition of multiple schema sources.

Here's a simple example of a schema rule file:

# A predicate that checks if a column is not null or has a default value.
predicate "column" "not_null_or_have_default" {
  or {
    default {
      ne = null
    }
    null {
      eq = false
    }
  }
}

rule "schema" "disallow-null-columns" {
  description = "require columns to be not null or have a default value"
  table {
    column {
      assert {
        predicate = predicate.column.not_null_or_have_default
        message   = "column ${self.name} must be not null or have a default value"
      }
    }
  }
}

The example above defines a rule that checks if all columns in a schema are either not null or have a default value.

To use it, you would add the rule file to you atlas.hcl configuration:

lint {
  rule "hcl" "table-policies" {
    src = ["schema.rule.hcl"]
  }
}

env "local" {
  src = "file://schema.lt.hcl"
  dev = "sqlite://?mode=memory"
}

Suppose we used the following schema:

schema "main" {
}

table "hello" {
  schema = schema.main
  column "id" {
    type = int
    null = true
  }
}

Next, Atlas offers two ways to invoke the linting process:

1. Using the migrate lint command in the Atlas CLI. This lints only new changes that are introduced in the linted changeset - typically the new migration files in the current PR.
2. Using the newly added schema lint command in the Atlas CLI. This lints the entire schema, including all tables and columns, and can be used to enforce schema rules during CI/CD pipelines.

Let's use the schema lint command to lint the schema:

atlas schema lint --env local -u env://src

The command will output the following error:

Analyzing schema objects (3 objects in total):

  rule "disallow-null-columns":
    -- schema.lt.hcl:7: column id must be not null or have a default value

  -------------------------
  -- 27.791µs
  -- 1 diagnostic

Great! Atlas has detected that the id column in the hello table is nullable and has raised an error. You can now update the schema to fix the error and rerun the linting process.

For more information on defining custom schema rules, check the docs for custom schema rules.

Manage schemas for LLM-based projects with pgvector​

pgvector has become the de-facto standard for storing high-dimensional vectors in PostgreSQL. It is widely used in LLM-based projects that require fast similarity search for large datasets. Use cases such as recommendation systems and RAG (Retrieval-Augmented Generation) pipelines rely on pgvector to store embeddings and perform similarity searches.

In this release, we've added support for managing schemas for projects that use pgvector. You can now use Atlas to automate schema migrations for your pgvector projects.

Suppose you are building a RAG pipeline that uses OpenAI embeddings to generate responses to user queries. Your schema might look like this:

schema "public" {
}

extension "vector" {
  schema  = schema.public
}

table "content" {
  schema = schema.public
  column "id" {
    type = int
  }
  column "text" {
    type = text
  }
  column "embedding" {
    type = sql("vector(1536)")
  }
  primary_key {
    columns = [ column.id ]
  }
  index "idx_emb" {
    type = "IVFFlat"
    on {
      column = column.embedding
      ops    = "vector_l2_ops"
    }
    storage_params {
      lists = 100
    }
  }
}

Notice three key elements in the schema definition:

1. The extension block defines the vector extension, enabling pgvector support for the schema.
2. The column block defines a column named embedding with the vector(1536) type, which stores 1536-dimensional vectors.
3. The index block defines an index on the embedding column, which uses the IVFFlat method for similarity search.

For more information on defining pgvector powered indexes, check the docs for index storage parameters.

Simpler setup for Drift Detection​

Schema drift, which refers to a state where there is a difference between the intended database schema and its actual state, can lead to significant issues such as application errors, deployment failures, and even service outages.

In this release, we've simplified the process of setting up drift checks using the Atlas Cloud UI.

If you already use Atlas to run migrations, enable drift detection to stay on top of any manual schema changes that may cause a painful outage.

Watch the demo:

Multi-project ER Diagrams​

Many teams using Atlas adopt microservices-oriented architectures to build their applications. This approach allows them to develop, scale, and deploy services independently, but it also introduces a unique challenge: database schemas are often spread across multiple logical (and sometimes physical) databases.

While these architectures excel at modularity (by decoupling teams from one another) they make it harder to visualize, manage, and reason about the overall data model. Traditional Entity-Relationship (ER) diagrams are designed for monolithic databases, where all relationships and constraints exist within a single system. However, in a microservices setup, relationships between entities often span across services, sometimes without explicit foreign keys or enforced constraints.

To help teams overcome this challenge, we are happy to announce today the availability of composite, multi-project ER diagrams in Atlas Cloud. Using this feature, teams can create diagrams that are composed of database objects from multiple Atlas projects.

See it in action:

Related Object Selection​

Over years of development, database schemas tend to become complex and may grow to encompass hundreds or even thousands of objects. Using the Atlas data modeling tool, users can curate smaller subgraphs of the schema, named “saved filters”, which can serve as documentation for a certain aspect of the system. To make this process easier, our team has added a new context menu that makes it easier to include connected objects (e.g tables that are connected via a Foreign Key) in saved filters.

Watch the example:

New Trust Center​

At Ariga, the company behind Atlas, we are committed to meeting the evolving security and compliance needs of our customers. As part this commitment we've consolidated our legal and compliance documentation into a single page in our new Trust Center.

Wrapping Up​

We hope you enjoy the new features and improvements. As always, we would love to hear your feedback and suggestions on our Discord server.