Schema as Code: Directory Structure for HCL Files

Question

How do I configure Atlas to load the database schema from nested directories with HCL files?

 /project
 ├── main.sql
 ├── extensions
 └── schemas
     └── public
         ├── public.sql
         ├── tables
         │   ├── pets.sql
         │   ├── posts.sql
         │   ├── profiles.sql
         │   └── user_groups.sql
         ├── functions/
         └── types/

Answer

By default, the file:// format accepts a file or directory, for example: file://schema.pg.hcl or file://schema/. However, using Atlas data-sources and the fileset function (which accepts a glob pattern), you can configure Atlas to load the HCL schema files from nested directories. For example:

atlas.hcl

data "hcl_schema" "app" {
  paths = fileset("schemas/**/*.hcl")
}

env "local" {
  src = data.hcl_schema.app.url
  dev = "<URL_TO_DEV_DATABASE>"
}

Note, the ** pattern tells Atlas to search through all subdirectories, no matter how deeply nested. The .hcl file extension limits the matches to HCL files only.

To verify the correctness of this config, run the following:

atlas schema inspect --env local --url env://src

Note: The env:// scheme lets you reference attributes defined in the current environment. In this case, env://src points to the URL from the data.hcl_schema.app data source.

Atlas Schema Import

Instead of creating schema files manually, you can let Atlas import your database schema to code using the atlas schema inspect command. Below are a few examples:

# Create nested folders by schema and object type (default)
atlas schema inspect -u '<database-url>' \
  --format '{{ hcl . | split | write "project" }}'

# Create one file per schema
atlas schema inspect -u '<database-url>' \
  --format '{{ hcl . | split "schema" | write "project" }}'

# Create one file per object type (functions, tables, etc.)
atlas schema inspect -u '<database-url>' \
  --format '{{ hcl . | split "type" | write "project" }}'

See the schema inspect documentation for more details.

---

IDE Support

To make working with HCL schemas easier, Atlas provides official plugins for popular IDEs. These plugins help you navigate, edit, format, and cross-reference objects across multiple files in your project.

Module Paths in JetBrains

The Atlas plugin for JetBrains supports module paths, allowing you to treat specific directories as module roots. This enables Atlas LS (language-server) to recognize and work with HCL files across nested directories.

To configure a module path for your project, follow these steps:

1. Open Settings > Languages & Frameworks > Atlas
2. Under Module Paths, click the + button to add a folder
3. Enter the absolute path to the directory you want to treat as a module root