Skip to main content

Project Configuration

Project Files​

Project files provide a convenient way to describe and interact with multiple environments when working with Atlas. A project file is a file named atlas.hcl and contains one or more env blocks. For example:

// Define an environment named "local"
env "local" {
  // Declare where the schema definition resides.
  // Also supported: ["multi.hcl", "file.hcl"].
  src = "./project/schema.hcl"

  // Define the URL of the database which is managed
  // in this environment.
  url = "mysql://user:pass@localhost:3306/schema"

  // Define the URL of the Dev Database for this environment
  // See: https://atlasgo.io/concepts/dev-database
  dev = "docker://mysql/8/schema"
}

env "dev" {
  // ... a different env
}

Once defined, a project's environment can be worked against using the --env flag. For example:

atlas schema apply --env local

Will run the schema apply command against the database that is defined for the local environment.

Unlabeled env blocks

It is possible to define an env block whose name is dynamically set during command execution using the --env flag. This is useful when multiple environments share the same configuration and the arguments are dynamically set during execution:

env {
  name = atlas.env
  url = var.url
  format {
    migrate {
      apply = format(
        "{{ json . | json_merge %q }}",
        jsonencode({
          EnvName : atlas.Env
        })
      )
    }
  }
}

Projects with Versioned Migrations​

Environments may declare a migration block to configure how versioned migrations work in the specific environment:

env "local" {
    // ..
    migration {
        // URL where the migration directory resides. Only filesystem directories
        // are currently supported but more options will be added in the future.
        dir = "file://migrations"
        // An optional format of the migration directory:
        // atlas (default) | flyway | liquibase | goose | golang-migrate | dbmate
        format = atlas
    }
}

Once defined, migrate commands can use this configuration, for example:

atlas migrate validate --env local

Will run the migrate validate command against the Dev Database defined in the local environment.

Passing Input Values​

Project files may pass input values to variables defined in the Atlas schema of the environment. To do this simply provide additional attributes to the environment block:

atlas.hcl
env "local" {
  url = "sqlite://test?mode=memory&_fk=1"
  src = "schema.hcl"

  // Other attributes are passed as input values to "schema.hcl":
  tenant = "rotemtam"
}

These values can then be consumed by variables defined in the schema file:

schema.hcl
variable "tenant" {
  type = string
}
schema "main" {
  name = var.tenant
}

Project Input Variables​

Project files may also declare input variables that can be supplied to the CLI at runtime. For example:

variable "tenant" {
  type = string
}

env "local" {
  url = "sqlite://test?mode=memory&_fk=1"
  src = "schema.hcl"

  // The value for "tenant" is determined by the user at runtime.
  tenant = var.tenant
}

To set the value for this variable at runtime, use the --var flag:

atlas schema apply --env local --var tenant=rotemtam

It is worth mentioning that when running Atlas commands within a project using the --env flag, all input values supplied at the command-line are passed only to the project file, and not propagated automatically to children schema files. This is done with the purpose of creating an explicit contract between the environment and the schema file.

Schema Arguments and Attributes​

Project configuration files support different types of blocks.

Input Variables​

Project files support defining input variables that can be injected through the CLI, read more here.

  • type - The type constraint of a variable.
  • default - Define if the variable is optional by setting its default value.
variable "tenants" {
  type = list(string)
}

variable "url" {
  type    = string
  default = "mysql://root:pass@localhost:3306/"
}

env "local" {
  // Reference an input variable.
  url = var.url
}

Local Values​

The locals block allows defining a list of local variables that can be reused multiple times in the project.

locals {
  tenants  = ["tenant_1", "tenant_2"]
  base_url = "mysql://${var.user}:${var.pass}@${var.addr}"
  
  // Reference local values. 
  db1_url  = "${local.base_url}/db1"
  db2_url  = "${local.base_url}/db2"
}

Data Sources​

Data sources enable users to retrieve information stored in an external service or database. The currently supported data sources are: sql, runtimevar and template_dir.

Data source: sql​

Arguments​
  • url - The URL of the target database.
  • query - Query to execute.
  • args - Optional arguments for any placeholder parameters in the query.
Attributes​
  • count - The number of returned rows.
  • values - The returned values. e.g. list(string).
  • value - The first value in the list, or nil.
data "sql" "tenants" {
  url = var.url
  query = <<EOS
SELECT `schema_name`
  FROM `information_schema`.`schemata`
  WHERE `schema_name` LIKE ?
EOS
  args = [var.pattern]
}

env "prod" {
  // Reference a data source.
  for_each = toset(data.sql.tenants.values)
  url      = urlsetpath(var.url, each.value)
}

Data source: runtimevar​

Arguments​
  • url - The URL identifies the variable. See, the CDK documentation for more information.
Attributes​
  • The loaded variable is a string type with no attributes.

The data source uses Application Default Credentials by default; if you have authenticated via gcloud auth application-default login, it will use those credentials.

atlas.hcl
data "runtimevar" "pass" {
  url = "gcpsecretmanager://projects/<project>/secrets/<secret>"
}

env "dev" {
  src = "schema.hcl"
  url = "mysql://root:${data.runtimevar.pass}@host:3306/database"
}

Usage example​

gcloud auth application-default login
atlas schema apply --env dev
GOOGLE_APPLICATION_CREDENTIALS="/path/to/credentials.json" atlas schema apply --env dev

Data source: template_dir​

The template_dir data source renders a migration directory from a template directory. It does this by parsing the entire directory as Go templates, executing top-level (template) files that have the .sql file extension, and generating an in-memory migration directory from them.

Arguments​
  • path - A path to the template directory.
Attributes​
  • url - A URL to the generated migration directory.
atlas.hcl
variable "path" {
  type        = string
  description = "A path to the template directory"
}

data "template_dir" "migrations" {
  path = var.path
  vars = {
    Key1 = "value1"
    Key2 = "value2"
    // Pass the --env value as a template variable.
    Env  = atlas.env
  }
}

env "dev" {
  url = var.url
  migration {
    dir = data.template_dir.migrations.url
  }
}

Environments​

The env block defines an environment block that can be selected by using the --env flag.

Arguments​

  • for_each - A meta-argument that accepts a map or a set of strings and is used to compute an env instance for each set or map item. See the example below.

  • url - The URL of the target database.

  • dev - The URL of the Dev Database.

  • schemas - A list of strings defines the schemas that Atlas manages.

  • exclude - A list of strings defines glob patterns used to filter resources on inspection.

  • migration - A block defines the migration configuration of the env.

    • dir - The URL to the migration directory.
    • baseline - An optional version to start the migration history from. Read more here.
    • lock_timeout - An optional timeout to wait for a database lock to be released. Defaults to 10s.
    • revisions_schema - An optional name to control the schema that the revisions table resides in.

  • format - A block defines the formatting configuration of the env per command (previously named log).

    • migrate
      • apply - Set custom formatting for migrate apply.
      • lint - Set custom formatting for migrate lint.
      • status - Set custom formatting for migrate status.
    • schema
      • apply - Set custom formatting for schema apply.
      • diff - Set custom formatting for schema diff.

  • lint - A block defines the migration linting configuration of the env.

    • format - Override the --format flag by setting a custom logging for migrate lint (previously named log).
    • lastest - A number configures the --latest option.
    • git.base - A run analysis against the base Git branch.
    • git.dir - A path to the repository working directory.
Multi Environment Example​

Atlas adopts the for_each meta-argument that Terraform uses for env blocks. Setting the for_each argument will compute an env block for each item in the provided value. Note that for_each accepts a map or a set of strings.

atlas.hcl
env "prod" {
  for_each = toset(data.sql.tenants.values)
  url      = urlsetpath(var.url, each.value)
  migration {
    dir = "file://migrations"
  }
  format {
    migrate {
      apply = format(
        "{{ json . | json_merge %q }}",
        jsonencode({
          Tenant : each.value
        })
      )
    }
  }
}

Configure Migration Linting​

Project files may declare lint blocks to configure how migration linting runs in a specific environment or globally.

lint {
  destructive {
    // By default, destructive changes cause migration linting to error
    // on exit (code 1). Setting `error` to false disables this behavior.
    error = false
  }
  // Custom logging can be enabled using the `format` attribute (previously named `log`).
  format = <<EOS
{{- range $f := .Files }}
    {{- json $f }}
{{- end }}
EOS
}

env "local" {
  // Define a specific migration linting config for this environment.
  // This block inherits and overrides all attributes of the global config.
  lint {
    latest = 1
  }
}

env "ci" {
  lint {
    git {
      base = "master"
      // An optional attribute for setting the working
      // directory of the git command (-C flag).
      dir = "<path>"
    }
  }
}