Seed Data per Environment: Shared and Environment-Specific Rows
Atlas supports three ways to manage seed data that differs between environments:
- Fully declarative. The schema and the rows are both declarative, and
atlas schema applyplans the changes to both. Each environment's desired state can be computed at plan time, for example from a template directory or an external program. See Dynamic Data Sources. - Versioned schema, declarative data. The schema stays in one versioned migration directory, and the rows of each environment are managed declaratively: Atlas compares the declared rows with the database and applies the difference, without migration files for the rows.
- Schema and data in one versioned directory per environment. Each directory is generated from the shared schema and rows plus the environment's rows, and is applied and reverted like any other versioned migration directory.
This guide covers the third option.
Data management and the composite_schema data source are available to
Atlas Pro users. To use them, run:
atlas login
Atlas manages rows only in envs with a data block.
Without one, it ignores the INSERT statements in the schema.
Project Layout
Put the tables and the rows every environment needs in schema/, and each environment's rows in
seed/<env>/:
.
├── atlas.hcl
├── schema/
│ └── schema.sql
├── seed/
│ ├── dev/
│ │ └── data.sql
│ └── prod/
│ └── data.sql
└── migrations/
├── dev/
└── prod/
The seed files hold only INSERT statements, for tables defined in schema/:
- schema/schema.sql
- seed/dev/data.sql
- seed/prod/data.sql
CREATE TABLE countries (
code char(2) NOT NULL,
name text NOT NULL,
PRIMARY KEY (code)
);
CREATE TABLE settings (
name text NOT NULL,
value text NOT NULL,
PRIMARY KEY (name)
);
INSERT INTO countries (code, name) VALUES
('US', 'United States'),
('DE', 'Germany');
INSERT INTO settings (name, value) VALUES
('mail_sender', 'sandbox'),
('log_level', 'debug');
INSERT INTO settings (name, value) VALUES
('mail_sender', 'smtp'),
('log_level', 'info');
Configure the Project
data "composite_schema" "app" {
schema "public" {
url = "file://schema"
}
schema "public" {
url = "file://seed/${atlas.env}"
}
}
env {
name = atlas.env
url = getenv("DATABASE_URL")
dev = "docker://postgres/16/dev?search_path=public"
src = data.composite_schema.app.url
migration {
dir = "file://migrations/${atlas.env}"
}
data {
mode = SYNC
include = ["countries", "settings"]
max_rows = 1000
}
}
The second schema "public" block
extends the first with the rows of the
environment passed to --env. The unlabeled env block serves every environment: --env prod reads
seed/prod/ and writes to migrations/prod/. DATABASE_URL must include search_path=public, the
same scope as the dev database.
SYNC also deletes rows that were removed from the seed files. See
Data Configuration for the other modes and attributes.
Generate the Migrations
Run migrate diff for each environment:
atlas migrate diff --env dev initial
atlas migrate diff --env prod initial
The first file in each directory creates the tables and inserts the environment's rows, one INSERT
per table under a -- Seed "<table>" table comment.
After changing schema/ or a seed file, run migrate diff for each environment again. An environment
whose rows and schema did not change gets no new file.
migrate diff loads the rows on the dev database, so rows that don't fit the schema fail there, not
in a deployment. For example, after you add a NOT NULL column to settings, every seed file must
set it.
Apply and Revert
Start a local database for prod and apply its directory:
export DATABASE_URL=$(atlas tool docker --url "docker://postgres/16/demo?search_path=public" --name seed-prod)
atlas migrate apply --env prod
If an environment's database already has the tables and rows, set --baseline to the version of the
first file. Atlas marks that file as applied without running it. See
Baseline migration.
To revert the last applied file, run migrate down:
atlas migrate down --env prod
With SYNC, the computed plan reverts the file's row changes along with its schema changes. See
Down Migrations to review the plan with --dry-run or revert to a version or tag.
To write the revert yourself, add a down.sql section in the
txtar format:
-- atlas:txtar
-- migration.sql --
UPDATE "settings" SET "value" = 'warn' WHERE "name" = 'log_level';
-- down.sql --
UPDATE "settings" SET "value" = 'info' WHERE "name" = 'log_level';
Then update the directory checksum with atlas migrate hash --env prod.
Separate Directories for Schema and Data
A data directory applied on top of a shared schema directory needs more setup:
- Its own revisions table. Set
revisions_schemain itsmigrationblock, for example toatlas_data_revisions. - A baseline version or
--allow-dirtyon its firstmigrate apply, which otherwise fails withconnected database is not cleanbecause the schema directory's revisions exist. - A dev-database baseline that creates its tables.
migrate diffandmigrate downreplay the directory on the dev database, where the tables don't exist otherwise. - Deployment order: apply the schema directory first, and revert the data directory first.
- One env per table. With
SYNC, two envs that include the same table delete each other's rows.