Skip to main content

Can I store migration files in S3? (CLI and Terraform examples)

With Atlas, we advocate for treating migration directories as deployment artifacts resulting from a structured build process. The preferred approach is to push migration directories to the Atlas Schema Registry. In addition to its role as a migration directory storage, the Schema Registry provides a tight integration with the Atlas CLI and the Atlas Cloud UI, allowing you to deploy migrations, visualize schemas over time, review deployment logs and errors, and more.

However, some users prefer to store their migration directories in S3, typically due to internal policies or requirements.

Using dotenv (.env) files with Atlas

A .env file is a simple text file used to store environment variables for applications. It helps developers manage configuration settings, such as database credentials, API keys, and other sensitive information, without hardcoding them into the source code. By using a .env file, developers can keep their codebase clean, secure, and easily configurable across different environments.

For developers who want to use .env files with Atlas, there are two ways users can do so through the standard Atlas HCL configuration file:

Debugging Atlas HCL Using print()

With features such as composite schemas, custom rules, schema linting, and more, the atlas.hcl file can get complex. To validate or debug the file, you can use print statements as you commonly would with other languages.

Managing PostgreSQL Extensions in a Dedicated Migration Process

PostgreSQL extensions are additional modules that extend the functionality of the PostgreSQL database. Examples of popular extensions include PostGIS for geographic data handling, PGVector for vector similarity search, and pgcrypto for cryptographic functions.

Once installed, these extensions provide additional data types and objects that users' schemas can leverage. However, because extensions are managed at the database level (and can only be installed once per database rather than per schema), users may prefer handling these extensions separately from their primary application schema migrations.

For example, if you maintain multiple applications – each with its own schema(s) but several relying on the same PostgreSQL extension – determining which application should handle installation, upgrades, or removal can be challenging. To address this complexity, managing extensions through a dedicated migration process is beneficial. This separation ensures extension-related changes remain isolated.

This guide demonstrates how to use the --include flag (or env.include argument) provided by Atlas to limit migrations specifically to PostgreSQL extensions.