Skip to main content

AWS RDS Extensions in the Atlas Dev Database

Your application schema uses one of RDS PostgreSQL's AWS extensions: say a trigger that calls aws_lambda.invoke, or a routine that loads data through aws_s3.table_import_from_s3. It works on RDS. But any Atlas command that relies on a dev database, like migrate diff, schema diff, or schema plan, can't compile that schema, because a vanilla postgres Docker image doesn't have those extensions.

This is the question RDS users ask us most:

How do you handle the AWS extensions (aws_commons, aws_lambda) when they aren't available in the dev Docker image?

The answer is to mock them: give the dev database just enough of each extension to compile your schema, while the real implementations keep running on RDS. We'll use the AWS extensions as the worked example, but the pattern fits any cloud-proprietary extension with no public binary.

Why the dev database can't compile it​

Most Atlas commands need a dev database: a temporary, isolated database Atlas spins up to compile and validate your schema before planning any change. Parsing the SQL is straightforward. The harder part is confirming that a default, an expression, or a function call actually works, and only a real PostgreSQL engine running your target's exact version can do that.

RDS PostgreSQL ships extensions that live nowhere else: aws_commons, aws_lambda, and aws_s3. They let your schema reach into AWS: aws_lambda.invoke in a trigger, an aws_s3 import, and so on. But they have no public binary. You can't CREATE EXTENSION aws_lambda on a vanilla postgres image, because the code isn't there. So when Atlas loads your schema onto the dev database and reaches the first reference, PostgreSQL refuses it.

There are two ways around this. The heavy one is to point Atlas at a database that already has the real extensions installed: a dev block can target any running PostgreSQL, so many teams use a second RDS instance. It works, but it's slow to start and costs money to keep running.

There's a lighter way. At plan time Atlas never runs aws_lambda.invoke; it only needs the function to exist so the trigger, view, or default around it compiles. That means you don't need the real extension on any database. Instead, hand a throwaway Docker container a set of stubs: functions with the right signatures but empty bodies, just enough to compile. That's the default this guide uses: no second RDS instance to provision, nothing to install.

Prerequisites​

  1. Docker, to run the ephemeral dev-database container
  2. Atlas installed on your machine:

To download and install the latest release of the Atlas CLI, simply run the following in your terminal:

curl -sSf https://atlasgo.sh | sh
Atlas Pro Feature

The docker and dev blocks used throughout this guide are available to Atlas Pro users. To use them, run:

atlas login

This guide builds the following project layout:

.
├── atlas.hcl        
├── schema.sql       # your desired schema (calls the extension functions)
├── stubs/
│   └── aws.sql      # the dev-database stubs (from this guide)
└── migrations/

Stub the AWS extensions​

Here's a schema that leans on all three extensions: a trigger that invokes a Lambda on insert, and a helper that bulk-loads rows from S3:

schema.sql
CREATE TABLE "public"."events" (
    "id"         bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
    "payload"    jsonb NOT NULL,
    "created_at" timestamptz NOT NULL DEFAULT now()
);

-- On insert, invoke a Lambda function via the aws_lambda extension.
CREATE FUNCTION "public"."on_event_insert"() RETURNS trigger
    LANGUAGE plpgsql AS $$
BEGIN
    PERFORM aws_lambda.invoke(
        aws_commons.create_lambda_function_arn('process-event', 'us-east-1'),
        to_jsonb(NEW)
    );
    RETURN NEW;
END;
$$;

CREATE TRIGGER "trg_on_event_insert"
    AFTER INSERT ON "public"."events"
    FOR EACH ROW EXECUTE FUNCTION "public"."on_event_insert"();

-- Bulk-load rows from S3 via the aws_s3 extension.
CREATE FUNCTION "public"."import_events"("bucket" text, "path" text) RETURNS text
    LANGUAGE sql AS $$
    SELECT aws_s3.table_import_from_s3(
        'public.events', '', '(format csv)',
        aws_commons.create_s3_uri("bucket", "path", 'us-east-1')
    );
$$;

Atlas compiles this on a dev database before it plans anything. On a vanilla postgres image the AWS extensions are missing, so the compile stops at the first reference:

Error: schema.sql:24: pq: schema "aws_commons" does not exist at position 5:9 (3F000)

The fix is to run a baseline of stubs when the dev database starts, so those references resolve. The baseline attribute of the docker block is an init script Atlas runs right after the container starts, meant for exactly "objects not managed by you, such as PostgreSQL extensions or external functions used in your schema." Point it at a stub file:

atlas.hcl
docker "postgres" "dev" {
  image = "postgres:18" // match your RDS PostgreSQL major version
  baseline = file("stubs/aws.sql")
}

env "local" {
  src = "file://schema.sql"
  dev = docker.postgres.dev.url
  migration {
    dir = "file://migrations"
  }
}

Each stub recreates a function with its real signature but an empty body, so PostgreSQL accepts the call without the real extension ever being installed. Take aws_commons.create_s3_uri: RDS's version builds a URI struct from a bucket and path, but the stub just returns NULL with the same signature and return type:

CREATE FUNCTION aws_commons.create_s3_uri(bucket text, file_path text, region text DEFAULT NULL)
  RETURNS aws_commons._s3_uri_1 LANGUAGE sql AS $$ SELECT NULL::aws_commons._s3_uri_1 $$;

The full file applies the same idea to all three extensions:

stubs/aws.sql
CREATE SCHEMA IF NOT EXISTS aws_commons;
CREATE SCHEMA IF NOT EXISTS aws_lambda;
CREATE SCHEMA IF NOT EXISTS aws_s3;

-- Composite types (array types are auto-created). Create BEFORE the functions.
CREATE TYPE aws_commons._s3_uri_1              AS (bucket text, file_path text, region text);
CREATE TYPE aws_commons._lambda_function_arn_1 AS (function_name text, region text);
CREATE TYPE aws_commons._aws_credentials_1     AS (access_key text, secret_key text, session_token text);

-- aws_commons constructors
CREATE FUNCTION aws_commons.create_s3_uri(bucket text, file_path text, region text DEFAULT NULL)
  RETURNS aws_commons._s3_uri_1 LANGUAGE sql AS $$ SELECT NULL::aws_commons._s3_uri_1 $$;
CREATE FUNCTION aws_commons.create_lambda_function_arn(function_name text, region text DEFAULT NULL)
  RETURNS aws_commons._lambda_function_arn_1 LANGUAGE sql AS $$ SELECT NULL::aws_commons._lambda_function_arn_1 $$;
CREATE FUNCTION aws_commons.create_aws_credentials(access_key text, secret_key text, session_token text)
  RETURNS aws_commons._aws_credentials_1 LANGUAGE sql AS $$ SELECT NULL::aws_commons._aws_credentials_1 $$;

-- aws_lambda.invoke
CREATE FUNCTION aws_lambda.invoke(function_name aws_commons._lambda_function_arn_1, payload jsonb,
  invocation_type text DEFAULT 'RequestResponse', log_type text DEFAULT 'None', context jsonb DEFAULT NULL,
  qualifier varchar DEFAULT NULL, OUT status_code integer, OUT payload jsonb, OUT executed_version text, OUT log_result text)
  LANGUAGE sql AS $$ SELECT NULL::integer, NULL::jsonb, NULL::text, NULL::text $$;
CREATE FUNCTION aws_lambda.invoke(function_name aws_commons._lambda_function_arn_1, payload json,
  invocation_type text DEFAULT 'RequestResponse', log_type text DEFAULT 'None', context json DEFAULT NULL,
  qualifier varchar DEFAULT NULL, OUT status_code integer, OUT payload json, OUT executed_version text, OUT log_result text)
  LANGUAGE sql AS $$ SELECT NULL::integer, NULL::json, NULL::text, NULL::text $$;
CREATE FUNCTION aws_lambda.invoke(function_name text, payload jsonb, region text DEFAULT NULL,
  invocation_type text DEFAULT 'RequestResponse', log_type text DEFAULT 'None', context jsonb DEFAULT NULL,
  qualifier varchar DEFAULT NULL, OUT status_code integer, OUT payload jsonb, OUT executed_version text, OUT log_result text)
  LANGUAGE sql AS $$ SELECT NULL::integer, NULL::jsonb, NULL::text, NULL::text $$;
CREATE FUNCTION aws_lambda.invoke(function_name text, payload json, region text DEFAULT NULL,
  invocation_type text DEFAULT 'RequestResponse', log_type text DEFAULT 'None', context json DEFAULT NULL,
  qualifier varchar DEFAULT NULL, OUT status_code integer, OUT payload json, OUT executed_version text, OUT log_result text)
  LANGUAGE sql AS $$ SELECT NULL::integer, NULL::json, NULL::text, NULL::text $$;

-- aws_s3 import/export (both overloads each)
CREATE FUNCTION aws_s3.table_import_from_s3(table_name text, column_list text, options text,
  bucket text, file_path text, region text, access_key text DEFAULT NULL, secret_key text DEFAULT NULL, session_token text DEFAULT NULL)
  RETURNS text LANGUAGE sql AS $$ SELECT NULL::text $$;
CREATE FUNCTION aws_s3.table_import_from_s3(table_name text, column_list text, options text,
  s3_info aws_commons._s3_uri_1, credentials aws_commons._aws_credentials_1 DEFAULT NULL)
  RETURNS text LANGUAGE sql AS $$ SELECT NULL::text $$;
CREATE FUNCTION aws_s3.query_export_to_s3(query text, bucket text, file_path text, region text DEFAULT NULL,
  options text DEFAULT NULL, kms_key text DEFAULT NULL, OUT rows_uploaded bigint, OUT files_uploaded bigint, OUT bytes_uploaded bigint)
  LANGUAGE sql AS $$ SELECT NULL::bigint, NULL::bigint, NULL::bigint $$;
CREATE FUNCTION aws_s3.query_export_to_s3(query text, s3_info aws_commons._s3_uri_1 DEFAULT NULL,
  options text DEFAULT NULL, kms_key text DEFAULT NULL, OUT rows_uploaded bigint, OUT files_uploaded bigint, OUT bytes_uploaded bigint)
  LANGUAGE sql AS $$ SELECT NULL::bigint, NULL::bigint, NULL::bigint $$;

With the baseline in place, the same migrate diff compiles the schema and writes a clean migration:

atlas migrate diff initial --env local
migrations/20260709065321_initial.sql
-- Create "events" table
CREATE TABLE "public"."events" (
  "id" bigint NOT NULL GENERATED ALWAYS AS IDENTITY,
  "payload" jsonb NOT NULL,
  "created_at" timestamptz NOT NULL DEFAULT now(),
  PRIMARY KEY ("id")
);
-- Create "on_event_insert" function
CREATE FUNCTION "public"."on_event_insert" () RETURNS trigger LANGUAGE plpgsql AS $$
BEGIN
    PERFORM aws_lambda.invoke(
        aws_commons.create_lambda_function_arn('process-event', 'us-east-1'),
        to_jsonb(NEW)
    );
    RETURN NEW;
END;
$$;
-- Create trigger "trg_on_event_insert"
CREATE TRIGGER "trg_on_event_insert" AFTER INSERT ON "public"."events" FOR EACH ROW EXECUTE FUNCTION "public"."on_event_insert"();
-- Create "import_events" function
CREATE FUNCTION "public"."import_events" ("bucket" text, "path" text) RETURNS text LANGUAGE sql AS $$
SELECT aws_s3.table_import_from_s3(
        'public.events', '', '(format csv)',
        aws_commons.create_s3_uri("bucket", "path", 'us-east-1')
    );
$$;

Notice what isn't there: none of the stubbed schemas. Because they live in the dev database's baseline (its starting state), they sit on both sides of every diff and cancel out. Atlas never creates or drops them, and a second diff confirms it:

atlas migrate diff followup --env local
The migration directory is synced with the desired state, no changes to be made

That cancellation is why the stubs belong in baseline, not atop schema.sql. Paste them there and Atlas treats them as yours to manage, re-creating them on every plan.

Use a PostGIS image for binary extensions​

Not every extension is missing a binary. PostGIS and pgvector are ordinary open-source extensions with public images, so the binaries exist and you shouldn't stub them. Do the opposite: give the dev database an image that has PostGIS, and layer the AWS stubs on top with the same baseline.

Given a new schema that needs both a PostGIS geometry column and an aws_lambda trigger:

schema.sql
-- PostGIS is a real binary extension: declaring it enables the geometry type in the
-- build-block image (a harmless no-op in the pre-built image, which already has it).
-- The AWS functions stay stubbed.
CREATE EXTENSION IF NOT EXISTS "postgis";

CREATE TABLE "public"."locations" (
    "id"   bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
    "name" text NOT NULL,
    "geom" geometry(Point, 4326) NOT NULL
);

CREATE FUNCTION "public"."on_location_insert"() RETURNS trigger
    LANGUAGE plpgsql AS $$
BEGIN
    PERFORM aws_lambda.invoke(
        aws_commons.create_lambda_function_arn('geocode', 'us-east-1'),
        to_jsonb(NEW)
    );
    RETURN NEW;
END;
$$;

CREATE TRIGGER "trg_on_location_insert"
    AFTER INSERT ON "public"."locations"
    FOR EACH ROW EXECUTE FUNCTION "public"."on_location_insert"();

Reuse the same stubs/aws.sql and point the docker block at a PostGIS image. Only the docker block changes below; the env "local" block stays exactly as before. There are two ways to get PostGIS in, depending on how tightly you need to pin the version:

The postgis/postgis images ship PostGIS pre-installed, and the tag pins the version. Find the versions to match on your RDS instance:

SELECT current_setting('server_version_num');        -- e.g. 180003 -> PostgreSQL 18
SELECT default_version FROM pg_available_extensions   -- e.g. 3.4.x
  WHERE name = 'postgis';

Then use the matching tag (16-3.4 is just an example):

atlas.hcl
docker "postgres" "dev" {
  // Use the tag matching your RDS PostgreSQL/PostGIS versions.
  image = "postgis/postgis:16-3.4"
  baseline = file("stubs/aws.sql")
}

Either way the schema compiles: geometry(Point, 4326) resolves against the real PostGIS, and aws_lambda.invoke(...) against the stubs. The one visible difference is whether PostGIS appears in your migration, which depends on how it reached the dev database.

With the pre-built image, PostGIS is already installed. It behaves like the AWS stubs: present in the baseline and declared in your schema, so it cancels out and stays out of the plan. The migration contains only your application objects:

migrations/20260709071612_initial.sql
-- Create "locations" table
CREATE TABLE "public"."locations" (
  "id" bigint NOT NULL GENERATED ALWAYS AS IDENTITY,
  "name" text NOT NULL,
  "geom" public.geometry(Point,4326) NOT NULL,
  PRIMARY KEY ("id")
);
-- Create "on_location_insert" function
CREATE FUNCTION "public"."on_location_insert" () RETURNS trigger LANGUAGE plpgsql AS $$
BEGIN
    PERFORM aws_lambda.invoke(
        aws_commons.create_lambda_function_arn('geocode', 'us-east-1'),
        to_jsonb(NEW)
    );
    RETURN NEW;
END;
$$;
-- Create trigger "trg_on_location_insert"
CREATE TRIGGER "trg_on_location_insert" AFTER INSERT ON "public"."locations" FOR EACH ROW EXECUTE FUNCTION "public"."on_location_insert"();

One catch: since this migration carries no CREATE EXTENSION postgis, PostGIS must already be enabled on your RDS target. It's available there but not installed by default, so a role in rds_superuser runs CREATE EXTENSION postgis once; Atlas then manages only the geometry columns, never the extension itself.

The build block changes the dev database, and through it the migration. It installs the binary but doesn't pre-create the extension, so your schema's CREATE EXTENSION IF NOT EXISTS "postgis" runs on the dev database, Atlas starts managing it, and the migration now leads with it (recording whatever version the dev package shipped):

migrations/20260709071612_initial.sql
-- Create extension "postgis"
CREATE EXTENSION "postgis" WITH SCHEMA "public" VERSION "3.6.4";
-- Create "locations" table
-- ... (same table, function, and trigger as above)

But moving CREATE EXTENSION postgis into the migration changes where it runs, not who is allowed to run it. The build block reaches RDS only through this generated file, so two things must hold on the target, or migrate apply fails:

  • Privilege. CREATE EXTENSION postgis on RDS requires a role in rds_superuser (PostGIS is not a trusted extension). A least-privilege deploy role like the atlas_migrator used above (a member of rds_iam, not a superuser, without CREATE on the database) cannot create it. The build block does not remove the need for a privileged role; it shifts that need into migrate apply.
  • Version. The migration pins the dev image's PostGIS version (here 3.6.4), so RDS must offer that exact version. If RDS ships only, say, 3.6.1, then CREATE EXTENSION postgis VERSION "3.6.4" fails with the same has no installation script nor update path for version error the caution above flags, only now on the target.

So reach for the build block when a privileged role applies the migration and the versions line up; otherwise prefer the pre-built image and have a superuser enable PostGIS on RDS once.

Managing the extension object​

Everything so far assumes your schema only calls the extensions' functions. Managing one is a different case: if your schema also declares CREATE EXTENSION aws_lambda (or extension "aws_lambda" {}), Atlas tries to create the extension on the dev database, and for a cloud-only extension there's nothing to create:

Error: schema.sql:1: pq: extension "aws_lambda" is not available (0A000)

You might try keeping the declaration and adding exclude = ["*[type=extension]"]. That doesn't work: the CREATE EXTENSION runs while Atlas loads your schema onto the dev database, ahead of any exclude filtering, so the load fails first.

The fix is to stop managing it. RDS owns these extensions, and your stubs already provide the functions your schema calls, so drop the CREATE EXTENSION line and just call them, the way the earlier setup does. That plans cleanly.

That leaves the mirror image on the target: once the extension is enabled on RDS, Atlas sees it during inspection, and since your schema no longer declares it, Atlas would plan to drop it. Tell Atlas to leave target extensions alone:

atlas.hcl
env "rds" {
  url     = "postgres://..."          // your RDS instance
  dev     = docker.postgres.dev.url   // dev database with the stubs
  // Do not manage extension objects; RDS owns them.
  exclude = ["*[type=extension]"]
}

Narrow it to one extension with exclude = ["aws_lambda[type=extension]"]. The same lever handles PostGIS version drift: use version = "ANY" on the extension block, or exclude = ["*[type=extension].version"]. The extension management and version mismatch FAQs go deeper.

Configure the RDS environment​

Point one env "rds" at the real database for both commands, with data "aws_rds_token" supplying a short-lived IAM token:

atlas.hcl
variable "endpoint" {
  // host:port, e.g. mydb.abc123.us-east-1.rds.amazonaws.com:5432
  type = string
}

variable "region" {
  type    = string
  default = "us-east-1"
}

variable "database" {
  type = string
}

variable "username" {
  type    = string
  default = "atlas_migrator"
}

docker "postgres" "dev" {
  image    = "postgres:18"
  baseline = file("stubs/aws.sql")
}

data "aws_rds_token" "migrator" {
  region   = var.region
  endpoint = var.endpoint
  username = var.username
}

env "rds" {
  src = "file://schema.sql"
  url = "postgres://${var.username}:${urlescape(data.aws_rds_token.migrator)}@${var.endpoint}/${var.database}?sslmode=require"
  dev = docker.postgres.dev.url
  migration {
    dir = "file://migrations"
  }
  // RDS owns the cloud-only extensions; do not try to manage them.
  exclude = ["*[type=extension]"]
}

endpoint and database have no defaults; pass them each run:

atlas migrate diff initial --env rds \
  --var endpoint=mydb.abc123.us-east-1.rds.amazonaws.com:5432 \
  --var database=appdb

atlas migrate apply --env rds \
  --var endpoint=mydb.abc123.us-east-1.rds.amazonaws.com:5432 \
  --var database=appdb

aws_rds_token signs the token from your ambient AWS credentials, so IAM auth must be configured for the database user, see the RDS IAM migration role guide.

Apply this to other extensions​

The AWS extensions are just the recurring example; the technique is general. Whenever an extension has no public binary but your schema calls its functions:

First, find what you use. Grep for the extension's qualified calls (aws_lambda., aws_s3.) and stub only those. Then read the signatures from the source of truth rather than the docs: a database that has the extension installed. The catalog renders them exactly as CREATE FUNCTION needs, with OUT parameters, defaults, and all:

SELECT format('%I.%I', n.nspname, p.proname) AS func,
       pg_get_function_arguments(p.oid)      AS args,
       pg_get_function_result(p.oid)         AS returns
FROM pg_proc p
JOIN pg_namespace n ON n.oid = p.pronamespace
WHERE n.nspname IN ('aws_commons', 'aws_lambda', 'aws_s3')
ORDER BY 1, 2;

And the composite types the same way:

SELECT format('%I.%I', n.nspname, t.typname) AS type,
       string_agg(a.attname || ' ' || format_type(a.atttypid, a.atttypmod), ', '
                  ORDER BY a.attnum)         AS fields
FROM pg_type t
JOIN pg_namespace n ON n.oid = t.typnamespace
JOIN pg_class c     ON c.oid = t.typrelid
JOIN pg_attribute a ON a.attrelid = c.oid AND a.attnum > 0 AND NOT a.attisdropped
WHERE n.nspname = 'aws_commons' AND t.typtype = 'c'
GROUP BY 1
ORDER BY 1;

Run these against RDS (swap in your schema names) for the exact shape to mirror. For the AWS extensions on RDS PostgreSQL 18 they return the eleven functions and three composite types in stubs/aws.sql (ignore any extra internal functions). The types, for example:

                type                |                        fields
------------------------------------+------------------------------------------------------
 aws_commons._aws_credentials_1     | access_key text, secret_key text, session_token text
 aws_commons._lambda_function_arn_1 | function_name text, region text
 aws_commons._s3_uri_1              | bucket text, file_path text, region text

From there it's mechanical: recreate each object with its real signature and a no-op body (SELECT NULL::<return type>, OUT parameters intact), wire the file into baseline, and, if you were managing the extension object, stop and exclude extensions on the RDS env. Functions and composite types cover most schemas; operators or casts stub the same way.

The stubs only exist at dev time. Atlas compiles and diffs against them, while the real functions keep running on RDS, untouched during planning. An empty body works because nothing at plan time ever calls it for real.

References​