Sign Up
Docs
All Guides

Step-by-step Atlas walkthroughs

Databases

Database-specific Atlas guides

Setting up CI/CD

Wire Atlas into your pipeline

ORMs and Frameworks

Integrate Atlas with any ORM

Shift Left Database Security

Roles, permissions, and policies as code

Seed Data as Code

Lookup tables and reference data

Deploying on Kubernetes

Run via the Atlas Operator

Terraform Provider

Manage schemas via Terraform

Database Governance

End-to-end control and compliance

Schema Drift Detection

Block drift before and after deploys

AI-Safe Database Changes

Safe schema changes at machine speed

Modernize Schema Migrations

Apply modern CI/CD to schema changes

Standardize Schema Migrations

One migration tool to rule them all

Database per tenant

Manage thousands of databases as one

PRICING
Sign InSign Up
Back to changelog
New
•2 minute read

Redshift: External Schemas

Atlas now inspects, diffs, and migrates Redshift external schemas across all four source types - the AWS Glue Data Catalog (Redshift Spectrum), Kafka, Kinesis, and federated PostgreSQL - declared with an external block inside a schema in HCL.

Usage

Declare an external block inside a schema block to manage a Redshift external schema. For a Redshift Spectrum schema backed by the AWS Glue Data Catalog, use the data_catalog source:

schema.hcl
schema "es_catalog" {  external "data_catalog" {    database     = "spectrumdb"    iam_role     = "arn:aws:iam::123456789012:role/spectrum"    region       = "us-east-1"    catalog_role = "arn:aws:iam::123456789012:role/catalog"  }}

Atlas supports four source types. Each has its own connection and IAM options:

schema.hcl
schema "es_kafka" {  external "kafka" {    uri            = "b-1.kafka.example.internal:9092"    authentication = NONE  }}
schema "es_kinesis" {  external "kinesis" {    iam_role = "arn:aws:iam::123456789012:role/stream"    region   = "us-west-2"  }}
schema "es_postgres" {  external "postgres" {    database   = "pgdb"    schema     = "app"    uri        = "pg.example.internal"    port       = 5433    iam_role   = "arn:aws:iam::123456789012:role/federated"    secret_arn = "arn:aws:secretsmanager:us-east-1:123456789012:secret:pg"  }}
  • data_catalog - Redshift Spectrum over the AWS Glue Data Catalog. Takes database, iam_role, region, and an optional catalog_role.
  • kafka - Takes a uri and an authentication mode.
  • kinesis - Takes iam_role and region.
  • postgres - Federated queries against a PostgreSQL instance. Takes database, schema, uri, port, iam_role, and secret_arn.

Generated migrations

Adding an external schema emits a single CREATE EXTERNAL SCHEMA statement:

migration.sql
-- Add new external schema named "es_kafka"CREATE EXTERNAL SCHEMA "es_kafka" FROM KAFKA AUTHENTICATION NONE URI 'b-1.kafka.example.internal:9092';

Changing a source cannot be done in place, so Atlas recreates the schema by dropping and re-adding it:

migration.sql
-- Drop schema named "es_kafka"DROP SCHEMA "es_kafka" CASCADE;-- Add new external schema named "es_kafka"CREATE EXTERNAL SCHEMA "es_kafka" FROM KAFKA AUTHENTICATION NONE URI 'b-2.kafka.example.internal:9092';

Notes and limitations

  • Removing the external block converts the schema back into a regular, local Redshift schema.
  • Redshift manages an external schema's contents externally, so Atlas rejects attempts to define tables or other objects inside one.
  • Passwords and secrets are referenced by ARN (for example secret_arn) rather than stored in the schema definition.

Getting Started

Redshift support is part of Atlas Pro:

$ atlas login

Then point Atlas at your Redshift cluster and run atlas schema inspect or atlas migrate diff as usual. See the Redshift guide for more.

featureredshiftexternal-schemaspectrumddlinspectdiff
Back to changelog