Skip to main content

Managing YugabyteDB Schemas with the Atlas Kubernetes Operator

· 16 min read
Noa Rogoszinski
DevRel Engineer
Luan Tran Minh
Software Engineer

Teams don't adopt YugabyteDB by accident. They choose it when a single relational database primary can no longer keep up with their writes from payments, order processing, inventory, etc.

A distributed database carries more than just columns and types: every table and index records how data is split across the cluster, and those choices determine how the database performs at scale. Many teams run YugabyteDB next to their services on Kubernetes, where every other part of the system is versioned in a repository, reviewed in a pull request, and deployed by changing a manifest. A change to how a table is split deserves the same review as a change to its columns, and the same rollout to every cluster.

On YugabyteDB, Atlas models colocation, tablet pre-splitting, and index sharding as part of the schema, so the distribution layout is versioned, diffed, and deployed with the rest of it, instead of living in handwritten DDL that drifts between clusters. The Atlas Kubernetes Operator completes the picture: it applies each version of the migration directory to the database from a Kubernetes resource, the same way the cluster deploys a new image.

This post walks through how an application team can use Atlas and the Operator to declare, review, and evolve their distributed YugabyteDB schema.

Distributed PostgreSQL on Kubernetes​

YugabyteDB is a distributed SQL database built on the PostgreSQL query layer, allowing teams to continue using the SQL, drivers, tools, and ecosystem they already know. The underlying storage layer is distributed: data is sharded across nodes and replicated via Raft consensus, so the database scales horizontally and remains available through node, zone, or region failures. On Kubernetes, its nodes run as pods, spread across zones like the services around them.

A traditional single-primary database routes every write through a single node, so the only way to add write capacity is to use a larger server. YugabyteDB spreads writes across the cluster, so you can add capacity by adding nodes, without sacrificing ACID transactions or strong consistency.

Take online retail orders, for example. During an event or flash sale, order volume can increase dramatically within minutes, and Kubernetes scales out the checkout services to meet it. Each checkout writes to orders, line items, payments, and inventory reservations, and all of those writes must commit together. YugabyteDB spreads them across the cluster, distributed transactions prevent stock from being oversold, and replication keeps a failed node or zone from stopping checkouts. Data can be placed in regions close to customers to reduce latency or meet data residency requirements, while the application still sees a single PostgreSQL-compatible database.

The only catch is that this performance depends on schema decisions a single-node database never asks you to make, and those decisions have to ship to every cluster as reliably as the services that depend on them.

Distribution Is Part of the Schema​

In YugabyteDB, both tables and indexes are split into tablets. How each object is split across the cluster is a schema design decision, on the same level as choosing columns and types. YugabyteDB exposes three of these decisions in DDL, and Atlas, through its ysql:// driver, models each one as a schema attribute:

DecisionWhat it controlsDDLAtlas attribute
Hash or range shardingWhether rows of a table, or entries of an index, are hash-distributed or kept in sorted order by keyPRIMARY KEY (id HASH), ON orders (month_created ASC)sharding = HASH or sharding = RANGE on a primary key or index column
Pre-splittingHow many tablets a table starts with, or where their boundaries areSPLIT INTO 100 TABLETS, SPLIT AT VALUES (...)split_into for hash-sharded tables, split_at_values for range-sharded tables
ColocationWhether a small table shares one tablet with its neighbors or gets its ownWITH (colocation = false)colocation = false to opt a table out of a colocated database

Most of these are fixed when the object is created. A hash-sharded index can't become range-sharded, and a colocated table can't move to its own tablets, without dropping and recreating it.

A schema tool that models only tables, columns, and types either ignores these attributes, so a change to them never shows up in a diff, or plans a migration that drops them. Atlas compares them when it diffs two schemas, and writes them into the DDL of each migration file it generates.

Atlas and the Kubernetes Operator​

This post uses the versioned workflow, where each change is a SQL migration file committed to the repository. Some distribution settings only exist at creation time. split_into sets the number of tablets a table starts with, and YugabyteDB splits tablets automatically as the table grows, so Atlas doesn't read the count back from a live database, and automatic splitting never turns into drift or into a change in the next migration. A migration file keeps the SPLIT INTO clause exactly as Atlas generated it, and every cluster runs the same file, so every cluster creates the table with the same tablets. The schema inspection section of the YugabyteDB guide lists which distribution attributes Atlas reads back from a live database.

The Atlas Kubernetes Operator brings this to the cluster. An AtlasMigration resource names a database and a migration directory in the Atlas Registry. The Operator applies the migrations the database hasn't run yet, and records each one in the database's revision history.

The flow for every change:

  1. An engineer edits the schema in a feature branch and runs atlas migrate diff to generate a migration file.
  2. CI lints the new migration file and posts the results to the pull request.
  3. Reviewers approve the migration and the code change together. On merge, CI pushes the migration directory to the Atlas Registry, tagged with the commit.
  4. The AtlasMigration manifest is updated to the new tag, and the Operator applies the pending migrations to the database.

The rest of this post follows a retail team through this flow.

Setting Up​

This post picks up where Managing Distributed Schemas with Confidence: Atlas Now Supports YugabyteDB, on the YugabyteDB blog, leaves off. That post defines the team's schema and tries it on a local cluster. This one uses the same project:

  • shop is a regular database. It holds orders, defined in shop.hcl, with a hash-sharded primary key.
  • lookup is colocated. It holds the small currencies and promotions tables, defined in lookup.hcl, which share one tablet.
  • atlas.hcl defines the shop_local and lookup_local environments, each with a dev database that matches its target's colocation.

The cluster runs in the yugabyte namespace behind the yb-tservers service. Both databases are owned by shop_admin, and lookup is created WITH COLOCATION = true. Colocation is set when a database is created, and Atlas works inside existing databases, so creating them is a one-time setup step outside the migration workflow.

Installing the Operator​

Install the Operator with Helm:

helm install atlas-operator oci://ghcr.io/ariga/charts/atlas-operator

In production, enable the chart's persistence so the Operator keeps its cached Atlas Cloud grant across pod restarts.

The Operator authenticates with a bot token from your Atlas Cloud organization rather than your local session. Store the token and the database URLs in Secrets, in the namespace the AtlasMigration resources run in:

kubectl create secret generic atlas-token --from-literal=token=<your bot token>
kubectl create secret generic shop-db \
--from-literal=url='ysql://shop_admin:<password>@yb-tservers.yugabyte:5433/shop'
kubectl create secret generic lookup-db \
--from-literal=url='ysql://shop_admin:<password>@yb-tservers.yugabyte:5433/lookup'

Each URL names a database and no schema, so Atlas manages every schema in it. The host is the namespace-qualified service name, since the Operator connects from its own namespace. Atlas connects over TLS by default, so the URL assumes the cluster has client-to-server encryption enabled.

Generating the Migrations​

Each database gets its own migration directory. Add a migration block to each local environment in atlas.hcl. The large, write-heavy tables are held in shop, so its environment also gets a diff policy that makes Atlas build new indexes online, without blocking writes to a table that already holds data:

atlas.hcl
env "shop_local" {
url = "ysql://yugabyte@localhost:5433/shop?sslmode=disable"
dev = "docker://ysql/latest/dev"
schema {
src = "file://shop.hcl"
}
migration {
dir = "file://migrations/shop"
}
diff {
concurrent_index {
create = true
}
}
}

lookup_local gets the same migration block, with file://migrations/lookup as its directory. Then generate the first migration for each database:

atlas migrate diff init --env shop_local
atlas migrate diff init --env lookup_local

Atlas plans each schema on the environment's dev database and writes the result to the directory, with an atlas.sum checksum file that protects it from accidental edits:

migrations/
├── lookup/
│ ├── 20260415093012_init.sql
│ └── atlas.sum
└── shop/
├── 20260415093004_init.sql
└── atlas.sum

The distribution settings are in the DDL: orders gets a hash-sharded primary key, and the colocated tables get range-sharded primary keys (ASC), since a single shared tablet has nothing to hash across.

Publishing the Migrations​

Push both directories to the Atlas Registry. The first push updates the directory's latest version, which the Operator reads when it migrates a database down, and the second tags this exact state:

atlas migrate push shop --env shop_local
atlas migrate push shop:v1 --env shop_local
atlas migrate push lookup --env lookup_local
atlas migrate push lookup:v1 --env lookup_local

From here on, CI lints each new migration in the pull request and, on merge, pushes both directories with the commit SHA as their tag, as the GitHub Actions guide for versioned migrations sets up.

A Colocated Dev Database in the Cluster​

The Operator uses a dev database, too. It applies migration files as written, so the dev database doesn't change what a deployment runs. Migrating down is different: when a resource allows it with protectedFlows.migrateDown, the Operator itself plans the SQL that reverts each version, on the dev database. Like the local ones, that database must match its target's colocation. On a dev database that isn't colocated, the plan that reverts the drop of a colocation = false table recreates it as a colocated table.

For ysql:// targets, the Operator starts YugabyteDB in a pod and uses its default yugabyte database, which is not colocated. That matches shop, so its resource needs no dev database settings. lookup is colocated and needs a dev database that is, so the team gives its resource its own dev pod with devDB.spec.

The pod runs yugabyted, and its startup probe creates a colocated dev database once the server is up:

devURL: ysql://yugabyte@localhost:5433/dev?sslmode=disable
devDB:
spec:
containers:
- name: yugabyte-dev
image: yugabytedb/yugabyte:latest
command: ["bin/yugabyted", "start", "--background=false"]
env:
- name: HOME
value: /tmp
securityContext:
runAsUser: 0
ports:
- containerPort: 5433
name: ysql
startupProbe:
exec:
command:
- sh
- -ec
- |
bin/ysqlsh -h "$(hostname)" -U yugabyte -tAc "SELECT 1 FROM pg_database WHERE datname = 'dev'" | grep -q 1 ||
bin/ysqlsh -h "$(hostname)" -U yugabyte -c "CREATE DATABASE dev WITH COLOCATION = true"
failureThreshold: 30
periodSeconds: 10

The Operator waits for the probe to pass before it uses the pod, so Atlas never sees it before dev exists. The localhost host in devURL is a placeholder: the Operator replaces it with the pod's address. Like the target URL, devURL names a database and no schema, so the two have the same scope.

The yugabyted image runs as root, as in the dev pod the Operator creates for YugabyteDB on its own. A namespace that enforces the restricted Pod Security level rejects it, so run the AtlasMigration in a namespace that allows it. Pin image to the version your cluster runs, so the dev database behaves like the database the migrations are applied to.

By default, the Operator keeps each dev database running between deployments, so the next one starts faster. A yugabyted pod is a full database node, so a team with many resources may prefer prewarmDevDB: false on the resource or in the chart's values, which starts the dev database only when there are migrations to apply and stops it afterward.

Deploying the Migrations​

An AtlasMigration resource ties each database to its migration directory. The one for shop:

shop-migration.yaml
apiVersion: db.atlasgo.io/v1alpha1
kind: AtlasMigration
metadata:
name: shop
spec:
urlFrom:
secretKeyRef:
name: shop-db
key: url
cloud:
tokenFrom:
secretKeyRef:
name: atlas-token
key: token
dir:
remote:
name: shop
tag: v1

lookup-migration.yaml is the same, with the name lookup, the lookup-db Secret, the lookup directory, and the devURL and devDB from above.

Apply the manifests and wait for the resources to become ready:

kubectl apply -f shop-migration.yaml -f lookup-migration.yaml
kubectl wait --for=condition=Ready atlasmigration/shop atlasmigration/lookup --timeout=5m
kubectl get atlasmigration
NAME     READY   REASON
lookup True Applied
shop True Applied

The Operator ran each init migration as written, and created the revision history table, atlas_schema_revisions, in a schema of the same name. The resource reports the version the database is on:

kubectl get atlasmigration shop -o jsonpath='{.status.lastAppliedVersion}{"\n"}'
20260415093004

Adding a Range-Sharded Index​

Months later, orders holds millions of rows, and most reporting and support queries are bounded by time: orders from last month, or orders in a given quarter. month_created holds the first day of the month an order was placed in, so the team needs an index on it.

How that index is sharded decides how those queries perform. A hash-sharded index scatters adjacent values across all of its tablets, so a WHERE month_created BETWEEN ... query reads every one of them. A range-sharded index keeps values in sorted order, so the same query reads only the tablets that hold those months.

An engineer adds the index to shop.hcl in a feature branch, as in the YugabyteDB blog post, and sets sharding = RANGE on its column:

shop.hcl
table "orders" {
// ...
index "orders_month_created_idx" {
on {
column = column.month_created
sharding = RANGE
}
}
}

Then they generate the migration:

atlas migrate diff add_orders_month_created_idx --env shop_local

Atlas replays the migration directory on the dev database, compares the result with shop.hcl, and writes only the difference:

-- atlas:txmode none

-- Create index "orders_month_created_idx" to table: "orders"
CREATE INDEX CONCURRENTLY "orders_month_created_idx" ON "public"."orders" ("month_created" ASC);

The concurrent_index policy in shop_local makes Atlas build the index online, and the atlas:txmode none header runs the file outside a transaction, which YugabyteDB requires for an online build.

From this file alone, a reviewer can confirm the index is range-sharded: ASC on the leading column, where a hash-sharded index would have none. What the file leaves out matters as much: the distribution settings already in the directory match shop.hcl, so Atlas doesn't strip or rewrite them.

The engineer commits the file with shop.hcl, and CI lints it and posts the results to the pull request. Without the policy, Atlas would generate a plain CREATE INDEX that runs inside a transaction, and the lint would flag it, since YugabyteDB builds an index that way without online backfill. A wrong sharding choice caught in the pull request costs a comment. Caught in production, it means dropping and rebuilding an index over millions of rows.

After the merge, CI pushes the directory tagged with the merge commit's SHA. A commit can't name its own SHA, so moving the manifest to the new tag is a follow-up change, typically one CI commits to the repository the cluster deploys from:

shop-migration.yaml
  dir:
remote:
name: shop
tag: 3f9c2b1e7d4a6058c1e2f3a4b5c6d7e8f9a0b1c2

The Operator acts on changes to the resource, not on pushes to the registry, so updating the tag is the deployment step. The database is on init, so the Operator applies the one pending file.

Rolling Out to Every Cluster​

Each production cluster gets its own pair of AtlasMigration resources, with their own URL Secrets, pointing at the same tags. A cluster that is behind gets only the files it is missing, and a cluster that is already up to date has nothing to apply. A team with a staging cluster moves its manifests to a new tag first, and production follows once staging reports Ready=True.

To stop a deployment to a cluster whose schema was changed outside the migration directory, enable the drift check on the resource:

spec:
policy:
drift: {}

Before it applies pending migrations, the Operator compares the database with the schema the Atlas Registry holds for the version the database is on. If they differ, it applies nothing, and the resource reports Ready=False with the reason DriftDetected. Once the database matches again, for example after the manual change is reverted, the next retry applies the pending migrations. To record drift without blocking the deployment, set onError: CONTINUE, and to ignore objects managed outside Atlas, list them under exclude.

Order the rollout so the schema lands before the code that needs it: the AtlasMigration first, then the application once the resource reports Ready=True. With Argo CD, put them in separate sync waves. This order is for additive changes, like a new index or column. A change that removes something goes the other way: first deploy the code that no longer uses the column, then the migration that drops it.

Wrapping Up​

On YugabyteDB, colocation, pre-splitting, and hash or range sharding decide how a write-intensive workload performs, and most of them can't be changed once a table holds data. With Atlas and its Kubernetes Operator:

  • The schema, including its distribution layout, is code in the application repository.
  • Every change to the layout is a migration file in the pull request, next to the code it belongs to.
  • The Operator deploys a migration directory the way Kubernetes deploys an image: by tag, from the manifest.
  • Every cluster runs the same files, so every cluster gets the same distribution layout.

Get Started​

The Atlas Operator quickstart guide for versioned workflows uses a local cluster and deploys a migration directory in a few minutes. The versioned YugabyteDB guide walks through colocation, pre-splitting, and index sharding, and automatic migrations for YugabyteDB shows how to bring an existing database under Atlas. To lint and push migrations from CI, follow the GitHub Actions guide for versioned migrations.

Running YugabyteDB on Kubernetes and want to see how Atlas fits your clusters? Schedule a demo with our team.