A non-interactive diff now fails on a ClickHouse table that must be dropped and recreated with its data, instead of leaving the change out of the plan. Set allow_recreate in the modify_table block of the diff policy to plan it.
Atlas applies most ClickHouse table changes in place, and rebuilds a table into a copy when ClickHouse cannot alter it. Two cases cannot keep the rows during a rebuild, so Atlas drops the table and creates it again:
- A change to the engine, PARTITION BY, ORDER BY, or the primary key of a Replicated*MergeTree table whose engine sets a fixed ZooKeeper path. A copy of the table cannot register at the same path.
- Any change to a TimeSeries table. Its data lives in inner tables that are dropped with it.
Non-interactive runs fail instead of skipping
In a terminal, Atlas asks whether to abort or drop and recreate the table, as before.--auto-approve does not skip this question. When there is no terminal to ask in, as in CI or the Kubernetes Operator, Atlas used to leave the change out and report the schema as synced. It now stops and names the table:
# In CI, or wherever stdout is not a terminal:$ atlas schema apply --env prod --auto-approveError: Table "events" must be dropped and recreated for this change, and its data will be lost: its engine sets the ZooKeeper path '/clickhouse/tables/{shard}/default/events', which a copy of the table cannot reuse.Recreate the table manually, or set 'allow_recreate' in the 'modify_table' block of the diff policy to plan it
atlas schema diff fails on these tables in the same way, instead of printing a diff that leaves them out.
Allowing the rebuild
To plan the drop and recreate without a prompt, set allow_recreate in the modify_table block of the ClickHouse diff policy, for the whole project or for one environment:
env "prod" { diff "clickhouse" { modify_table { allow_recreate = true } }}
On the Kubernetes Operator, put the same env block in the project configuration of the resource.
-- Drop "events" tableDROP TABLE `events` SYNC;-- Create "events" tableCREATE TABLE `events` (`ts` DateTime, `id` UInt64) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/default/events', '{replica}') PRIMARY KEY tuple() ORDER BY (`id`, `ts`);
Replicated tables keyed by {uuid} keep their rows
A ZooKeeper path that contains the {uuid} macro resolves to a different path for each table, and so does the server default used by an engine with no path. Atlas now rebuilds these tables into a copy and keeps their rows, with no prompt and no policy needed:
CREATE TABLE `events_tmp` ON CLUSTER `{cluster}` (`ts` DateTime, `id` UInt64) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{cluster}/{shard}/{uuid}', '{replica}') PRIMARY KEY tuple() ORDER BY (`id`, `ts`) PARTITION BY (toYear(ts));INSERT INTO `events_tmp` (`ts`, `id`) SELECT `ts`, `id` FROM `events`;EXCHANGE TABLES `events` AND `events_tmp` ON CLUSTER `{cluster}`;DROP TABLE `events_tmp` ON CLUSTER `{cluster}` SYNC;
The same applies when a change moves a table to a new path, as when it becomes replicated, because no table holds that path yet. A path built from {database} and {table} is expanded when the table is created, so it still counts as fixed.
Getting Started
The ClickHouse driver is part of Atlas Pro:
$ atlas loginSee the ClickHouse cluster mode guide for replicated tables managed with ON CLUSTER.