Atlas now manages ClickHouse workload scheduling: define resources and workloads in the schema, and Atlas inspects, diffs, and migrates them.
ClickHouse workload scheduling shares server capacity between groups of queries. A resource names the capacity to schedule, such as disk I/O or CPU threads, and a workload sets the share a group of queries gets. Atlas now inspects, diffs, and migrates both.
Define Resources
A resource block lists what the resource schedules. This one schedules reads and writes on the default disk, and another schedules CPU threads:
resource "io" { read_disks = ["default"] write_disks = ["default"]} resource "cpu" { master_thread = true worker_thread = true}
Each attribute maps to one item of CREATE RESOURCE:
| Attribute | Resource item |
|---|---|
| read_disks = ["<disk>"] | READ DISK <disk> |
| write_disks = ["<disk>"] | WRITE DISK <disk> |
| read_any_disk = true | READ ANY DISK |
| write_any_disk = true | WRITE ANY DISK |
| master_thread = true | MASTER THREAD |
| worker_thread = true | WORKER THREAD |
| query = true | QUERY |
| memory_reservation = true | MEMORY RESERVATION |
Define Workloads
Workloads form a tree. A workload block takes a parent and a setting block per scheduling setting, such as weight, priority, or a limit like max_io_requests. Set resource on a setting to apply it to one resource only:
workload "all" { setting "max_io_requests" { value = 100 resource = resource.io } setting "max_concurrent_threads" { value = 16 resource = resource.cpu }} workload "production" { parent = workload.all setting "weight" { value = 9 }} workload "development" { parent = workload.all setting "weight" { value = 1 }}
Atlas plans:
-- Create resource "cpu"CREATE RESOURCE `cpu` (MASTER THREAD, WORKER THREAD);-- Create resource "io"CREATE RESOURCE `io` (READ DISK `default`, WRITE DISK `default`);-- Create workload "all"CREATE WORKLOAD `all` SETTINGS `max_io_requests` = 100 FOR `io`, `max_concurrent_threads` = 16 FOR `cpu`;-- Create workload "development"CREATE WORKLOAD `development` IN `all` SETTINGS `weight` = 1;-- Create workload "production"CREATE WORKLOAD `production` IN `all` SETTINGS `weight` = 9;
Assign Queries to a Workload
A query runs in the workload named by its workload setting. To place the queries of a role in one workload, set it in a settings profile. Atlas manages the role and the profile in the same schema when roles and settings_profiles are also on in mode "clickhouse":
role "developer" {} settings_profile "developer_queries" { to = [role.developer] setting "workload" { value = "development" }}
Change a Workload
ClickHouse has no ALTER RESOURCE or ALTER WORKLOAD, so a change is planned as CREATE OR REPLACE. Adding a CPU limit to development:
workload "development" { parent = workload.all setting "weight" { value = 1 } setting "max_concurrent_threads" { value = 4 resource = resource.cpu }}
-- Modify workload "development"CREATE OR REPLACE WORKLOAD `development` IN `all` SETTINGS `weight` = 1, `max_concurrent_threads` = 4 FOR `cpu`;
Getting Started
Turn on resources and workloads in the mode "clickhouse" block. Resources and workloads belong to the server, not to a database, so use URLs that name no database for both the target and the dev database:
docker "clickhouse" "dev" { image = "clickhouse/clickhouse-server:25.8"} env "prod" { // No database in the URL: resources and workloads belong to the server. url = getenv("CLICKHOUSE_URL") dev = docker.clickhouse.dev.url schema { src = "file://schema.ch.hcl" mode "clickhouse" { resources = true workloads = true } }}
With no database in the URL, the schema describes the whole server: Atlas plans to drop every database and table it does not declare. Declare them in the same schema as your resources and workloads.
- Workload scheduling needs ClickHouse 24.11 or later. CPU, query, and memory resources need a later release, so run the dev database on the same version as the target.
- In cluster mode, if the servers set workload_zookeeper_path, also add has_workload_zookeeper_path=true to both URLs. Those servers replicate resources and workloads on their own and reject ON CLUSTER on them.
The ClickHouse driver is part of Atlas Pro:
$ atlas login