Back to changelog
New
•3 minute read

ClickHouse: Workload Scheduling Resources and Workloads

Atlas now manages ClickHouse workload scheduling: define resources and workloads in the schema, and Atlas inspects, diffs, and migrates them.

ClickHouse workload scheduling shares server capacity between groups of queries. A resource names the capacity to schedule, such as disk I/O or CPU threads, and a workload sets the share a group of queries gets. Atlas now inspects, diffs, and migrates both.

Define Resources

A resource block lists what the resource schedules. This one schedules reads and writes on the default disk, and another schedules CPU threads:

schema.ch.hcl
resource "io" {  read_disks  = ["default"]  write_disks = ["default"]}
resource "cpu" {  master_thread = true  worker_thread = true}

Each attribute maps to one item of CREATE RESOURCE:

AttributeResource item
read_disks = ["<disk>"]READ DISK <disk>
write_disks = ["<disk>"]WRITE DISK <disk>
read_any_disk = trueREAD ANY DISK
write_any_disk = trueWRITE ANY DISK
master_thread = trueMASTER THREAD
worker_thread = trueWORKER THREAD
query = trueQUERY
memory_reservation = trueMEMORY RESERVATION

Define Workloads

Workloads form a tree. A workload block takes a parent and a setting block per scheduling setting, such as weight, priority, or a limit like max_io_requests. Set resource on a setting to apply it to one resource only:

schema.ch.hcl
workload "all" {  setting "max_io_requests" {    value    = 100    resource = resource.io  }  setting "max_concurrent_threads" {    value    = 16    resource = resource.cpu  }}
workload "production" {  parent = workload.all  setting "weight" {    value = 9  }}
workload "development" {  parent = workload.all  setting "weight" {    value = 1  }}

Atlas plans:

atlas migrate diff
-- Create resource "cpu"CREATE RESOURCE `cpu` (MASTER THREAD, WORKER THREAD);-- Create resource "io"CREATE RESOURCE `io` (READ DISK `default`, WRITE DISK `default`);-- Create workload "all"CREATE WORKLOAD `all` SETTINGS `max_io_requests` = 100 FOR `io`, `max_concurrent_threads` = 16 FOR `cpu`;-- Create workload "development"CREATE WORKLOAD `development` IN `all` SETTINGS `weight` = 1;-- Create workload "production"CREATE WORKLOAD `production` IN `all` SETTINGS `weight` = 9;

Assign Queries to a Workload

A query runs in the workload named by its workload setting. To place the queries of a role in one workload, set it in a settings profile. Atlas manages the role and the profile in the same schema when roles and settings_profiles are also on in mode "clickhouse":

schema.ch.hcl
role "developer" {}
settings_profile "developer_queries" {  to = [role.developer]  setting "workload" {    value = "development"  }}

Change a Workload

ClickHouse has no ALTER RESOURCE or ALTER WORKLOAD, so a change is planned as CREATE OR REPLACE. Adding a CPU limit to development:

schema.ch.hcl
workload "development" {  parent = workload.all  setting "weight" {    value = 1  }  setting "max_concurrent_threads" {    value    = 4    resource = resource.cpu  }}
atlas migrate diff
-- Modify workload "development"CREATE OR REPLACE WORKLOAD `development` IN `all` SETTINGS `weight` = 1, `max_concurrent_threads` = 4 FOR `cpu`;

Getting Started

Turn on resources and workloads in the mode "clickhouse" block. Resources and workloads belong to the server, not to a database, so use URLs that name no database for both the target and the dev database:

atlas.hcl
docker "clickhouse" "dev" {  image = "clickhouse/clickhouse-server:25.8"}
env "prod" {  // No database in the URL: resources and workloads belong to the server.  url = getenv("CLICKHOUSE_URL")  dev = docker.clickhouse.dev.url  schema {    src = "file://schema.ch.hcl"    mode "clickhouse" {      resources = true      workloads = true    }  }}

With no database in the URL, the schema describes the whole server: Atlas plans to drop every database and table it does not declare. Declare them in the same schema as your resources and workloads.

  • Workload scheduling needs ClickHouse 24.11 or later. CPU, query, and memory resources need a later release, so run the dev database on the same version as the target.
  • In cluster mode, if the servers set workload_zookeeper_path, also add has_workload_zookeeper_path=true to both URLs. Those servers replicate resources and workloads on their own and reject ON CLUSTER on them.

The ClickHouse driver is part of Atlas Pro:

$ atlas login
featureclickhouseworkload schedulingresourceworkloadatlas pro