ClickHouse® Cluster Fault Tolerance

ClickHouse® cluster fault tolerance is affected by:

• availability of data replication;
• node allocation type.

Data replication

Replication is supported only for replicated tables. Learn more about replicated tables in the Replicated* table engines section of the ClickHouse® documentation.

To ensure data replication and enable the creation of replicated tables, the cluster must consist of:

• at least one shard with two or more nodes;
• and node groups ClickHouse® Keeper.

You can specify the number of nodes in a shard when creating a cluster. You can also add or remove nodes in a shard of an existing cluster. The maximum number of nodes in a shard is ten.

If you specify more than one node in a shard when creating a cluster, ClickHouse® Keeper will be added automatically.

To add nodes to an existing cluster consisting of a single shard with one node, you must first add ClickHouse® Keeper. When adding ClickHouse® Keeper, existing MergeTree tables are automatically converted to replicated ones (for example, to ReplicatedMergeTree). When adding new nodes to such a cluster, they are automatically synchronized with existing replicas in the shard.

Node allocation type

The node allocation type in a cluster depends on:

• the number of nodes in shards;
• the type of pool where the cluster is located;
• the number of segments in the pool;
• configuration line.

You can view the list of pools and segments in the Servercore Infrastructure table of the Locations: countries, regions, availability zones, pools, and data center addresses instruction.

Four node allocation types are available:

• Single-Segment;
• Multi-Segment;
• Single-AZ;
• Multi-AZ.

Single-Segment

Nodes are placed in the same segment of a single-zone pool. Applicable:

• clusters consisting of one shard with one node;
• shards with multiple nodes, if the cluster is located in a pool that consists of a single segment;
• shards with multiple nodes in HighFreq configuration lines.

note

For example, you have created a cluster in the ru-2 pool with ClickHouse® Keeper and a shard with two nodes. For both groups of nodes, you used configurations from the HighFreq line. All nodes will be placed in the same pool segment, for example, ru-2c.

Multi-Segment

Nodes are placed in different segments of a single-zone pool. Applicable to clusters that consist of shards with multiple nodes if the cluster is located in a pool with multiple segments.

Segments are located in different racks of the same data center. The nodes of each shard are distributed across segments sequentially.

note

For example, you have created a cluster in the ru-1 pool with one shard containing four nodes. The first three nodes will be placed sequentially in pool segments ru-1a, ru-1b, and ru-1c. The fourth node will be placed in segment ru-1a. If you add a fifth node, it will be placed in pool segment ru-1b.

Single-AZ

Nodes are placed in the same segment of a multi-zone pool. Applicable to clusters consisting of a single shard with one node if the cluster is located in the ru-6 pool.

If you increase the number of nodes in a shard, the node allocation type will change to Multi-AZ.

note

For example, you have created a cluster in the ru-6 pool with one shard containing one node. The node will be placed in the same pool segment, for example, ru-6a.

Multi-AZ

Nodes are placed in different segments of a multi-zone pool. Applicable to clusters that consist of shards with multiple nodes if the cluster is located in the ru-6 pool.

Each pool segment is located in a separate data center. The nodes of each shard are distributed across segments sequentially.

note

For example, you have created a cluster in the ru-6 pool with one shard containing four nodes. The first three nodes will be placed sequentially in pool segments ru-6a, ru-6b, and ru-6c. The fourth node will be placed in segment ru-6a. If you add a fifth node, it will be placed in pool segment ru-6b.

Add a node

A shard can contain no more than ten nodes. During the node addition process, other nodes in the shard will continue to operate for read and write operations.

1. If the cluster consists of only one shard with one node, add ClickHouse® Keeper.
2. In the Dashboard, on the top menu, click Products and select Managed Databases.
3. Open the Active tab.
4. Open the cluster page → Settings tab.
5. In the shard menu, select Change node count.
6. Open the Add tab.
7. Specify the new number of nodes in the shard. If there are no free IP addresses in the subnet to which the cluster is connected, you cannot add a node. Each new node occupies a new IP address in the subnet.
8. Click Save.

Delete a node

carefully

Deleting a node will remove all data located on that node.

Other nodes in the shard will continue to operate in read and write mode during deletion. The public IP address associated with the node will be deleted.

1. In the Dashboard, on the top menu, click Products and select Managed Databases.
2. Open the Active tab.
3. Open the cluster page → Settings tab.
4. In the shard menu, select Change node count.
5. Open the Delete tab.
6. Select the node you want to remove. At least one node must remain in the shard. You cannot remove all nodes from a shard.
7. Click Save.

ClickHouse® is a registered trademark of ClickHouse, Inc. https://clickhouse.com.