ClickHouse® Cluster Fault Tolerance

ClickHouse® cluster fault tolerance is affected by:

• availability of data replication;
• node allocation type.

Data replication

Replication is supported only for replicated tables. Read more about replicated tables in the ClickHouse® documentation guide Replicated* table engines.

To ensure data replication and enable the creation of replicated tables, the cluster must consist of:

• at least one shard with two or more nodes;
• and node groups ClickHouse® Keeper.

You can specify the number of nodes in a shard when creating a cluster. You can also add or delete nodes in a shard of an existing cluster. The maximum number of nodes in a shard is ten.

If you specify more than one node in a shard when creating a cluster, ClickHouse® Keeper will be added automatically.

To add nodes to an existing cluster consisting of one shard with one node, you must first add ClickHouse® Keeper. When adding ClickHouse® Keeper, existing MergeTree tables are automatically converted to replicated tables (for example, to ReplicatedMergeTree). When adding new nodes to such a cluster, they are automatically synchronized with the existing replicas in the shard.

Node allocation type

The node allocation type in a cluster depends on:

• the number of nodes in shards;
• the type of pool where the cluster is located;
• the number of segments in the pool;
• configuration line.

You can view the list of pools and segments in the Servercore Infrastructure table of the Locations: countries, regions, availability zones, pools, and data center addresses instruction.

Four node allocation types are available:

• Single-Segment;
• Multi-Segment;
• Single-AZ;
• Multi-AZ.

Single-Segment

nodes are placed in the same segment of a single-AZ pool. Applicable to:

• clusters consisting of one shard with one node;
• shards with multiple nodes, if the cluster is located in a pool that consists of a single segment;
• shards with multiple nodes in HighFreq configuration lines.

note

For example, you have created a cluster in the ru-2 pool with ClickHouse® Keeper and a shard with two nodes. For both node groups, you used configurations from the HighFreq line. All nodes will be placed in the same pool segment, for example, ru-2c.

Multi-Segment

nodes are placed in different segments of a single-AZ pool. Applicable for clusters that consist of shards with multiple nodes if the cluster is located in a multi-segment pool.

Segments are located in different racks of the same data center. Nodes of each shard are distributed across segments sequentially.

note

For example, you have created a cluster in the ru-1 pool with one shard, in which there are four nodes. The first three nodes will be placed sequentially in the pool segments ru-1a, ru-1b, and ru-1c. The fourth node will be placed in the ru-1a segment. If you add a fifth node, it will be placed in the ru-1b pool segment.

Single-AZ

nodes are placed in the same segment of a multi-AZ pool. Applicable for clusters that consist of one shard with one node if the cluster is located in the ru-6 pool.

If you increase the number of nodes in a shard, the node allocation type will change to Multi-AZ.

note

For example, you have created a cluster in the ru-6 pool with one shard, in which there is one node. The node will be placed in one pool segment, for example, ru-6a.

Multi-AZ

nodes are placed in different segments of a multi-AZ pool. Applicable for clusters that consist of shards with multiple nodes if the cluster is located in the ru-6 pool.

Each pool segment is located in a separate data center. Nodes of each shard are distributed across segments sequentially.

note

For example, you have created a cluster in the ru-6 pool with one shard, in which there are four nodes. The first three nodes will be placed sequentially in the pool segments ru-6a, ru-6b, and ru-6c. The fourth node will be placed in the ru-6a segment. If you add a fifth node, it will be placed in the ru-6b pool segment.

Add a node

There can be no more than ten nodes in a shard. During the node addition process, other shard nodes will continue to operate for read and write operations.

1. If the cluster consists of only one shard with one node, add ClickHouse® Keeper.
2. In the Dashboard, on the top menu, click Products and select Managed Databases.
3. Open the Active tab.
4. Open the cluster page → Settings tab.
5. In the shard menu, select Change node count.
6. Open the Add tab.
7. Specify the new number of nodes in the shard. If there are no free IP addresses in the subnet to which the cluster is connected, you cannot add the node. Each new node occupies a new IP address in the subnet.
8. Click Save.

Delete a node

carefully

Deleting a node will remove all data located on that node.

Other nodes in the shard will continue to operate in read and write mode during deletion. The public IP address associated with the node will be deleted.

1. In the Dashboard, on the top menu, click Products and select Managed Databases.
2. Open the Active tab.
3. Open the cluster page → Settings tab.
4. In the shard menu, select Change node count.
5. Open the Delete tab.
6. Select the node you want to delete. There must be at least one node remaining in the shard. You cannot delete all nodes from a shard.
7. Click Save.

ClickHouse® is a registered trademark of ClickHouse, Inc. https://clickhouse.com.