Monitoring clusters, node groups, and ClickHouse nodes

In Managed Databases for ClickHouse, you can monitor the status of a cluster, node groups, and nodes in the dashboard.

You can:

• view cluster status;
• view cluster and node group statuses;
• view node statuses in node groups.

The cluster status indicates the health of its nodes. Statuses display operations involving the cluster, node groups, and nodes. You can view the operation history.

For a more detailed analysis, you can view cluster node metrics as charts in the dashboard.

When analyzing charts, keep in mind that the time in the dashboard corresponds to the time on your device and does not depend on the region where the cluster is hosted.

note

For example, you have created a cluster in Tashkent, in the uz-1 pool. Tashkent is in the UTC+5 time zone. On the device from which you logged into the control panel, the UTC+3 time zone is set. The time on the metrics charts will be displayed in UTC+3.

View cluster status

1. In the dashboard, click Products in the top menu and select Managed Databases.

2. Open the Active tab.

3. View the status in the cluster card.

   Running	The cluster is working properly
   Degraded	One or more nodes are unavailable or operating with limitations. Check the node statuses. If you cannot determine the cause yourself, create a ticket
   Down	All cluster nodes are unavailable, create a ticket
   Unknown	The cluster is being created, or its status is unknown. If this status applies to a cluster with a status other than CREATING, create a ticket

View cluster and node group statuses

1. In the dashboard, click Products in the top menu and select Managed Databases.

2. Open the Active tab.

3. View the cluster statuses and node group statuses in the cluster card.

Cluster statuses

CREATING	The cluster is being created
ACTIVE	The cluster is active; no operations are being performed on it
UPDATING	The cluster is being updated
RESIZING	The cluster is being scaled
REBUILDING	Cluster nodes are being recreated
LOCKING	The cluster is being locked
LOCKED	The cluster is locked. This status may appear due to a resource lock if there are insufficient funds on your balance. To unlock the cluster, top up your balance. If you cannot determine the cause of the lock yourself, create a ticket
UNLOCKING	The cluster is being unlocked
DELETING	The cluster is being deleted
DELETED	The cluster has been deleted
MAINTENANCE	The cluster is undergoing maintenance
ERROR	An error has occurred, create a ticket

Node group statuses

CREATING	The node group is being created
ACTIVE	The node group is active; no operations are being performed on it
UPDATING	The node group is being updated
RESIZING	The node group is scaling. While this operation is in progress, the same status is assigned to the cluster and the nodes of the scaling group. Other node groups in the cluster, except for ClickHouse Keeper, are assigned the UPDATING status
DELETING	The node group is being deleted. While this operation is in progress, other node groups in the cluster, except for ClickHouse Keeper, are assigned the UPDATING status
DELETED	The node group has been deleted
ERROR	An error has occurred, create a ticket

View node statuses in node groups

1. In the dashboard, click Products in the top menu and select Managed Databases.

2. Open the Active tab.

3. Open the cluster page → Connection tab.

4. In the Connection addresses block, check the Status column in the node line.

   CREATING	The node is being created
   ACTIVE	The node is active; no operations are being performed on it
   UPDATING	The node is being updated
   RESIZING	The node is scaling. This status appears when the node group containing the node is scaling
   DELETING	The node is being deleted
   DELETED	The node has been deleted
   DISK FULL	The node disk is full; the node is in read-only mode. To allow read and write operations on the node, clear the disk or scale the node group and select a configuration with a larger disk size than the previous configuration
   DOWN	The node is unavailable, create a ticket
   ERROR	An error has occurred, create a ticket

View operation history

1. In the dashboard, click Products in the top menu and select Managed Databases.
2. Open the Active tab.
3. Open the cluster page → Operation history tab.
4. View the operations.

View cluster node metrics

1. In the dashboard, click Products in the top menu and select Managed Databases.
2. Open the Active tab.
3. Open the cluster page → Monitoring tab.
4. Select a node group.
5. If there are multiple nodes in the group, select the node whose metrics you want to see.
6. View the available cluster node metrics.

Cluster node metrics in the dashboard

Memory	Occupied memory excluding OS cache and buffers, in percent or gigabytes
vCPU	Percentage of cluster node core utilization
CPU iowait	Percentage of time the processor spent waiting for input/output
Disk	Occupied disk space in percent or gigabytes. It takes into account the part of the disk space reserved for service needs and unavailable for database placement. For more information about disk space reservation, see the instructions Using disk space in a ClickHouse cluster
Load Average	The average system load over a period of time. It shows the number of processes being handled by the cluster cores. The value is presented as three metrics: for one minute, five minutes, and 15 minutes. These values must not exceed the number of cores on the node
OOM	Number of processes that terminated with an Out of Memory error due to insufficient RAM
Disk load	Read and write speed in KB/s or number of read/write operations per second
Network load	Number of bits or packets sent and received via the network interface