Monitoring of Redis cluster, nodes and databases

In Redis cloud databases, you can monitor the health of the cluster.

To assess the overall health of the cluster ,look at the status of the cluster.

For a more detailed analysis, you can:

view the status of the cluster nodes — in the form of graphs in the control panel;
view the status of databases — in the form of graphs in the control panel;
export cluster and database node metrics in Prometheus format.

The time in the control panel corresponds to the time set on your device and is independent of the region where the cluster is hosted.

note

For example, you have created a cluster in Tashkent, in pool uz-1. Tashkent is in the UTC+5 time zone. On the device from which you logged into the control panel, the time zone is set to UTC+3. The time on the metrics charts will be displayed in UTC+3.

View cluster status

In the Dashboard, on the top menu, click Products and select Cloud Databases.
Open the Active tab.

In the cluster row, look at the status.

ACTIVE	The cluster is available
CREATING	A cluster is created
UPDATING	Changes are applied to the cluster
RESIZING	The cluster is scalable
ERROR	There was an error, create a ticket
DISK FULL	The disk is full and the cluster is read-only. To make the cluster read-write, scale the cluster and select a configuration with a larger disk size
DEGRADED	Some nodes in the cluster are unavailable
DELETING	The cluster is being deleted

View the status of the node cluster

In the Dashboard, on the top menu, click Products and select Cloud Databases.
Open the Active tab.
Open the cluster page → Monitoring tab.
In the Cluster Monitoring block, click Cluster Nodes.
Select the nodes whose metrics you want to view.
Look at the available metrics of the node cluster.

Cluster node metrics in the control panel

Memory	Memory utilization excluding cache and operating system buffers in percent or gigabytes
vCPU	How many percent of the node cluster cores are utilized
CPU iowait	How much percent of the processor's time was spent waiting for I/O
Disk	Used disk space in percent or gigabytes. This includes the portion of disk space that is reserved for service needs and is not available for hosting databases. For more information about reserving disk space, see Using Disk Space in a Redis Cluster
Load Average	The average value of system load over a period of time. Shows how many processes are processed by the cluster cores. The indicator is presented in the form of three values — for one minute, five minutes and 15 minutes. These values should not be more than the number of cores on the node
OOM	Number of processes that ended with an `Out of Memory` error due to lack of RAM
Disk load	Data read and write speed in KB/s or number of read and write operations per second
Network load	The number of bits or packets sent and received over the network interface

View the status of the databases

In the Dashboard, on the top menu, click Products and select Cloud Databases.
Open the Active tab.
Open the cluster page → Monitoring tab.
In the Cluster Monitoring block, click Databases.
Select the nodes whose metrics you want to view.
Take a look at the available database metrics.

Database metrics in the dashboard

Preempted and obsolete keys	Two parameters are displayed: `Evicted` — number of keys evicted due to exceeding the memory limit; `Expired` — number of keys with expired lifetime
Number of keys	Number of keys in all databases and number of keys with a specified lifetime (TTL)
Connections	Number of connections to the cluster databases
Slow queries	Number of slow requests per second. Requests that take longer than 0.01 seconds to complete are considered slow requests
Inquiries	Number of successful and unsuccessful requests per second. Unsuccessful requests are those in which the key does not exist, is superseded due to exceeding the memory limit, or its lifetime (TTL) has expired

1. Get a token

The token gives access to the metrics of all project clusters in a single pool.

In the Dashboard, on the top menu, click Products and select Cloud Databases.
Open the Active tab.
Open the cluster page → Monitoring tab.
In the Tokens for Prometheus block, click Create token. The token will be generated automatically.
Copy the token. To do this, clickin the token line.

2. Get metrics in Prometheus format

Historical information for clusters is not available — metrics are requested only in real time. The list of all metrics that are supported in cloud databases and their description can be viewed in the Metrics table in Prometheus format.

Configuration file
CLI

Add to the Prometheus configuration file:
```
scrape_configs:
  - job_name: get-metrics-from-dbaas
    scrape_interval: 1m
    static_configs:
      - targets:
        - '<domain>'
    scheme: https
    authorization:
      type: Bearer
      credentials: <token>
```
Specify:
- <domain> — domain of the Managed Databases API. This is the part of the URL to access the API without https:// and /v1 for example uz-1.dbaas.api.servercore.com. URL depends on region and pool. you can look in URL list;
- <token> — the token you copied when receiving the token in step 5.
Open a page in your browser where Prometheus-formatted metrics will be available:
```
http://<ip_address>:9090/targets
```
Specify <ip_address> — the IP address where Prometheus is installed.
Independently configure monitoring and alerts for database clusters.

Open the CLI.
To get the metrics, submit a request:
```
curl -L "https://<domain>/metrics" -H "Authorization: Bearer <token>"
```
Specify:
- <domain> — domain of the Managed Databases API. This is the part of the URL to access the API without https:// and /v1 for example uz-1.dbaas.api.servercore.com. URL depends on region and pool. you can look in URL list;
- <token> — the token you copied when receiving the token in step 5.
Available metrics in Prometheus format will appear in the response.
Independently configure monitoring and alerts for database clusters.

Metrics in Prometheus format

Metrics in Prometheus format are provided for all clusters. A specific cluster can be found by the database cluster identifier in the ds_id label.

Infrastructure level metrics
Application level metrics

dbaas_memory_percent	Memory utilization excluding cache and operating system buffers (RAM) in percent
dbaas_memory_bytes	Occupied memory excluding cache and operating system buffers (RAM) in bytes
dbaas_oom_count	Number of processes that terminated with an `Out of Memory` error due to lack of RAM
dbaas_cpu	Percent vCPU utilization on database cluster nodes
dbaas_cpu_iowait	I/O waiting time in percent
dbaas_disk_percent	Occupied disk space in percent. It takes into account the part of disk space reserved for service needs and not available for databases. For more information about reserving disk space, see Using disk space in a Redis cluster in this manual
dbaas_disk_bytes	Occupied disk space in bytes. It takes into account the part of disk space reserved for service needs and not available for databases. For more information about reserving disk space, see Using disk space in a Redis cluster in this manual
dbaas_disk_read_iops	Number of read operations per second
dbaas_disk_write_iops	Number of recording operations per second
dbaas_disk_read_bytes	Disk read speed in bytes per second
dbaas_disk_write_bytes	Data write speed to disk in bytes per second
dbaas_node_load1	The average value of system load in one minute. Shows how many processes are processed by the cluster cores
dbaas_node_load5	The average system utilization over five minutes. Shows how many processes are processed by the cluster cores
dbaas_node_load15	The average value of system load over 15 minutes. Shows how many processes are processed by the cluster cores
dbaas_network_receive_bytes	Number of bytes received through the network interface
dbaas_network_transmit_bytes	Number of bytes sent through the network interface
dbaas_network_receive_packets	Number of packets received through the network interface per second
dbaas_network_transmit_packets	Number of packets sent through the network interface per second
dbaas_role	Role of the node: `0` — role unknown; `1` — master; `2` — replica

dbaas_connected_clients	Number of connections to the cluster databases
dbaas_keyspace_hits_total	Number of successful requests per second
dbaas_keyspace_misses_total	Number of failed queries per second. Failed requests are those that indicate that the key does not exist, has been superseded due to exceeding the memory limit, or its lifetime has expired
dbaas_db_keys	Total number of keys in all databases
dbaas_db_keys_expiring	Total number of obsolete keys
dbaas_evicted_keys_total	Number of keys preempted due to exceeding the memory limit
dbaas_expired_keys_total	Number of keys with expired lifetime
dbaas_slowlog_length	Number of slow requests per second. Requests that take longer than 0.01 seconds to complete are considered slow requests