Disk space utilization in the Kafka cluster

In cloud databases, part of the disk space is reserved for service needs:

• for the file system — approximately 4% of the disk volume;
• for the operating system, service components and logs — approximately 8 GB.

The reserved portion of the disk space is not available to host databases. Keep this in mind when selecting a configuration lineup.

You can monitor disk occupancy using disk occupancy notifications and metrics. For more information about metrics, see the Kafka Cluster and Node Monitoring instructions.

When the cluster disk is 95% or more full, the cluster will go to DISK_FULL status and will be read-only. This is to prevent the cluster from becoming completely locked up or corrupted due to lack of free space. To make the cluster read-write ,free up disk space or scale the cluster and select a configuration with a disk size larger than the previous configuration.

Disk fullness notifications

Notifications of disk fullness are sent to the Account Owner's email and to users who are subscribed to the "Services and Services" notification category. Notifications are sent when the disk is 80% and 95% full.

Free up disk space

To free up disk space, you can:

• to delete the top;
• configure a data cleansing policy.

Delete topic

1. In the Dashboard, on the top menu, click Products and select Cloud Databases.
2. Open the Active tab.
3. Open the cluster page → Topics tab.
4. From themenu of the topic, select Delete.
5. Enter the name of the topic to confirm deletion.
6. Click Delete.

Customize the data cleansing policy

warning

After changing the log.retention.minutes and log.retention.hours parameters in the DBMS settings, the cluster will be restarted and will become unavailable for the restart time.

To speed up disk cleanup, change the storage policy in the cluster settings. Kafka will automatically delete data according to the new policy.

1. In the Dashboard, on the top menu, click Products and select Cloud Databases.
2. Open the Active tab.
3. Open the cluster page → Settings tab.
4. In the DBMS Settings block, click Edit.
5. Specify new parameter values to customize the data cleansing policy.
6. Click Save.

Options for customizing the data cleansing policy

log.segment.bytes	The maximum size of a single segment in the topic section in bytes. When the current segment is full, Kafka will create a new segment. The default value is 1,073,741,824
log.retention.bytes	The maximum size of the topic partition in bytes. When the partition is full, Kafka will start deleting old segments. Default value is -1 (maximum size is not limited)
log.retention.ms	The segment retention time in milliseconds. If not specified, the log.retention.minutes parameter is used. If -1 is specified, the retention time is not limited even if log. retention``.minutes or log.retention.hours is specified
log.retention.minutes	The segment retention time in minutes. If not specified, the log.retention.hours parameter is used
log.retention.hours	Segment retention time in hours. Used if the log.retention.ms and log.retention.minutes parameters are not set. The default value is 168