Using disk space in a Kafka cluster

In Managed Databases, part of the disk space is reserved for service needs.The total amount of reserved space depends on the Kafka version.

	Kafka 3.5	Kafka 4.1
File system reservation	4% of disk capacity	4% of disk capacity
Reservation for the operating system, service components, and logs	8 GB	15 GB

The reserved portion of the disk space is not available to host databases.Keep this in mind when choosing a configuration lineup.

You can track disk usage using disk occupancy notifications and metrics. Read more about metrics in the Kafka cluster and node monitoring guide.

When a cluster disk is 95% full or more, the cluster will enter the DISK_FULL status and will operate in read-only mode.This is necessary to prevent the cluster from being completely locked or damaged due to a lack of free space.To make the cluster operate in read-write mode, free up disk space or scale the cluster and choose a configuration with a larger disk size than in the previous configuration.

Disk occupancy notifications

Disk occupancy notifications are sent to the email address of the Account Owner and users who are subscribed to the Services and Products notification category.Notifications are sent when the disk is 80% and 95% full.

Free up disk space

To free up disk space, you can:

• delete a topic;
• configure a data cleanup policy.

Delete a topic

1. In the Dashboard, on the top menu, click Products and select Managed Databases.
2. Open the Active tab.
3. Open the cluster page → Topics tab.
4. In the menu of the topic, select Delete.
5. Enter the topic name to confirm deletion.
6. Click Delete.

Configure a data cleanup policy

warning

After changing the log.retention.minutes and log.retention.hours parameters in the DBMS settings, the cluster will be restarted and will be unavailable during the restart.

To speed up disk clearing, change the data retention policy in the cluster settings.Kafka will automatically delete data based on the new rules.

1. In the Dashboard, on the top menu, click Products and select Managed Databases.
2. Open the Active tab.
3. Open the cluster page → Settings tab.
4. In the DBMS settings block, click Edit.
5. Specify the new values for the parameters for configuring a data cleanup policy.
6. Click Save.

Parameters for configuring a data cleanup policy

log.segment.bytes	The maximum size of a single segment in a topic partition in bytes. When the current segment is full, Kafka will create a new one. The default value is 1 073 741 824
log.retention.bytes	The maximum size of a topic partition in bytes. When the partition is full, Kafka will start deleting old segments. The default value is -1 (maximum size is unlimited)
log.retention.ms	The segment retention time in milliseconds. If not set, the log.retention.minutes parameter is used. If -1 is specified, the retention time is unlimited, even if the log.retention.minutes or log.retention.hours parameters are set
log.retention.minutes	The segment retention time in minutes. If not set, the log.retention.hours parameter is used
log.retention.hours	The segment retention time in hours. Used if the log.retention.ms and log.retention.minutes parameters are not specified. The default value is 168