Skip to main content

Kafka cluster logs

The Logs service is used for logging. For more information about how the service works and its cost, see the Logs service overview instructions. You can check the Logs service availability across locations in Logs availability by location.

For a Kafka Managed Database cluster, events within the Kafka Controller and Kafka Broker components are logged.

To collect cluster logs in the Logs service, enable logging. You can get logs in the control panel, using API requests, or via the AWS CLI. If you no longer need to collect logs, you can disable logging.

Logs are stored for 30 days.

Enable logging

You can enable logging via the API or in the Control Panel:

Enable logging in an existing cluster

Logging uses part of the cluster's computing resources. To control the amount of resources consumed, monitor cluster node metrics. For details on how to view metrics, see the Monitoring Kafka cluster and nodes instructions.

  1. In the control panel, in the top menu, click Products and select Managed Databases.
  2. Open the Active tab.
  3. Open the cluster page → Settings tab.
  4. In the Logging block, in the Log service field, click Connect.
  5. Select a log group or create a new group.
  6. If you selected a new group, enter its name.
  7. Click Save.

Get logs

Use the Get logs instructions in the Logs section.

For the Managed Databases Kafka cluster, the event streams kafka-controller.log-<instance_id> and kafka-broker.log-<instance_id> are created, where <instance_id> is the Kafka cluster node ID.

See the description of log fields for the Kafka cluster.

Log fields

_HOSTNAMECluster node name in the format dbaas-<instance_id>, where <instance_id> is the Kafka cluster node ID
SYSLOG_IDENTIFIERLog source name
SYSLOG_TIMESTAMPEvent time in the log
MESSAGEDescription of the event submitted by the log source

Disable logging

  1. In the control panel, in the top menu, click Products and select Cloud Databases.
  2. Open the Active tab.
  3. Open the cluster page → Settings tab.
  4. In the Logging section, in the Log service field, click .
  5. Clear the Collect cluster technical logs into log service checkbox.
  6. Click Save.