Skip to main content

Kafka cluster logs

The Logs service is used for logging. More information about how the service works and its pricing can be found in the Logs service description guide. To view the availability of the Logs service in locations, see the Availability of logs in locations.

For a Kafka Managed Database cluster, events within the Kafka Controller and Kafka Broker components are logged.

To collect cluster logs in the Logs service, enable logging. You can retrieve logs in the Control Panel using API requests or via AWS CLI. If you no longer need to collect logs, you can disable logging.

Logs are stored for 30 days.

Enable logging

You can enable logging via the API or in the Control Panel:

Enable logging in an existing cluster

Logging uses a portion of the cluster's computing resources. To control the amount of consumed resources, monitor the cluster node metrics. For more information on how to view metrics, see the article Monitoring of the cluster and nodes Kafka.

  1. In the Control Panel, in the top menu, click Products and select Managed Databases.
  2. Open the Active tab.
  3. Open the cluster page → Settings tab.
  4. In the Logging block, in the Log service field, click Connect.
  5. Select a log group or create a new group.
  6. If you selected a new group, enter its name.
  7. Click Save.

Get logs

Follow the Get logs instruction in the Logs section.

Event streams kafka-controller.log-<instance_id> and kafka-broker.log-<instance_id>, where <instance_id> is a Kafka cluster node ID, have been created for the Kafka Managed Database cluster.

See the description of log fields for a Kafka cluster.

Log fields

_HOSTNAMECluster node name in the dbaas-<instance_id> format, where <instance_id> is a Kafka cluster node ID
SYSLOG_IDENTIFIERLog source name
SYSLOG_TIMESTAMPEvent time in the log
MESSAGEDescription of the event submitted by the log source

Disable logging

  1. In the Dashboard, on the top menu, click Products and select Managed Databases.
  2. Open the Active tab.
  3. Open the cluster page → Settings tab.
  4. In the Logging block in the Log Service field, click .
  5. Uncheck the Collect technical cluster logs to log service checkbox.
  6. Click Save.