Vector

Vector is a high-performance tool for collecting, processing, and sending logs, metrics, and other monitoring data in your infrastructure.

With Vector, you can work with logs in the Logs service: add events from your own infrastructure.

Before you begin, configure Vector.

Configure Vector

Add a service user.
Issue an S3 key to the user.
Install the tool.
Configure adding events.

1. Add a service user

Add a service user with permission in the Projects scope and role:

member or logs.admin for read and write access to logs;
logs.writer for write access to logs;
reader or logs.viewer for read-only access to logs.

Users can be added by the Account Owner or users with the iam.admin role.

2. Issue an S3 key to the user

Control panel users can issue S3 keys to themselves, but we recommend creating service users and issuing S3 keys to them.

Only the Account Owner or a user with the iam.admin role can issue S3 keys to other users. A service user cannot obtain an S3 key independently because they do not have access to the control panel — the Account Owner or iam.admin must issue a key to them.

You must create a separate key for each project. You can issue multiple keys for a single project.

In the control panel, in the top menu, click IAM.
Go to the section with the desired user type:
- Control panel users — for users with access to the control panel;
- Service users — for users with programmatic access without access to the control panel.
Open the user page → Access tab.
In the S3 keys block, click Add key.
Enter a key name.
Select the project for which the key will work.
Click Generate. Two values will be generated:
- Access key — Access Key ID, a key identifier;
- Secret key — Secret Access Key, a secret key.
Click Copy and save the key — it cannot be viewed after closing the window.

3. Install the tool

Use the Install Vector guide from the official Vector documentation.

4. Configure the tool to add events

Open the CLI.
Open the /etc/vector/vector.yaml configuration file in the nano text editor:
```
nano /etc/vector/vector.yaml
```

Add the configuration for adding logs to the vector.yaml file. Example configuration:

sources:
journald:
    type: "journald"

transforms:
clean_systemd:
    type: "remap"
    inputs: ["journald"]
    source: |
        . = {
            "hostname": get_hostname!(),
            "message": .message,
            "priority": .PRIORITY,
            "syslog_facility": .SYSLOG_FACILITY,
            "syslog_identifier": .SYSLOG_IDENTIFIER,
            "syslog_timestamp": .SYSLOG_TIMESTAMP,
            "pid": ._PID,
            "runtime_scope": ._RUNTIME_SCOPE
        }
        . = compact(., nullish: true)

        .timestamp = now()

filter_important:
    type: filter
    inputs: ["clean_systemd"]
    condition: |
    to_int!(.priority) <= 4

sinks:
cloudwatch:
    type: "aws_cloudwatch_logs"
    inputs: ["filter_important"]
    group_name: <logs_group_name>
    stream_name: <logs_stream_name>
    region: <pool>
    endpoint: <log_endpoint>
    encoding:
    codec: "json"
    auth:
    access_key_id: <access_key>
    secret_access_key: <secret_key>

Specify:

<log_group_name> — name of the log group to add events to, for example, user-log-group;
<log_stream_name> — name of the stream to add events to, for example, user-log-stream;
<pool> — pool, for example, kz-1 ;
<log_endpoint> — URL for accessing the Cloud Logging service API. You can find the list of URLs in the Cloud Logging subsection of the List of URLs manual;
<access_key> — value of the Access key field from the S3 key;
<secret_key> — value of the Secret key field from the S3 key.

Exit the nano text editor saving changes: press Ctrl+X and then Y+Enter.

Run Vector:

vector --config-yaml /etc/vector/vector.yaml