Vector

Vector is a high-performance tool for collecting, processing, and sending logs, metrics, and other monitoring data in your infrastructure.

With Vector, you can work with logs in the Logs service: add events from your own infrastructure.

Before you begin, configure Vector.

Configure Vector

Add a service user.
Issue an S3 key to the user.
Install the tool.
Configure event adding.

1. Add a service user

Add a service user with access permission in the Projects area and the role:

member or logs.admin for read and write access to logs;
logs.write for log write access;
reader or logs.viewer for log read-only access.

Users can be added by the Account Owner or users with the iam.admin role.

2. Issue an S3 key to the user

Users with access to the control panel can issue S3 keys for themselves, but we recommend creating service users and issuing S3 keys to them.

Only the Account Owner or a user with the iam.admin role can issue S3 keys to other users. A service user cannot get an S3 key independently because they do not have access to the control panel — the Account Owner or iam.admin.

You must create a separate key for each project. You can issue multiple keys for one project.

In the control panel, click IAM.
Go to the section for the required user type:
- Users — for users with access to the control panel;
- Service users — for service users.
Open the user page → in the Access tab.
In the S3 keys block, click Add key.
Enter a key name.
Select a project for which the key will work.
Click Generate. Two values will be generated:
- Access key — Access Key ID, a key identifier;
- Secret key — Secret Access Key, a secret key.
Click Copy and save the key — you will not be able to view it after closing the window.

3. Install the tool

Use the Install Vector instructions in the official Vector documentation.

4. Configure the tool to add events

Open the CLI.
Open the /etc/vector/vector.yaml configuration file in the nano text editor:
```
nano /etc/vector/vector.yaml
```

Add the configuration to add logs to the vector.yaml file. Example configuration:

sources:
journald:
    type: "journald"

transforms:
clean_systemd:
    type: "remap"
    inputs: ["journald"]
    source: |
        . = {
            "hostname": get_hostname!(),
            "message": .message,
            "priority": .PRIORITY,
            "syslog_facility": .SYSLOG_FACILITY,
            "syslog_identifier": .SYSLOG_IDENTIFIER,
            "syslog_timestamp": .SYSLOG_TIMESTAMP,
            "pid": ._PID,
            "runtime_scope": ._RUNTIME_SCOPE
        }
        . = compact(., nullish: true)

        .timestamp = now()

filter_important:
    type: filter
    inputs: ["clean_systemd"]
    condition: |
    to_int!(.priority) <= 4

sinks:
cloudwatch:
    type: "aws_cloudwatch_logs"
    inputs: ["filter_important"]
    group_name: <logs_group_name>
    stream_name: <logs_stream_name>
    region: <pool>
    endpoint: <log_endpoint>
    encoding:
    codec: "json"
    auth:
    access_key_id: <access_key>
    secret_access_key: <secret_key>

Specify:

<log_group_name> — the name of the log group to which events should be added, for example, user-log-group;
<log_stream_name> — the name of the stream to which events should be added, for example, user-log-stream;
<pool> — pool, for example kz-1;
<log_endpoint> — the URL for accessing the Logs service API. You can find the list of URLs in the List of URLs instructions;
<access_key> — the value of the Access key field from the S3 key;
<secret_key> — the value of the Secret key field from the S3 key.

Exit the nano text editor and save your changes: press Ctrl+X, then press Y+Enter.

Run Vector:

vector --config-yaml /etc/vector/vector.yaml