Create and configure a load balancer in a Managed Kubernetes cluster for Envoy Gateway

warning

Once a Managed Kubernetes cluster has been created, we recommend that all balancer actions be performed only via kubectl. Changes made in any other way are not saved in Kubernetes manifests. When recreating the cluster, balancer or synchronizing manifests, such changes will be undone.

The load balancer in Managed Kubernetes is used to distribute incoming traffic between pods.

1. Check quotas

Make sure that the pool has a quota of at least one public IP address allocated. To do this, see the quota consumption of the cloud platform.

2. Create EnvoyProxy

Create a yaml file with a manifest for the EnvoyProxy object.

Example of EnvoyProxy manifest:

---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
    name: custom-proxy-config
    namespace: default
spec:
    provider:
        type: Kubernetes
        kubernetes:
            envoyService:
                externalTrafficPolicy: Cluster
                type: LoadBalancer

Apply the manifest:
```
kubectl apply -f <envoyproxy.yaml>
```
Specify <envoyproxy.yaml> is the name of the yaml file with the manifest to create the EnvoyProxy object.

3. Create GatewayClass

Create a yaml file with a manifest for the GatewayClass object.

GatewayClass manifest example:

---
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
    name: eg
spec:
    controllerName: gateway.envoyproxy.io/gatewayclass-controller
    parametersRef:
        group: gateway.envoyproxy.io
        kind: EnvoyProxy
        name: custom-proxy-config
        namespace: default

Apply the manifest:
```
kubectl apply -f <gatewayclass.yaml>
```
Specify <gatewayclass.yaml> is the name of the manifest yaml file to create the GatewayClass object.

4. Create Gateway

Create a yaml file with a manifest for the Gateway object.

Gateway manifest example:

---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
    name: eg
spec:
    gatewayClassName: eg
    infrastructure:
        annotations:
            loadbalancer.openstack.org/keep-floatingip: "true"
    listeners:
        - name: http
        protocol: HTTP
        port: 80

In the annotations block you can pass additional parameters for the load balancer. The instructions provide frequently used annotations that may be useful when creating a load balancer.

Apply the manifest:
```
kubectl apply -f <gateway.yaml>
```
Specify <gateway.yaml> is the name of the manifest yaml file to create the Gateway object.

The created load balancer will appear in the Control Panel: in the top menu, click Products and select Cloud Servers → Balancers section → Balancers tab.

Configure the load balancer

Specify flavor and balancer type

By default, without specifying an annotation, a load balancer of type Basic with redundancy is created.

To create a balancer with a different type, use the annotation:

loadbalancer.openstack.org/flavor-id: "<flavor_id>"

Specify <flavor_id> - flavor ID. Flavor IDs correspond to load balancer types and define the number of vCPUs, RAM, and the number of load balancer instances. For example, ac18763b-1fc5-457d-9fa7-b0d339ffb336 is the ID to create a balancer with the Advanced type with redundancy in the ru-9 pool. You can view the list of load balancer flavors in all pools in the table or view the list of load balancer flavors in a specific pool through the OpenStack CLI.

You cannot change the type for a created balancer - you must create a new manifest with the desired annotation.

Create a balancer without a public IP address

By default, an unannotated balancer with a public IP address is created.

To create a balancer without a public IP address, use the annotation:

service.beta.kubernetes.io/openstack-internal-load-balancer: "true"

You cannot replace a parameter for a created balancer - you must create a new manifest with the desired annotation.

Create a balancer with IP address from other subnets

By default, the balancer is created on the same network as the cluster nodes and a public IP address is allocated to it.

You can create a balancer on any other subnet - public, private, or cross-project.

Add a subnet annotation to the manifest:
```
loadbalancer.openstack.org/subnet-id: "<subnet_uuid>"
```
Specify <subnet_uuid> - subnet ID, can be viewed with openstack subnet list.
To ensure that a public IP address is not automatically created, add an annotation:
```
service.beta.kubernetes.io/openstack-internal-load-balancer: "true"
```

To specify the IP address of the balancer, modify the manifest for the EnvoyProxy object:

---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
    name: custom-proxy-config
    namespace: default
spec:
    provider:
        type: Kubernetes
        kubernetes:
            envoyService:
                externalTrafficPolicy: Cluster
                type: LoadBalancer
                loadBalancerIP: "<ip_address>"

Specify <ip_address> is the IP address of the balancer from the subnet you selected in step 1.

You cannot replace the annotation for a created balancer - you must create a new manifest with the desired annotation.

Add connection settings

Annotations are used to control connection settings between incoming requests and the balancer or between the balancer and servers:

maximum connections;
connection timeout for incoming requests;
connection timeout for balancer requests to servers;
inactivity timeout;
TCP wait timeout.

Connection settings are set for the balancer rule. The connection settings set in the annotations can be viewed in the control panel: in the top menu, click Products and select Cloud Servers → Balancers section → Balancers tab → Balancers page → open the rule card → open the Advanced Rule Settings block.

Maximum connections

To specify the maximum connections, use annotation:

loadbalancer.openstack.org/connection-limit: "<value>"

Specify <value> - the maximum number of connections per second. The default value is "-1" (not limited).

You can update a parameter in an already created balancer.

You can view the parameter in the Control Panel: in the top menu, click Products and select Cloud Servers → Balancers section → Balancers tab → Balancers page → open the rule card → open the Advanced Rule Settings block → the Incoming Requests to Balancer block → the Maximum Connections field.

Connection timeout for incoming requests

To specify the connection timeout for incoming requests to the balancer, use the annotation:

loadbalancer.openstack.org/timeout-client-data: "<value>"

Specify <value> - timeout value in milliseconds. The default value is "50000".

You can update a parameter in an already created balancer.

The parameter can be viewed in the control panel: in the top menu, click Products and select Cloud Servers → Balancers section → Balancers tab → Balancers page → open the rule card → open the Advanced rule settings block → the Incoming requests to the balancer block → the Connection timeout, ms field.

Connection timeout for balancer requests to servers

To specify the connection timeout for balancer requests to servers, use the annotation:

loadbalancer.openstack.org/timeout-member-connect: "<value>"

Specify <value> - timeout value in milliseconds. The default value is "5000".

You can update a parameter in an already created balancer.

You can view the parameter in the control panel: in the top menu, click Products and select Cloud Servers → Balancers section → Balancers tab → Balancers page → open the rule card → open the Advanced rule settings block → the Requests from balancer to servers block → the Connection timeout, ms field.

Inactivity timeout

The inactivity timeout for balancer requests to servers is the amount of time that the current connection is considered "alive" even if no data is being transferred.

To specify the inactivity timeout, use the annotation:

loadbalancer.openstack.org/timeout-member-data: "<value>"

Specify <value> - timeout value in milliseconds. The default value is "50000".

You can update a parameter in an already created balancer.

The parameter can be viewed in the control panel: in the top menu, click Products and select Cloud Servers → Balancers section → Balancers tab → Balancers page → open the rule card → open the Advanced Rule Settings block → open the Requests from Balancer to Servers block → the Inactivity Timeout, ms field.

TCP wait timeout

When a new TCP session is established, data is sometimes not transmitted immediately. The parameter defines the time during which the balancer waits for data transmission for inspection on an already established connection.

To specify the TCP wait timeout for balancer requests to servers, use the annotation:

loadbalancer.openstack.org/timeout-tcp-inspect: "<value>"

Specify <value> - timeout value in milliseconds. The default value is "0".

You can update a parameter in an already created balancer.

The parameter can be viewed in the control panel: in the top menu, click Products and select Cloud Servers → Balancers section → Balancers tab → Balancers page → open the rule card → open the Advanced Rule Settings block → the Requests from Balancer to Servers block → the TCP Timeout, ms field.

Enable rule checking

To enable or disable validation for rules, use the annotation:

loadbalancer.openstack.org/enable-health-monitor: "<value>"

Specify <value> - "true" to enable or "false" to disable. The default value is "true".

You can update a parameter in an already created balancer.

Save the client's IP address

To receive the client IP address, add an X-Forwarded-For header or add a TCP → PROXY rule.

Add header X-Forwarded-For

Without specifying an annotation, the balancer passes only the original body of the HTTP request to the server, replacing the client's IP address with its own.

To ensure that servers receive this information for correct operation or analysis, include an X-Forwarded-For header in the request to the server, use the annotation:

loadbalancer.openstack.org/x-forwarded-for: "true"

The rule will use the HTTP → HTTP scheme instead of TCP → TCP. If you want to use HTTPS instead of the HTTP protocol, terminate the TLS connection.

You cannot replace a parameter for a created balancer - you must create a new manifest with the desired annotation.

Do not use together with PROXY protocol. When a TCP → Proxy rule is added, the X-Forwarded-For header automatically goes to the service behind the balancer.

Add TCP → Proxy rule

The PROXY protocol is used to transfer connection information from the source requesting the connection to the destination for which the connection was requested.

The logs of a feed to which a connection is made through a balancer with TCP → PROXY rules will show the real IP address of the connecting person, not the address of the balancer.

To create a balancer with a TCP → PROXY rule, use the annotation:

loadbalancer.openstack.org/proxy-protocol: "true"

The X-Forwarded-For header will automatically go to the service behind the balancer. No additional annotations need to be added for it to work.

You cannot replace a parameter for a created balancer - you must create a new manifest with the desired annotation.

Save the public IP address

To retain the public IP address when recreating the balancer, use the annotation:

loadbalancer.openstack.org/keep-floatingip: "true"

In the EnvoyProxy object manifest, specify this or another public IP address in the loadBalancerIP field:

---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: custom-proxy-config
  namespace: default
spec:
  provider:
    type: Kubernetes
    kubernetes:
      envoyService:
        externalTrafficPolicy: Cluster
        type: LoadBalancer
        loadBalancerIP: "<ip_address>"

You can use the annotation for an already created balancer.