Create and configure a load balancer in a Managed Kubernetes cluster for Envoy Gateway

warning

Once a Managed Kubernetes cluster has been created, we recommend that all balancer actions be performed only via kubectl. Changes made in any other way are not saved in Kubernetes manifests. When recreating the cluster, balancer or synchronizing manifests, such changes will be undone.

The load balancer in Managed Kubernetes is used to distribute incoming traffic between pods.

1. Check quotas

Make sure that the pool has a quota of at least one public IP address allocated. To do this, see the quota consumption of the cloud platform.

2. Connect to the cluster

To connect to the cluster, use the Connect to a Managed Kubernetes Cluster instructions.

3. Create EnvoyProxy

Create a yaml file with a manifest for the EnvoyProxy object.

Example of EnvoyProxy manifest:

---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
    name: custom-proxy-config
    namespace: default
spec:
    provider:
        type: Kubernetes
        kubernetes:
            envoyService:
                externalTrafficPolicy: Cluster
                type: LoadBalancer

Apply the manifest:
```
kubectl apply -f <file_name>
```
Specify <file_name> - the name of the yaml file with the manifest to create the EnvoyProxy object. For example, envoyproxy.yaml.

4. Create GatewayClass

Create a yaml file with a manifest for the GatewayClass object.

GatewayClass manifest example:

---
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
    name: eg
spec:
    controllerName: gateway.envoyproxy.io/gatewayclass-controller
    parametersRef:
        group: gateway.envoyproxy.io
        kind: EnvoyProxy
        name: custom-proxy-config
        namespace: default

Apply the manifest:
```
kubectl apply -f <file_name>
```
Specify <file_name> - the name of the yaml file with the manifest to create the GatewayClass object. For example, gatewayclass.yaml.

5. Create Gateway

Create a yaml file with a manifest for the Gateway object.

Gateway manifest example:

---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
    name: eg
spec:
    gatewayClassName: eg
    infrastructure:
        annotations:
            loadbalancer.openstack.org/keep-floatingip: "true"
    listeners:
        - name: http
        protocol: HTTP
        port: 80

In the annotations block of the values.yaml file, add the necessary parameters for the load balancer - more information about load balancer parameters in the Configure Load Balancer subsection.

Apply the manifest:
```
kubectl apply -f <file_name>
```
Specify <file_name> - the name of the yaml file with the manifest to create the Gateway object. For example, gateway.yaml.

The created load balancer will appear in the Control Panel: in the top menu, click Products and select Cloud Servers → Balancers section → Balancers tab.

Configure the load balancer

Specify flavor and balancer type

By default, without specifying an annotation, a load balancer of type Basic with redundancy is created.

To create a balancer with a different type, use the annotation:

loadbalancer.openstack.org/flavor-id: "<flavor_id>"

Specify <flavor_id> - flavor ID. Flavor IDs correspond to load balancer types and define the number of vCPUs, RAM, and number of load balancer instances. For example, ac18763b-1fc5-457d-9fa7-b0d339ffb336 is the ID to create a balancer with the Advanced type with redundancy in the ru-9 pool. You can view the list of load balancer flavors in all pools in the table or view the list of load balancer flavors in a specific pool through the OpenStack CLI.

You cannot change the type for a created balancer - you must create a new balancer with the desired annotation.

Create a balancer without a public IP address

By default, an unannotated balancer with a public IP address is created.

To create a balancer without a public IP address, use the annotation:

service.beta.kubernetes.io/openstack-internal-load-balancer: "true"

You cannot replace a parameter for a created balancer - you must create a new manifest with the desired annotation.

Create a balancer with IP address from other subnets

By default, the balancer is created on the same network as the cluster nodes and a public IP address is allocated to it.

You can create a balancer on any other subnet - public, private, or cross-project.

Specify the subnet. To do this, add an annotation to the Gateway object manifest:
```
loadbalancer.openstack.org/subnet-id: "<subnet_uuid>"
```
Specify <subnet_uuid> - subnet ID, can be viewed with openstack subnet list.
Disable automatic creation of a public IP address. To do this, add an annotation to the Gateway object manifest:
```
service.beta.kubernetes.io/openstack-internal-load-balancer: "true"
```

Specify the IP address of the balancer. To do this, change the manifest of the EnvoyProxy object:

---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
    name: custom-proxy-config
    namespace: default
spec:
    provider:
        type: Kubernetes
        kubernetes:
            envoyService:
                externalTrafficPolicy: Cluster
                type: LoadBalancer
                loadBalancerIP: "<ip_address>"

Specify <ip_address> is the IP address of the balancer from the subnet you selected in step 1.

You cannot change the type for a created balancer - you must create a new balancer with the desired annotation.

Add connection settings

Annotations are used to control connection settings between incoming requests and the balancer or between the balancer and servers:

maximum connections;
connection timeout for incoming requests;
connection timeout for balancer requests to servers;
inactivity timeout;
TCP wait timeout.

Connection settings are set for the balancer rule. The connection settings set in the annotations can be viewed in the control panel: in the top menu, click Products and select Cloud Servers → Balancers section → Balancers tab → Balancers page → open the rule card → open the Advanced Rule Settings block.

Maximum connections

To specify the maximum connections, add an annotation to the Gateway object manifest:

loadbalancer.openstack.org/connection-limit: "<value>"

Specify <value> - the maximum number of connections per second. The default value is -1 (not limited).

You can update a parameter in an already created balancer.

You can view the parameter in the Control Panel: in the top menu, click Products and select Cloud Servers → Balancers section → Balancers tab → Balancers page → open the rule card → open the Advanced Rule Settings block → the Incoming Requests to Balancer block → the Maximum Connections field.

Connection timeout for incoming requests

To specify the connection timeout for incoming requests to the balancer, add an annotation to the Gateway object manifest:

loadbalancer.openstack.org/timeout-client-data: "<value>"

Specify <value> - timeout value in milliseconds. The default value is 50000.

You can update a parameter in an already created balancer.

The parameter can be viewed in the control panel: in the top menu, click Products and select Cloud Servers → Balancers section → Balancers tab → Balancers page → open the rule card → open the Advanced rule settings block → the Incoming requests to the balancer block → the Connection timeout, ms field.

Connection timeout for balancer requests to servers

To specify the connection timeout for balancer requests to servers, add an annotation to the Gateway object manifest:

loadbalancer.openstack.org/timeout-member-connect: "<value>"

Specify <value> - timeout value in milliseconds. The default value is 5000.

You can update a parameter in an already created balancer.

You can view the parameter in the control panel: in the top menu, click Products and select Cloud Servers → Balancers section → Balancers tab → Balancers page → open the rule card → open the Advanced rule settings block → the Requests from balancer to servers block → the Connection timeout, ms field.

Inactivity timeout

The inactivity timeout for balancer requests to servers is the amount of time that the current connection is considered "alive" even if no data is being transferred.

To specify the inactivity timeout, add an annotation to the Gateway object manifest:

loadbalancer.openstack.org/timeout-member-data: "<value>"

Specify <value> - timeout value in milliseconds. The default value is 50000.

You can update a parameter in an already created balancer.

The parameter can be viewed in the control panel: in the top menu, click Products and select Cloud Servers → Balancers section → Balancers tab → Balancers page → open the rule card → open the Advanced Rule Settings block → open the Requests from Balancer to Servers block → the Inactivity Timeout, ms field.

TCP wait timeout

When a new TCP session is established, data is sometimes not transmitted immediately. The parameter defines the time during which the balancer waits for data transmission for inspection on an already established connection.

To specify the TCP wait timeout for balancer requests to servers, add an annotation to the Gateway object manifest:

loadbalancer.openstack.org/timeout-tcp-inspect: "<value>"

Specify <value> - timeout value in milliseconds. The default value is 0.

You can update a parameter in an already created balancer.

The parameter can be viewed in the control panel: in the top menu, click Products and select Cloud Servers → Balancers section → Balancers tab → Balancers page → open the rule card → open the Advanced Rule Settings block → the Requests from Balancer to Servers block → the TCP Timeout, ms field.

Enable rule checking

To enable or disable validation for rules, add an annotation to the Gateway object manifest:

loadbalancer.openstack.org/enable-health-monitor: "<value>"

Specify <value> - rule checking status: true - to enable checking or false - to disable checking. The default value is true.

You can update a parameter in an already created balancer.

Save the client's IP address

To receive the client IP address, add an X-Forwarded-For header or a TCP → PROXY rule.

Add header X-Forwarded-For
Add TCP → Proxy rule

Without specifying an annotation, the balancer passes only the original body of the HTTP request to the server, replacing the client's IP address with its own.

To ensure that servers receive this information for correct operation or analysis, include an X-Forwarded-For header in the request to the server. To do this, add an annotation to the Gateway object manifest:

loadbalancer.openstack.org/x-forwarded-for: "true"

The rule will use the HTTP → HTTP scheme instead of TCP → TCP. If you want to use HTTPS instead of the HTTP protocol, terminate the TLS connection.

You cannot change the type for a created balancer - you must create a new balancer with the desired annotation.

Do not use together with PROXY protocol. When a TCP → Proxy rule is added, the X-Forwarded-For header automatically goes to the service behind the balancer.

The PROXY protocol is used to transfer connection information from the source requesting the connection to the destination for which the connection was requested.

The logs of a feed to which a connection is made through a balancer with TCP → PROXY rules will show the real IP address of the connecting person, not the address of the balancer.

To create a balancer with a TCP → PROXY rule, add an annotation to the Gateway object manifest:

loadbalancer.openstack.org/proxy-protocol: "true"

The X-Forwarded-For header will automatically go to the service behind the balancer. No additional annotations need to be added for it to work.

You cannot change the type for a created balancer - you must create a new balancer with the desired annotation.

Save the public IP address

To retain the public IP address when recreating the balancer, add an annotation to the Gateway object manifest:

loadbalancer.openstack.org/keep-floatingip: "true"

In the EnvoyProxy object manifest, specify this or another public IP address in the loadBalancerIP field:

---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: custom-proxy-config
  namespace: default
spec:
  provider:
    type: Kubernetes
    kubernetes:
      envoyService:
        externalTrafficPolicy: Cluster
        type: LoadBalancer
        loadBalancerIP: "<ip_address>"

Specify <ip_address> is the public IP address you want to keep when recreating the balancer.

You can use the annotation for an already created balancer.