Create and configure a load balancer in a Managed Kubernetes cluster for Envoy Gateway

warning

After creating a Managed Kubernetes cluster, we recommend performing all actions with load balancers only via kubectl. Changes made in any other way are not saved in Kubernetes manifests. If the cluster or load balancer is recreated, or if manifests are synchronized, such changes will be reverted.

A load balancer in Managed Kubernetes is used to distribute incoming traffic between pods.

Create a load balancer

If you need to expose a service to the internet, use the Expose a service to the internet guide.

1. Check quotas.
2. Connect to the cluster.
3. Create an EnvoyProxy object.
4. Create a GatewayClass object.
5. Create a Gateway object.

1. Check quotas

Ensure that the pool has a quota for at least one public floating IP address. To do this, view your cloud platform quota consumption.

2. Connect to the cluster

To connect to the cluster, use the Connect to a Managed Kubernetes cluster guide.

3. Create an EnvoyProxy object

1. Create a YAML file with the manifest for the EnvoyProxy object.

   EnvoyProxy manifest example:

   ---
   apiVersion: gateway.envoyproxy.io/v1alpha1
   kind: EnvoyProxy
   metadata:
       name: custom-proxy-config
       namespace: default
   spec:
       provider:
           type: Kubernetes
           kubernetes:
               envoyService:
                   externalTrafficPolicy: Cluster
                   type: LoadBalancer

2. Apply the manifest:

   kubectl apply -f <file_name>

   Specify <file_name> — the name of the YAML file with the manifest for creating the EnvoyProxy object. For example, envoyproxy.yaml.

4. Create a GatewayClass object

1. Create a YAML file with the manifest for the GatewayClass object.

   GatewayClass manifest example:

   ---
   apiVersion: gateway.networking.k8s.io/v1
   kind: GatewayClass
   metadata:
       name: eg
   spec:
       controllerName: gateway.envoyproxy.io/gatewayclass-controller
       parametersRef:
           group: gateway.envoyproxy.io
           kind: EnvoyProxy
           name: custom-proxy-config
           namespace: default

2. Apply the manifest:

   kubectl apply -f <file_name>

   Specify <file_name> — the name of the YAML file with the manifest for creating the GatewayClass object. For example, gatewayclass.yaml.

5. Create a Gateway object

1. Create a YAML file with the manifest for the Gateway object.

   Gateway manifest example:

   ---
   apiVersion: gateway.networking.k8s.io/v1
   kind: Gateway
   metadata:
       name: eg
   spec:
       gatewayClassName: eg
       infrastructure:
           annotations:
               loadbalancer.openstack.org/keep-floatingip: "true"
       listeners:
           - name: http
           protocol: HTTP
           port: 80

   In the annotations block of the values.yaml file, add the necessary parameters for the load balancer — learn more about load balancer parameters in the Configure a load balancer subsection.

2. Apply the manifest:

   kubectl apply -f <file_name>

   Specify <file_name> — the name of the YAML file with the manifest for creating the Gateway object. For example, gateway.yaml.

The created load balancer will appear in the Control Panel: in the top menu, click Products and select Cloud Servers → section Load Balancers → tab Load Balancers.

Configure a load balancer

Specify a flavor and load balancer type

By default, if no annotation is specified, a load balancer is created of the Basic with redundancy type.

To create a load balancer with a different type, use the annotation:

loadbalancer.openstack.org/flavor-id: "<flavor_id>"

Specify <flavor_id> — the flavor ID. Flavors correspond to load balancer types and determine the number of vCPU, RAM, and the number of load balancer instances. For example, ac18763b-1fc5-457d-9fa7-b0d339ffb336 is the ID for creating an Advanced with redundancy load balancer in the ru-9 pool. You can view the full list of load balancer flavors in a table or view the list of load balancer flavors in a specific pool via the OpenStack CLI.

You cannot change the type of an existing load balancer — you must create a new load balancer with the required annotation.

Create a load balancer without a public IP address

By default, an unannotated balancer with a public IP address is created.

To create a load balancer without a public IP address, use the annotation:

service.beta.kubernetes.io/openstack-internal-load-balancer: "true"

This parameter cannot be changed for an existing load balancer — you need to create a new manifest with the required annotation.

Create a load balancer with an IP address from other subnets

By default, the load balancer is created in the same network as the cluster nodes, and a public IP address is assigned to it.

You can create a load balancer in any other subnet: public, private, or cross-project.

1. Specify a subnet. To do this, add the following annotation to the Gateway object manifest:

   loadbalancer.openstack.org/subnet-id: "<subnet_uuid>"

   Specify <subnet_uuid> — the subnet ID; you can view it using openstack subnet list.

2. Disable the automatic creation of a public IP address. To do this, add the following annotation to the Gateway object manifest:

   service.beta.kubernetes.io/openstack-internal-load-balancer: "true"

3. Specify an IP address for the load balancer. To do this, modify the EnvoyProxy object manifest:

   ---
   apiVersion: gateway.envoyproxy.io/v1alpha1
   kind: EnvoyProxy
   metadata:
       name: custom-proxy-config
       namespace: default
   spec:
       provider:
           type: Kubernetes
           kubernetes:
               envoyService:
                   externalTrafficPolicy: Cluster
                   type: LoadBalancer
                   loadBalancerIP: "<ip_address>"

   Specify <ip_address> — the load balancer IP address from the subnet you selected in step 1.

   You cannot change the type of an existing load balancer — you must create a new load balancer with the required annotation.

Add connection settings

The following annotations are used to manage connection settings between incoming requests and the load balancer, or between the load balancer and servers:

• maximum connections;
• connection timeout for incoming requests;
• connection timeout for balancer requests to servers;
• inactivity timeout;
• TCP waiting timeout.

Connection settings are configured for a load balancer rule. You can view the connection settings configured in annotations in the Control Panel: in the top menu, click Products and select Cloud Servers → section Load Balancers → tab Load Balancers → load balancer page → open the rule card → open the Advanced rule settings.

Maximum connections

To specify the maximum number of connections, add the following annotation to the Gateway object manifest:

loadbalancer.openstack.org/connection-limit: "<value>"

Specify <value> — the maximum number of connections per second. Default is -1 (unlimited).

You can update this parameter for an existing load balancer.

You can view this parameter in the Control Panel: in the top menu, click Products and select Cloud Servers → section Load Balancers → tab Load Balancers → load balancer page → open the rule card → open the Advanced rule settings → block Incoming requests to the balancer → field Maximum connections.

Connection timeout for incoming requests

To specify the connection timeout for incoming requests to the load balancer, add the following annotation to the Gateway object manifest:

loadbalancer.openstack.org/timeout-client-data: "<value>"

Specify <value> — the timeout value in milliseconds. Default is 50000.

You can update this parameter for an existing load balancer.

You can view this parameter in the Control Panel: in the top menu, click Products and select Cloud Servers → section Load Balancers → tab Load Balancers → load balancer page → open the rule card → open the Advanced rule settings → block Incoming requests to the balancer → field Connection timeout, ms.

Connection timeout for balancer requests to servers

To specify the connection timeout for balancer requests to servers, add the following annotation to the Gateway object manifest:

loadbalancer.openstack.org/timeout-member-connect: "<value>"

Specify <value> — the timeout value in milliseconds. Default is 5000.

You can update this parameter for an existing load balancer.

You can view this parameter in the Control Panel: in the top menu, click Products and select Cloud Servers → section Load Balancers → tab Load Balancers → load balancer page → open the rule card → open the Advanced rule settings → block Requests from balancer to servers → field Connection timeout, ms.

Inactivity timeout

The inactivity timeout for requests from the balancer to servers is the time during which the current connection is considered "alive", even if no data is being transmitted.

To specify the inactivity timeout, add the following annotation to the Gateway object manifest:

loadbalancer.openstack.org/timeout-member-data: "<value>"

Specify <value> — the timeout value in milliseconds. Default is 50000.

You can update this parameter for an existing load balancer.

You can view this parameter in the Control Panel: in the top menu, click Products and select Cloud Servers → section Load Balancers → tab Load Balancers → load balancer page → open the rule card → open the Advanced rule settings → block Requests from balancer to servers → field Inactivity timeout, ms.

TCP waiting timeout

When establishing a new TCP session, data is sometimes not transmitted immediately. This parameter defines the time during which the load balancer waits for data transmission for inspection over an already established connection.

To specify the TCP waiting timeout for balancer requests to servers, add the following annotation to the Gateway object manifest:

loadbalancer.openstack.org/timeout-tcp-inspect: "<value>"

Specify <value> — the timeout value in milliseconds. Default is 0.

You can update this parameter for an existing load balancer.

You can view this parameter in the Control Panel: in the top menu, click Products and select Cloud Servers → section Load Balancers → tab Load Balancers → load balancer page → open the rule card → open the Advanced rule settings → block Requests from balancer to servers → field TCP waiting timeout, ms.

Enable rule validation

To enable or disable validation for rules, add the following annotation to the Gateway object manifest:

loadbalancer.openstack.org/enable-health-monitor: "<value>"

Specify <value> — the rule validation status: true to enable validation, or false to disable it. The default value is true.

You can update this parameter for an existing load balancer.

Save the client IP address

To receive the client IP address, add an X-Forwarded-For header or a TCP → PROXY rule.

Add an X-Forwarded-For header

Without specifying an annotation, the load balancer transfers only the original HTTP request body to the server, replacing the client IP address with its own.

To ensure servers receive this information for correct operation or analysis, include the X-Forwarded-For header in the request to the server. To do this, add the following annotation to the Gateway object manifest:

loadbalancer.openstack.org/x-forwarded-for: "true"

The rule will use the HTTP → HTTP scheme instead of TCP → TCP. If you need to use HTTPS instead of the HTTP protocol, terminate the TLS connection.

You cannot change the type of an existing load balancer — you must create a new load balancer with the required annotation.

Do not use together with the PROXY protocol. When adding a TCP → Proxy rule, the X-Forwarded-For header is automatically passed to the service behind the load balancer.

Add a TCP → Proxy rule

The PROXY protocol is used to pass connection information from the source requesting the connection to the destination for which the connection was requested.

In the logs of the pod connected through a load balancer with TCP → PROXY rules, the real IP address of the user connecting will be specified, not the load balancer's address.

To create a load balancer with a TCP → PROXY rule, add the following annotation to the Gateway object manifest:

loadbalancer.openstack.org/proxy-protocol: "true"

The X-Forwarded-For header is automatically passed to the service behind the load balancer. You do not need to add additional annotations for it to work.

You cannot change the type of an existing load balancer — you must create a new load balancer with the required annotation.

Save a public IP address

To save a public IP address when recreating a load balancer, add the following annotation to the Gateway object manifest:

loadbalancer.openstack.org/keep-floatingip: "true"

In the EnvoyProxy object manifest, in the loadBalancerIP field, specify this or another public IP address:

---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: custom-proxy-config
  namespace: default
spec:
  provider:
    type: Kubernetes
    kubernetes:
      envoyService:
        externalTrafficPolicy: Cluster
        type: LoadBalancer
        loadBalancerIP: "<ip_address>"

Specify <ip_address> — the public IP address you want to save when the load balancer is recreated.

You can use this annotation for an already created load balancer.