Create and configure a load balancer in a Managed Kubernetes cluster for Envoy Gateway

warning

After creating a Managed Kubernetes cluster, we recommend performing all actions with load balancers only via kubectl. Changes made in other ways are not saved in Kubernetes manifests. When recreating a cluster, load balancer, or synchronizing manifests, such changes will be reverted.

A load balancer in Managed Kubernetes is used to distribute incoming traffic between pods.

Create a load balancer

If you need to expose a service to the internet, use the Expose service to the internet instructions.

1. Check quotas.
2. Connect to the cluster.
3. Create an EnvoyProxy object.
4. Create a GatewayClass object.
5. Create a Gateway object.

1. Check quotas

Make sure that the pool has an allocated quota for at least one public floating IP address. To do this, view Cloud Platform quota usage.

2. Connect to the cluster

To connect to the cluster, use the Connect to a Managed Kubernetes cluster instructions.

3. Create an EnvoyProxy object

1. Create a YAML file with the manifest for the EnvoyProxy object.

   EnvoyProxy manifest example:

   ---
   apiVersion: gateway.envoyproxy.io/v1alpha1
   kind: EnvoyProxy
   metadata:
       name: custom-proxy-config
       namespace: default
   spec:
       provider:
           type: Kubernetes
           kubernetes:
               envoyService:
                   externalTrafficPolicy: Cluster
                   type: LoadBalancer

2. Apply the manifest:

   kubectl apply -f <file_name>

   Specify <file_name> — the name of the YAML file with the manifest for creating the EnvoyProxy object. For example, envoyproxy.yaml.

4. Create a GatewayClass object

1. Create a YAML file with the manifest for the GatewayClass object.

   GatewayClass manifest example:

   ---
   apiVersion: gateway.networking.k8s.io/v1
   kind: GatewayClass
   metadata:
       name: eg
   spec:
       controllerName: gateway.envoyproxy.io/gatewayclass-controller
       parametersRef:
           group: gateway.envoyproxy.io
           kind: EnvoyProxy
           name: custom-proxy-config
           namespace: default

2. Apply the manifest:

   kubectl apply -f <file_name>

   Specify <file_name> — the name of the YAML file with the manifest for creating the GatewayClass object. For example, gatewayclass.yaml.

5. Create a Gateway object

1. Create a YAML file with the manifest for the Gateway object.

   Gateway manifest example:

   ---
   apiVersion: gateway.networking.k8s.io/v1
   kind: Gateway
   metadata:
       name: eg
   spec:
       gatewayClassName: eg
       infrastructure:
           annotations:
               loadbalancer.openstack.org/keep-floatingip: "true"
       listeners:
           - name: http
           protocol: HTTP
           port: 80

   In the annotations block of the values.yaml file, add the necessary parameters for the load balancer — learn more about load balancer parameters in the Configure load balancer subsection.

2. Apply the manifest:

   kubectl apply -f <file_name>

   Specify <file_name> — the name of the YAML file with the manifest for creating the Gateway object. For example, gateway.yaml.

The created load balancer will appear in the Control panel: in the top menu, click Products and select Cloud Servers → the Load Balancers section → the Load Balancers tab.

Configure a load balancer

Specify a flavor and load balancer type

By default, without specifying an annotation, a load balancer of the type Basic with redundancy is created.

To create a load balancer with a different type, use the annotation:

loadbalancer.openstack.org/flavor-id: "<flavor_id>"

Specify <flavor_id> — the flavor ID. Flavors correspond to load balancer types and determine the number of vCPUs, RAM, and the number of load balancer instances. For example, ac18763b-1fc5-457d-9fa7-b0d339ffb336 is the ID for creating a load balancer with the Advanced with redundancy type in the ru-9 pool. You can view the list of load balancer flavors in all pools in the table or view the list of load balancer flavors in a specific pool via OpenStack CLI.

You cannot change the type of an already created load balancer — you need to create a new load balancer with the required annotation.

Create a load balancer without a public IP address

By default, an unannotated balancer with a public IP address is created.

To create a load balancer without a public IP address, use the annotation:

service.beta.kubernetes.io/openstack-internal-load-balancer: "true"

This parameter cannot be changed for an existing load balancer — you need to create a new manifest with the required annotation.

Create a load balancer with an IP address from other subnets

By default, the load balancer is created in the same network as the cluster nodes, and a public IP address is assigned to it.

You can create a load balancer in any other subnet: public, private, or cross-project.

1. Specify a subnet. To do this, add the following annotation to the Gateway object manifest:

   loadbalancer.openstack.org/subnet-id: "<subnet_uuid>"

   Specify <subnet_uuid> — subnet ID, you can view it using openstack subnet list.

2. Disable the automatic creation of a public IP address. To do this, add the following annotation to the Gateway object manifest:

   service.beta.kubernetes.io/openstack-internal-load-balancer: "true"

3. Specify an IP address for the load balancer. To do this, modify the EnvoyProxy object manifest:

   ---
   apiVersion: gateway.envoyproxy.io/v1alpha1
   kind: EnvoyProxy
   metadata:
       name: custom-proxy-config
       namespace: default
   spec:
       provider:
           type: Kubernetes
           kubernetes:
               envoyService:
                   externalTrafficPolicy: Cluster
                   type: LoadBalancer
                   loadBalancerIP: "<ip_address>"

   Specify <ip_address> — the load balancer IP address from the subnet you selected in step 1.

   You cannot change the type of an already created load balancer — you need to create a new load balancer with the required annotation.

Add connection settings

The following annotations are used to manage connection settings between incoming requests and the load balancer, or between the load balancer and servers:

• maximum connections;
• connection timeout for incoming requests;
• connection timeout for balancer requests to servers;
• inactivity timeout;
• TCP timeout.

Connection settings are configured for a load balancer rule. You can view the connection settings specified in the annotations in the Control panel: in the top menu, click Products and select Cloud Servers → the Load Balancers section → the Load Balancers tab → load balancer page → open the rule card → open the Advanced rule settings block.

Maximum connections

To specify the maximum number of connections, add the following annotation to the Gateway object manifest:

loadbalancer.openstack.org/connection-limit: "<value>"

Specify <value> — the maximum number of connections per second. Default is -1 (unlimited).

You can update this parameter for an existing load balancer.

The parameter can be viewed in the Control panel: in the top menu, click Products and select Cloud Servers → the Load Balancers section → the Load Balancers tab → load balancer page → open the rule card → open the Advanced rule settings block → the Requests incoming to load balancer block → the Maximum connections field.

Connection timeout for incoming requests

To specify the connection timeout for incoming requests to the load balancer, add the following annotation to the Gateway object manifest:

loadbalancer.openstack.org/timeout-client-data: "<value>"

Specify <value> — timeout value in milliseconds. Default is 50000.

You can update this parameter for an existing load balancer.

The parameter can be viewed in the Control panel: in the top menu, click Products and select Cloud Servers → the Load Balancers section → the Load Balancers tab → load balancer page → open the rule card → open the Advanced rule settings block → the Requests incoming to load balancer block → the Connection timeout, ms field.

Connection timeout for balancer requests to servers

To specify the connection timeout for balancer requests to servers, add the following annotation to the Gateway object manifest:

loadbalancer.openstack.org/timeout-member-connect: "<value>"

Specify <value> — timeout value in milliseconds. Default is 5000.

You can update this parameter for an existing load balancer.

The parameter can be viewed in the Control panel: in the top menu, click Products and select Cloud Servers → the Load Balancers section → the Load Balancers tab → load balancer page → open the rule card → open the Advanced rule settings block → the Requests from load balancer to servers block → the Connection timeout, ms field.

Inactivity timeout

The inactivity timeout for requests from the balancer to servers is the time during which the current connection is considered "alive", even if no data is being transmitted.

To specify the inactivity timeout, add the following annotation to the Gateway object manifest:

loadbalancer.openstack.org/timeout-member-data: "<value>"

Specify <value> — timeout value in milliseconds. Default is 50000.

You can update this parameter for an existing load balancer.

The parameter can be viewed in the Control panel: in the top menu, click Products and select Cloud Servers → the Load Balancers section → the Load Balancers tab → load balancer page → open the rule card → open the Advanced rule settings block → the Requests from load balancer to servers block → the Inactivity timeout, ms field.

TCP waiting timeout

When establishing a new TCP session, data is sometimes not transmitted immediately. This parameter defines the time during which the load balancer waits for data transmission for inspection over an already established connection.

To specify the TCP waiting timeout for balancer requests to servers, add the following annotation to the Gateway object manifest:

loadbalancer.openstack.org/timeout-tcp-inspect: "<value>"

Specify <value> — timeout value in milliseconds. Default is 0.

You can update this parameter for an existing load balancer.

The parameter can be viewed in the Control panel: in the top menu, click Products and select Cloud Servers → the Load Balancers section → the Load Balancers tab → load balancer page → open the rule card → open the Advanced rule settings block → the Requests from load balancer to servers block → the TCP timeout, ms field.

Enable rule validation

To enable or disable validation for rules, add the following annotation to the Gateway object manifest:

loadbalancer.openstack.org/enable-health-monitor: "<value>"

Specify <value> — rule validation status: true to enable validation, or false to disable validation. Default value is true.

You can update this parameter for an existing load balancer.

Save the client IP address

To receive the client IP address, add an X-Forwarded-For header or a TCP → PROXY rule.

Add an X-Forwarded-For header

Without specifying an annotation, the load balancer transfers only the original HTTP request body to the server, replacing the client IP address with its own.

For servers to receive this information for proper operation or analysis, include the X-Forwarded-For header in the request to the server. To do this, add an annotation to the Gateway object manifest:

loadbalancer.openstack.org/x-forwarded-for: "true"

The rule will use the HTTP → HTTP scheme instead of TCP → TCP. If you need to use HTTPS instead of the HTTP protocol, terminate the TLS connection.

You cannot change the type of an already created load balancer — you need to create a new load balancer with the required annotation.

Do not use with the PROXY protocol. When adding the TCP → Proxy rule, the X-Forwarded-For header is automatically passed to the service behind the load balancer.

Add a TCP → Proxy rule

The PROXY protocol is used to pass connection information from the source requesting the connection to the destination for which the connection was requested.

In the logs of the pod connected through a load balancer with TCP → PROXY rules, the real IP address of the user connecting will be specified, not the load balancer's address.

To create a load balancer with a TCP → PROXY rule, add the following annotation to the Gateway object manifest:

loadbalancer.openstack.org/proxy-protocol: "true"

The X-Forwarded-For header will automatically be passed to the service behind the load balancer. No additional annotations need to be added for it to work.

You cannot change the type of an already created load balancer — you need to create a new load balancer with the required annotation.

Save a public IP address

To save a public IP address when recreating a load balancer, add the following annotation to the Gateway object manifest:

loadbalancer.openstack.org/keep-floatingip: "true"

In the EnvoyProxy object manifest, in the loadBalancerIP field, specify this or another public IP address:

---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: custom-proxy-config
  namespace: default
spec:
  provider:
    type: Kubernetes
    kubernetes:
      envoyService:
        externalTrafficPolicy: Cluster
        type: LoadBalancer
        loadBalancerIP: "<ip_address>"

Specify <ip_address> — the public IP address that you want to preserve when recreating the load balancer.

You can use this annotation for an already created load balancer.