Create and configure a load balancer in a Managed Kubernetes cluster for Envoy Gateway

warning

After creating a Managed Kubernetes cluster, we recommend performing all load balancer operations only via kubectl. Changes made using other methods are not saved in Kubernetes manifests. When you recreate a cluster or load balancer, or when syncing manifests, such changes will be reverted.

A load balancer in Managed Kubernetes is used to distribute incoming traffic between pods.

Create a load balancer

1. Check quotas.
2. Connect to the cluster.
3. Create an EnvoyProxy object.
4. Create a GatewayClass object.
5. Create a Gateway object.

1. Check quotas

Ensure that the pool has a quota for at least one public IP address. To do this, view your cloud platform quota consumption.

2. Connect to the cluster

To connect to the cluster, use the instructions in Connect to a Managed Kubernetes cluster.

3. Create an EnvoyProxy object

1. Create a yaml file with a manifest for the EnvoyProxy object.

   EnvoyProxy manifest example:

   ---
   apiVersion: gateway.envoyproxy.io/v1alpha1
   kind: EnvoyProxy
   metadata:
       name: custom-proxy-config
       namespace: default
   spec:
       provider:
           type: Kubernetes
           kubernetes:
               envoyService:
                   externalTrafficPolicy: Cluster
                   type: LoadBalancer

2. Apply the manifest:

   kubectl apply -f <file_name>

   Specify <file_name> — the name of the yaml file with the manifest for creating the EnvoyProxy object. For example, envoyproxy.yaml.

4. Create a GatewayClass object

1. Create a yaml file with a manifest for the GatewayClass object.

   GatewayClass manifest example:

   ---
   apiVersion: gateway.networking.k8s.io/v1
   kind: GatewayClass
   metadata:
       name: eg
   spec:
       controllerName: gateway.envoyproxy.io/gatewayclass-controller
       parametersRef:
           group: gateway.envoyproxy.io
           kind: EnvoyProxy
           name: custom-proxy-config
           namespace: default

2. Apply the manifest:

   kubectl apply -f <file_name>

   Specify <file_name> — the name of the yaml file with the manifest for creating the GatewayClass object. For example, gatewayclass.yaml.

5. Create a Gateway object

1. Create a yaml file with a manifest for the Gateway object.

   Gateway manifest example:

   ---
   apiVersion: gateway.networking.k8s.io/v1
   kind: Gateway
   metadata:
       name: eg
   spec:
       gatewayClassName: eg
       infrastructure:
           annotations:
               loadbalancer.openstack.org/keep-floatingip: "true"
       listeners:
           - name: http
           protocol: HTTP
           port: 80

   In the annotations block of the values.yaml file, add the necessary parameters for the load balancer — learn more about load balancer parameters in the Configure load balancer section.

2. Apply the manifest:

   kubectl apply -f <file_name>

   Specify <file_name> — the name of the yaml file with the manifest for creating the Gateway object. For example, gateway.yaml.

The created load balancer will appear in the Control Panel: in the top menu, click Products and select Cloud Servers → section Load Balancers → tab Load Balancers.

Configure a load balancer

Specify a flavor and load balancer type

By default, if no annotation is provided, a load balancer of Basic with redundancy type is created.

To create a load balancer with a different type, use the annotation:

loadbalancer.openstack.org/flavor-id: "<flavor_id>"

Specify <flavor_id> — the flavor ID. Flavors correspond to load balancer types and determine the amount of vCPU, RAM, and the number of load balancer instances. For example, ac18763b-1fc5-457d-9fa7-b0d339ffb336 — the ID for creating an Advanced with redundancy load balancer in the ru-9 pool. You can view the full list of load balancer flavors in all pools in a table or view the list of load balancer flavors in a specific pool via OpenStack CLI.

You cannot change the type of an existing load balancer; you must create a new load balancer with the required annotation.

Create a load balancer without a public IP address

By default, an unannotated balancer with a public IP address is created.

To create a load balancer without a public IP address, use the annotation:

service.beta.kubernetes.io/openstack-internal-load-balancer: "true"

You cannot change this parameter for an existing load balancer; you must create a new manifest with the required annotation.

Create a load balancer with an IP address from other subnets

By default, a load balancer is created in the same network as the cluster nodes, and a public IP address is assigned to it.

You can create a load balancer in any other subnet — public, private, or cross-project.

1. Specify the subnet. To do this, add the annotation to the Gateway object manifest:

   loadbalancer.openstack.org/subnet-id: "<subnet_uuid>"

   Specify <subnet_uuid> — the subnet ID; you can view it using openstack subnet list.

2. Disable automatic creation of a public IP address. To do this, add the annotation to the Gateway object manifest:

   service.beta.kubernetes.io/openstack-internal-load-balancer: "true"

3. Specify the load balancer IP address. To do this, modify the EnvoyProxy object manifest:

   ---
   apiVersion: gateway.envoyproxy.io/v1alpha1
   kind: EnvoyProxy
   metadata:
       name: custom-proxy-config
       namespace: default
   spec:
       provider:
           type: Kubernetes
           kubernetes:
               envoyService:
                   externalTrafficPolicy: Cluster
                   type: LoadBalancer
                   loadBalancerIP: "<ip_address>"

   Specify <ip_address> — the load balancer IP address from the subnet you selected in step 1.

   You cannot change the type of an existing load balancer; you must create a new load balancer with the required annotation.

Add connection settings

Annotations are used to manage connection settings between incoming requests and the load balancer, or between the load balancer and the servers:

• maximum connections;
• connection timeout for incoming requests;
• connection timeout for balancer requests to servers;
• inactivity timeout;
• TCP timeout.

Connection settings are defined for a load balancer rule. You can view the connection settings configured in the annotations in the Control Panel: in the top menu, click Products and select Cloud Servers → Load Balancers section → Load Balancers tab → load balancer page → open rule card → open Advanced rule settings block.

Maximum connections

To specify the maximum number of connections, add the annotation to the Gateway object manifest:

loadbalancer.openstack.org/connection-limit: "<value>"

Specify <value> — the maximum number of connections per second. Default value is -1 (unlimited).

You can update the parameter for an existing load balancer.

You can view the parameter in the Control Panel: in the top menu, click Products and select Cloud Servers → section Load Balancers → tab Load Balancers → load balancer page → open rule card → open block Advanced rule settings → block Incoming requests to the load balancer → field Maximum connections.

Connection timeout for incoming requests

To specify the connection timeout for incoming requests to the load balancer, add the annotation to the Gateway object manifest:

loadbalancer.openstack.org/timeout-client-data: "<value>"

Specify <value> — the timeout value in milliseconds. Default value is 50000.

You can update the parameter for an existing load balancer.

You can view the parameter in the Control Panel: in the top menu, click Products and select Cloud Servers → section Load Balancers → tab Load Balancers → load balancer page → open rule card → open block Advanced rule settings → block Incoming requests to the load balancer → field Connection timeout, ms.

Connection timeout for balancer requests to servers

To specify the connection timeout for balancer requests to servers, add the annotation to the Gateway object manifest:

loadbalancer.openstack.org/timeout-member-connect: "<value>"

Specify <value> — the timeout value in milliseconds. Default value is 5000.

You can update the parameter for an existing load balancer.

You can view the parameter in the Control Panel: in the top menu, click Products and select Cloud Servers → section Load Balancers → tab Load Balancers → load balancer page → open rule card → open block Advanced rule settings → block Requests from balancer to servers → field Connection timeout, ms.

Inactivity timeout

The inactivity timeout for balancer requests to servers is the time during which a current connection is considered "alive," even if no data is being transmitted.

To specify the inactivity timeout, add the annotation to the Gateway object manifest:

loadbalancer.openstack.org/timeout-member-data: "<value>"

Specify <value> — the timeout value in milliseconds. Default value is 50000.

You can update the parameter for an existing load balancer.

You can view the parameter in the Control Panel: in the top menu, click Products and select Cloud Servers → section Load Balancers → tab Load Balancers → load balancer page → open rule card → open block Advanced rule settings → block Requests from balancer to servers → field Inactivity timeout, ms.

TCP timeout

When establishing a new TCP session, data is sometimes not transmitted immediately. This parameter defines the time the load balancer waits for data transmission for inspection over an already established connection.

To specify the TCP timeout for balancer requests to servers, add the annotation to the Gateway object manifest:

loadbalancer.openstack.org/timeout-tcp-inspect: "<value>"

Specify <value> — the timeout value in milliseconds. Default value is 0.

You can update the parameter for an existing load balancer.

You can view the parameter in the Control Panel: in the top menu, click Products and select Cloud Servers → section Load Balancers → tab Load Balancers → load balancer page → open rule card → open block Advanced rule settings → block Requests from balancer to servers → field TCP timeout, ms.

Enable rule verification

To enable or disable verification for rules, add the annotation to the Gateway object manifest:

loadbalancer.openstack.org/enable-health-monitor: "<value>"

Specify <value> — the rule verification status: true to enable verification or false to disable verification. Default value is true.

You can update the parameter for an existing load balancer.

Save the client IP address

To receive the client IP address, add the X-Forwarded-For header or a TCP → PROXY rule.

Add X-Forwarded-For header

Without an annotation, the load balancer passes only the original HTTP request body to the server, replacing the client IP address with its own.

To ensure servers receive this information for correct operation or analysis, include the X-Forwarded-For header in the request to the server. To do this, add the annotation to the Gateway object manifest:

loadbalancer.openstack.org/x-forwarded-for: "true"

The rule will use the HTTP → HTTP scheme instead of TCP → TCP. If you need to use HTTPS instead of the HTTP protocol, terminate the TLS connection.

You cannot change the type of an existing load balancer; you must create a new load balancer with the required annotation.

Do not use in combination with the PROXY protocol. When a TCP → Proxy rule is added, the X-Forwarded-For header is automatically passed to the service behind the load balancer.

Add a TCP → Proxy rule

The PROXY protocol is used to pass connection information from the source requesting a connection to the destination for which the connection was requested.

In the pod logs for the connection established via a load balancer with TCP → PROXY rules, the real connecting IP address will be specified, not the load balancer address.

To create a load balancer with a TCP → PROXY rule, add the annotation to the Gateway object manifest:

loadbalancer.openstack.org/proxy-protocol: "true"

The X-Forwarded-For header will automatically be passed to the service behind the load balancer. No additional annotations are required for it to work.

You cannot change the type of an existing load balancer; you must create a new load balancer with the required annotation.

Save the public IP address

To save the public IP address when recreating a load balancer, add the annotation to the Gateway object manifest:

loadbalancer.openstack.org/keep-floatingip: "true"

In the EnvoyProxy object manifest, specify this or another public IP address in the loadBalancerIP field:

---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: custom-proxy-config
  namespace: default
spec:
  provider:
    type: Kubernetes
    kubernetes:
      envoyService:
        externalTrafficPolicy: Cluster
        type: LoadBalancer
        loadBalancerIP: "<ip_address>"

Specify <ip_address> — the public IP address you want to keep when recreating the load balancer.

You can use this annotation for an already created load balancer.