Skip to main content

Cloud server with GitLab Runner

GitLab Runner is open source software used to automate and execute tasks (pipelines) in GitLab CI/CD.Works as an agent that sequentially executes the steps defined in the pipelines tasks.Allows you to run build, test and deploy applications automatically and manage the entire process of running CI/CD tasks in GitLab.

You can create a cloud server with a ready-made GitLab Runner application. The application runs inside the Docker container of the cloud server.In Russia, the server runs with a customized SelectOS 1.1 operating system. In other countries — Ubuntu 22.04.

If you need to run multiple runners in a single Docker container, increase the number of runners. To run multiple CI/CD tasks, enable parallel task execution.

Create a cloud server with GitLab Runner

In order for GitLab Runner to work together with a GitLab server from another network, the cloud server with GitLab Runner must be accessible from the Internet.To do this, you must create a private subnet and connect a public IP address when creating the server.To configure GitLab Runner, you must also specify user data — user configuration parameters of the operating system — when creating the server.

  1. Optional: create a public IP address.

  2. Create a cloud server with GitLab Runner.

1. Optional: create a public IP address

Create a public IP address so that the cloud server with GitLab Runner can be accessed from the internet.

Use the Create a Public IP Address subsection of the Public IP Addresses instruction.

2. Create a cloud server with GitLab Runner

  1. In the Dashboard, on the top menu, click Products and select Cloud Servers.

  2. Click Create Server.

  3. Fill in the blocks:

  4. Check the price of the cloud server.

  5. Click Create.

Name and location

  1. Enter the server name. This will be set as the host name in the operating system.

  2. Select the region and pool segment in which the server will be created.The list of available server configurations and resource costs depends on the pool segment.Once the server is created, you cannot change the pool segment.

Source

  1. Open the Applications tab.

  2. Select Cloud GitLab Runner.

  3. Optional: if you need another current or archived version of the application, select the desired version in the Version field.

Configuration

Select a configuration from 2 vCPU, RAM от 2 ГБ и размером загрузочного диска от 8 ГБ.Two types of configurations are available for the server:

The configurations use different processors depending on the line and pool segment.After the server is created, you can change the configuration.

  1. Click Fixed.

  2. Open the tab with the desired ruler.

  3. Select a configuration.

  4. If both local and network disks are available in the selected configuration, select the disk to be used as the boot disk:

    • local disk — check the box Local SSD NVMe disk. A server with a local disk can only be created from images and applications;
    • network drive — do not check the checkbox.

    The amount of RAM allocated to the server may be less than the amount specified in the configuration — the operating system kernel reserves some RAM depending on the kernel version and distribution. You can check the allocated amount on the server with sudo dmesg | grep Memory.

Disks

  1. If you did not check the Local SSD NVMe disk checkbox when setting up the configuration. the first specified network drive will be used as the server boot disk.To configure it:

    1.1 Select the type of network boot disk.

    1.2 Specify the size of the network boot disk in GB or TB. Observe the maximum size limits of network disks.

    1.3 If you selected the Universal v2 disk type, specify the total number of read and write operations in IOPS. After creating a disk, you can change the number of IOPS — decrease or increase.The number of IOPS changes is unlimited.

  2. Чтобы добавить дополнительный network disk server :

    2.1 Click Add.

    2.2 Select the type of network drive.

    2.3. Укажите размер сетевого диска в ГБ или ТБ. consider the maximum size limits of network drives.

    2.4 If you selected the Universal v2 disk type, specify the total number of read and write operations in IOPS. After creating a disk, you can change the number of IOPS — decrease or increase. There is no limit to the number of IOPS changes.

    After the server is created, you can connect new additional disks.

Network

The server can be added to a new private subnet or to an existing one.

Private — A subnet without access from the Internet or with a single static public IP address.The public IP address connects to a server that will be accessible from the Internet.

  1. Click Private Subnet.

  2. In the Public IP address for Internet access field, select the public IP address you created earlier or create a new one.

  3. Expand the block with private subnet settings.

  4. In the Subnet field, select an existing subnet.

  5. In the Private IP field, specify the private IP address of the server. The public IP address will be automatically connected to the private address.

Safety

Select security groups to filter traffic on server ports. Without security groups, traffic will not be allowed. If there is no block, port security is disabled on the server network . With traffic filtering disabled, all traffic will be allowed.

Access

  1. Place an SSH key for the project on the server for secure connection:

    1.1 If the SSH key for the project is not added to the cloud platform, click Add SSH Key, enter the key name, insert the public key in OpenSSH format, and click Add.

    1.2. If an SSH key for the project is added to the cloud platform. select the existing key in the SSH key field.SSH key is available only in the pool in which it is hosted.

  2. Optional: in the Password field for "root":

    2.1 Copy the password of the root user — a user with unlimited rights to all actions on the system.

    2.2 Save the password in a safe place and do not transmit it in public.

Additional settings

  1. If you plan to create multiple servers and want to improve the fault tolerance of your infrastructure, add the server to a placement group:

    1.1 To create a new group, in the Placement Group field, click New.

    1.2. Select New Group and enter a group name.

    1.3 Select a policy for hosting on different hosts:

    • preferably soft-anti-affinity. The system will try to place servers on different hosts. If there is no suitable host when creating a server, it will be created on the same host;
    • anti-affinity is mandatory. Servers in a group must be located on different hosts. If there is no suitable host when creating a server, the server will not be created.

    1.4 If a group has been created, select the placement group in the Placement Group field.

  2. To add additional information or filter servers in the list, add server tags. Operating system and configuration tags are automatically added. To add a new tag, enter a tag in the Tags field.

  3. To add a script that will be executed by the cloud-init agent when the operating system first starts up, in the Automation block in the User data field:

    • open the Text tab and paste the script with text;
    • or open the File tab and upload the file with the script.
#cloud-config

write_files:
- path: "/opt/gomplate/values/user-values.yaml"
  permissions: "0644"
  content: |
    gitlabURL: "<gitlab_server_url>"
    token: "<runner_token>"

Specify:

Increase the number of runners

  1. Connect to the cloud server.

  2. In the /opt/gomplate/templates/gitlab-runner-env.tpl file, add the URL:

    echo 'export RUNNER_GITLAB_URL_2="https://<gitlab_server_url>"' >> /opt/gomplate/templates/gitlab-runner-env.tpl

    Specify <gitlab_server_url> — The URL of the server with GitLab to which GitLab Runner connects.

  3. If the second runner is being added for a different GitLab server, add the runner token to the /opt/gomplate/templates/gitlab-runner-env.tpl file:

    echo 'export RUNNER_REGISTRATION_TOKEN_2="<runner_token>"' >> /opt/gomplate/templates/gitlab-runner-env.tpl

    Specify <runner_token> — the registration token for GitLab Runner. You can get it from the Create a project runner with a runner authentication token instruction in  the official GitLab documentation.

  4. In the /opt/gitlab-runner-pre-flight.sh script, add the registration of a new runner:

    • to the register_runner function:

      sed -i '/register_runner () {/,/^}/ {
      /^}/ i\
      echo "Register additional runner"\
      docker run -i --rm \\\
          -v /etc/gitlab-runner:/etc/gitlab-runner \\\
          gitlab/gitlab-runner:ubuntu-v17.5.4 register \\\
          --non-interactive \\\
          --url "\${RUNNER_GITLAB_URL_2}" \\\
          --token "\${RUNNER_REGISTRATION_TOKEN_2}" \\\
          --template-config /etc/gitlab-runner/config-template.toml \\\
          --executor "docker" \\\
          \${RUNNER_ADDITIONAL_PARAMS}
      }' /opt/gitlab-runner-pre-flight.sh

    • into the register_legacy_runner function:

      sed -i '/register_legacy_runner () {/,/^}/ {
      /^}/ i\
      echo "Register additional runner"\
      docker run -i --rm \\\
          -v /etc/gitlab-runner:/etc/gitlab-runner \\\
          gitlab/gitlab-runner:ubuntu-v17.5.4 register \\\
          --non-interactive \\\
          --url "\${RUNNER_GITLAB_URL_2}" \\\
          --registration-token "\${RUNNER_REGISTRATION_TOKEN_2}" \\\
          --template-config /etc/gitlab-runner/config-template.toml \\\
          --executor "docker" \\\
          \${RUNNER_ADDITIONAL_PARAMS}
      }' /opt/gitlab-runner-pre-flight.sh

  5. Reboot the service:

    sudo systemctl restart gitlab-runner

Enable parallel execution of CI/CD tasks

warning

Running CI/CD tasks in parallel can reduce the performance of the runner.

  1. Connect to the cloud server.

  2. In the /etc/gitlab-runner/config.toml file, specify the number of concurrent tasks in the concurrent parameter:

    sed -i '1 i\concurrent = 2' /etc/gitlab-runner/config.toml