git.forsaken-ashbirds.net/docs/hosting-runners-rootless.md

Creating a new ForgeJo Runner Host

⚠️ Notice on IPv6-only hosts: Ubuntu 24.04 installs an outdated version of podman that is not properly capable of basic IPv6 in rootless mode.
Somewhere in the networking stack, somebody messes up the routing causing all IPv6 networks to be unreachable from inside the container. Podman maintainers/triagers seem to be keen on saying "this won't be an issue in Podman 5 with pasta anymore", however, that build is NOT available for Ubuntu 24.04 in its packet sources.

Building from source does work, however GitHub seems to be unable to provide their service on IPv6 for some unknown *** reason. The setup therefore requires IPv4 support (which can be provided by an HTTP(S)_PROXY).

Machine Setup

1. Install Ubuntu (24.04)

For example on the Hetzner cloud.

# As root, install updates.
apt-get update && apt-get upgrade -y
apt install -y systemd-container passt

2. Install podman (rootless) FROM SOURCE

(Hint: You can run all of the build stuff without root privileges, however apt install and make install will require sudo. 🙂)

# As root
# Ubuntu (>=23) thinks it is a good idea to disallow user namespaces for non-privileged users forcing us all to either use root or create apparmor profiles tailored for podman and its tools.
echo "kernel.apparmor_restrict_unprivileged_userns = 0" > "/etc/sysctl.d/99-rootless-podman.conf"
echo "kernel.unprivileged_userns_clone = 1" >> "/etc/sysctl.d/99-rootless-podman.conf"

# Create a non-root user to run all the build steps
# This is technically optional, but build scripts should not run with root privileges.
useradd -m -s /bin/bash podman-build
adduser podman-build sudo
passwd podman-build
machinectl shell podman-build@ # Switches shell to this user (including systemd container)
# You can use "sudo -k" to remove cached credentials after installing build dependencies

The podman binaries in the Ubuntu repositories for 24.04 are too outdated (4.X.X) for proper IPv6 support.
(According to the Podman devs, all of it is better with Pasta - which is the preferred rootless networking backend for Podman 5+)

2.1 Install crun from source

The crun version shipped by Ubuntu 24.04 is too old for Podman 5+. So we need to build it from source too...

git clone https://github.com/containers/crun
cd crun
git checkout "<latest-release-version>"
# <!-- Install dependencies for Ubuntu according to repository readme --!>
./autogen.sh
./configure
make
sudo make install # We need to change the prefix to overwrite what Ubuntu ships

2.2 Build podman from source

See their docs on how to do that.
(You can skip the 99-userns.conf command because we already did that above ⬆️)

⚠️ Please note the following ⚠️

• After cloning the podman repository, checkout the latest release tag (the docs forget to mention that!)
• Make sure to install libapparmor-dev libsystemd-dev before compiling so that the compatibility can be included in the installation.
  We recommend running make with BUILDTAGS="selinux seccomp apparmor exclude_graphdriver_devicemapper systemd"
• Run make vendor before running make install because for whatever reason it can be left out sometimes??

2.2 Containers configuration file

While the podman package installs the default configuration file below /usr/share/..., the make install command does not.
To make configuration easier for you later, please download the default configuration to /etc/containers/containers.conf.

# As root
wget -O /etc/containers/containers.conf https://raw.githubusercontent.com/containers/common/refs/heads/main/pkg/config/containers.conf
sed -i 's/.*#runtime =.*/runtime = "\/usr\/local\/bin\/crun"/' /etc/containers/containers.conf # This updates the crun version used by podman - it really tries to use the outdated one otherwise.

# Note - This should yield "pasta", even on Ubuntu 24.04, if things worked so far.
podman info | grep rootlessNetworkCmd
# Note - This should show a crun version ABOVE 1.17
podman info | grep crun

3. Create a new user for the runner

Since we don't want to the new forgejo runner to be root on out machine, we create a new user for it:

# As root
useradd -s /bin/bash --create-home forgejo-runner
loginctl enable-linger forgejo-runner

Enable Podman docker-socket on user

# As root
machinectl shell forgejo-runner@ # <-- This is basically "sudo -Hi XXX" but makes sure the systemd container is switched too.

systemctl --user enable --now podman.socket podman
echo 'export DOCKER_HOST=unix://$XDG_RUNTIME_DIR/podman/podman.sock' >> ~/.profile

4. Install the forgejo-runner

# As forgejo-runner
# Verify these URLs are still the version you want to install!!!
DOWNLOAD_URL="https://code.forgejo.org/forgejo/runner/releases/download/v5.0.4/forgejo-runner-5.0.4-linux-amd64"
SIG_URL="https://code.forgejo.org/forgejo/runner/releases/download/v5.0.4/forgejo-runner-5.0.4-linux-amd64.asc"
gpg --keyserver keys.openpgp.org --recv EB114F5E6C0DC2BCDD183550A4B61A2DC5923710 # Installs the signing key used by forgejo for their releases
wget -O forgejo-runner "$DOWNLOAD_URL"
wget -O forgejo-runner.asc "$SIG_URL"
gpg --verify forgejo-runner.asc forgejo-runner
# The output should now contain the following:
# Good signature from "Forgejo <contact@forgejo.org>"
#		aka "Forgejo Releases <release@forgejo.org>"

mkdir -p ~/.local/bin
mv ./forgejo-runner ~/.local/bin/forgejo-runner
chmod 750 ~/.local/bin/forgejo-runner
source .profile # Refreshes shell vars, because .profile was modified.

5. Configure and register the forgejo-runner

__The official runner registration is __ here Or, if you're volunteering another runner for our instance, please contact us to receive the necessary registration information. :)

# As forgejo-runner (recreate shell to update PATH)
forgejo-runner generate-config > config.yml
# You should open the config.yml and enable IPv6 support! (Your cloud host might charge extra for IPv4 connectivity)

# We're about to enter secrets into the terminal, disable history:
set +o history
RUNNER_INST_URL="https://git.forsaken-ashbirds.net"
RUNNER_NAME="<PLEASE ENTER A UNIQUE NAME FOR YOUR RUNNER HERE!!!>"
RUNNER_TOKEN="<The token from the UI>"
RUNNER_LABELS="ubuntu-24.04,docker,podman,self-hosted" # Update these labels if you intend to change stuff!

# Re-enable history :)
set -o history
forgejo-runner register --instance "$RUNNER_INST_URL" --name "$RUNNER_NAME" --token "$RUNNER_TOKEN" --labels "$RUNNER_LABELS" --no-interactive
# You should see the following afterwards:
# INFO Runner registered successfully

Check the runner is working

# As forgejo-runner
echo "XDG_RUNTIME_DIR=/run/user/$(id -u)" > .runner-env
echo "DOCKER_HOST=unix:///run/user/$(id -u)/podman/podman.sock" >> .runner-env

mkdir -p ~/.config/systemd/user
# Download the file "docs/forgejo-runner.service" from this repository to "~/.config/systemd/user"
# Or create a new file there and paste the contents.


systemctl --user enable --now forgejo-runner
# View logs by using:
journalctl -xe --user-unit=forgejo-runner
# View status by using:
systemctl --user status forgejo-runner

Go into Forgejo and check that your runner is shown as UP.