15.435 -> 15.458 - Identifying and Fixing Yarn Lock Issue that causes rXg to stop running
April 15, 2024
Introduced in version 15.435 and patched in 15.459, this bug causes the yarn.lock file to grow exponentially with each start of the rxgd process. The file's size eventually causes rxgd to hang when starting.
To identify that your system is affected, here are some things to look out for:
To identify that your system is affected, here are some things to look out for:
1. Health notices that the backend daemon is not running, missing, or not hitting the cycle file
2. LAN interfaces are inaccessible after reboot
3. SSH session or console shows 'Welcome to FreeBSD!' login rather than RGNets rXg login after reboot
4. swp_pager_getswapspace: failed and swap_pager: out of swap space messages on console or after running dmesg
5. When rxgd starts, the log hangs with the last message typically stating that an installed Gem is looking for funding
To fix this, you will need to run the following commands from the console or SSH as root to obtain and install a patch:
2. LAN interfaces are inaccessible after reboot
3. SSH session or console shows 'Welcome to FreeBSD!' login rather than RGNets rXg login after reboot
4. swp_pager_getswapspace: failed and swap_pager: out of swap space messages on console or after running dmesg
5. When rxgd starts, the log hangs with the last message typically stating that an installed Gem is looking for funding
To fix this, you will need to run the following commands from the console or SSH as root to obtain and install a patch:
kr
This will attempt to stop the daemon. If the process cannot stop on its own, you may need to force it by noting the process ID it is attempting to stop, and running
kill -9 <PID of rxgd>
Next run the following three commands to patch the bug and restart the daemon
cd /space/rxg && curl https://build.rgnets.com/build/patches/2902.patch | git apply wget https://build.rgnets.com/build/patches/yarn.lock -O /space/rxg/console/yarn.lock rr
You will need to run these commands on all nodes. After successfully patching, it is highly recommended to upgrade your system to the latest official version.