I tried LM Studio (directly in Bazzite) and it has the same issue of not running through the GPU. It also always seem to end up stopping generating anything after a few moments when I use it with SillyTavern.
Tried the koboldcpp AUR package through Distrobox, and when I select the ROCm option it crashes with a CUDA error. lol
Using the Vulkan option it still seems to run through the CPU for some reason.
I tried LM Studio (directly in Bazzite) and it has the same issue of not running through the GPU. It also always seem to end up stopping generating anything after a few moments when I use it with SillyTavern.
Tried the koboldcpp AUR package through Distrobox, and when I select the ROCm option it crashes with a CUDA error. lol Using the Vulkan option it still seems to run through the CPU for some reason.