after months of ollama seeing only my CPU (maybe due to my Vulkan or ROCM setup) it finally occurred to me to try the docker image, and it's working perfectly on my GPU now! https://docs.ollama.com/docker

it could be permissions: my user is in render and video, but the docker container runs as root

you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 5 months ago

Thanks for mentioning ramalama, I was not aware of that one yet!

  • source
  • parent