141
Do you host your own AI? (aussie.zone)
submitted 1 month ago by to c/selfhosted@lemmy.world
202 comments fedilink hide all child comments
 

Do you host your own ML / AI / LLM? What do you use, and what do you use it for?

you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 1 month ago*

I use LMStudio, because it has quality of life improvements like nice GUI and huggingface search engine. Also they have Vulkan backend that at least on 7900XTX is ~10% faster than rocm (on LLama 3 8b Q4_0 it gets 115Tokens/s vs 105 on rocm)

  • permalink
  • fedilink
  • source
  • parent