▲ 141 ▼ Do you host your own AI? (aussie.zone) submitted 1 month ago by SuspiciousCarrot78@aussie.zone to c/selfhosted@lemmy.world 202 comments fedilink hide all child comments Do you host your own ML / AI / LLM? What do you use, and what do you use it for?
[+] tristynalxander@mander.xyz 4 points 1 month ago* (last edited 3 weeks ago) (1 child) [deleted] permalink fedilink source parent hideshow 2 child comments replies: [–] BlackLaZoR@lemmy.world 2 points 1 month ago* I use LMStudio, because it has quality of life improvements like nice GUI and huggingface search engine. Also they have Vulkan backend that at least on 7900XTX is ~10% faster than rocm (on LLama 3 8b Q4_0 it gets 115Tokens/s vs 105 on rocm) permalink fedilink source parent
[–] BlackLaZoR@lemmy.world 2 points 1 month ago* I use LMStudio, because it has quality of life improvements like nice GUI and huggingface search engine. Also they have Vulkan backend that at least on 7900XTX is ~10% faster than rocm (on LLama 3 8b Q4_0 it gets 115Tokens/s vs 105 on rocm) permalink fedilink source parent