141
Do you host your own AI? (aussie.zone)
submitted 1 month ago by to c/selfhosted@lemmy.world
202 comments fedilink hide all child comments
 

Do you host your own ML / AI / LLM? What do you use, and what do you use it for?

you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 1 month ago (1 child)

Curious about the quant tho.

Q8 from unsloth.

Something like Qwen3.5-122b

My go to model for knowledge. Definitely much faster at Q5 but it lacks the tool calling quality of the Qwen3.6 models. Really hoping we see a Qwen3.6-122b soon...

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 1 month ago

    In case you missed the Ornith 1.0 release (Qwen and Gemma RL finetunes for agentic / coding workloads), they look interesting to bridge the gap until we see larger 3.6 models or a 3.7 release. I didn't test them yet but according to benchmarks, the 35b MoE seems to be more or less on par with Qwen3.6 27b dense, while ofc a lot faster.

  • permalink
  • fedilink
  • source
  • parent