▲ 31 ▼ Best model for programming? (lemmy.nz) submitted 1 year ago by absGeekNZ@lemmy.nz to c/localllama@sh.itjust.works 20 comments fedilink hide all child comments I upgraded my PC, I have just upped the RAM to 64GB and got a Radeon 9700xt 16GB GPU. I want to do some personal programming projects and was wondering what is the best model that would fit my HW? I mostly use python, but dabble a little in go, and am interested in looking at rust. I am running this on my desktop, the processor is a AMD 3700X (8c/16t).
[–] JamonBear@sh.itjust.works 3 points 1 year ago (1 child) Is there a different recommendation for autocomplete? permalink fedilink source parent hideshow 2 child comments replies: [–] badcodecat@lemux.minnix.dev 3 points 1 year ago in general, you would want something fast (probably something that fits in your GPU/VRAM) so you can get suggestions as fast as you can type. for chat, you'll probably want the most intelligent/lorgest model you can run, it's likely fine if it's running on the CPU/RAM since the quality of an individual answer is more important than the speed in which many small answers can be generated. so, probably qwen for both, but, different sizes/quant for different use cases. permalink fedilink source parent
[–] badcodecat@lemux.minnix.dev 3 points 1 year ago in general, you would want something fast (probably something that fits in your GPU/VRAM) so you can get suggestions as fast as you can type. for chat, you'll probably want the most intelligent/lorgest model you can run, it's likely fine if it's running on the CPU/RAM since the quality of an individual answer is more important than the speed in which many small answers can be generated. so, probably qwen for both, but, different sizes/quant for different use cases. permalink fedilink source parent