you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 6 months ago

It goes down to number of vram / unified ram you have. There is no magic to make 8b perform like top tier subscription based LLMs (likely in 500b+ range, wouldn't be surprised if trillions).

If you can get to 32b / 80b models, that's where magic starts to happen.

  • source
  • parent