196
Smaug-72B-v0.1: The New Open-Source LLM Roaring to the Top of the Leaderboard (huggingface.co)
submitted 2 years ago by to c/technology@lemmy.world
60 comments fedilink hide all child comments
 

Abacus.ai:

We recently released Smaug-72B-v0.1 which has taken first place on the Open LLM Leaderboard by HuggingFace. It is the first open-source model to have an average score more than 80.

you are viewing a single comment's thread
view the rest of the comments
[–] 10 points 2 years ago (1 child)

Unless you’re getting used datacenter grade hardware for next to free, I doubt this. You need 130 gb of VRAM on your GPUs

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 6 points 2 years ago (1 child)

    So can I run it on my Radeon RX 5700? I overclocked it some and am running it as a 5700 XT, if that helps.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 2 years ago

    To run this model locally at gpt4 writing speed you need at least 2 x 3090 or 2 x 7900xtx. VRAM is the limiting factor in 99% of cases for interference. You could try a smaller model like mistral-instruct or SOLAR with your hardware though.

  • permalink
  • fedilink
  • source
  • parent