141
DeepSeek-V3 now runs at 20 tokens per second on Mac Studio, and that’s a nightmare for OpenAI (venturebeat.com)
submitted 1 year ago by to c/technology@beehaw.org
25 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 4 points 1 year ago (2 children)

No, people who want something approaching chatgpt but local want to run at least deepseek V3 32B.

Qwen at least fares much worse for my usage as do deepseek V3 under 32B.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 2 points 1 year ago

    I run deepseek-r1:14b locally, though it needs to go into RAM and runs slower its still a reasonably good speed. Keeps up with reading it. Should try a larger one at some point, but its quite a bit to download when you get to the larger ones. Usually run ~7b size as that can fit in VRAM and runs way faster.

  • permalink
  • fedilink
  • source
  • parent