508
DeepSeek Permanently Reduces The Price Of Its Flagship V4 Model By 75 Percent (tech.yahoo.com)
submitted 2 months ago by to c/technology@lemmy.world
133 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 19 points 2 months ago (2 children)

They invented a hybrid attention design that drastically reduces the amount of memory needed for the KV cache at inference time. Like, dividing it by 10. And memory is a large part of the cost of inference.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] [S] 9 points 2 months ago

    This is a main part of the reason, yes. They actually innovated and did something that pushed the technology forward to be much more efficient, which we first saw with DeepSeek R1 for different reasons.

  • permalink
  • fedilink
  • source
  • parent