529
AI chatbots unable to accurately summarise news, BBC finds (www.bbc.com)
submitted 2 years ago by to c/technology@lemmy.world
132 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 2 years ago (1 child)

This is really a non-issue, as the LLM itself should have no problem at setting a reasonable value itself. User wants a summary? Obviously maximum factual. He wants gaming ideas? Etc.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 2 years ago*

    For local LLMs, this is an issue because it breaks your prompt cache and slows things down, without a specific tiny model to "categorize" text... which few have really worked on.

    I don't think the corporate APIs or UIs even do this. You are not wrong, but it's just not done for some reason.

    It could be that the trainers don't realize its an issue. For instance, "0.5-0.7" is the recommended range for Deepseek R1, but I find much lower or slightly higher is far better, depending on the category and other sampling parameters.

  • permalink
  • fedilink
  • source
  • parent