615
AI giants learn what everyone else on the modern internet already knows (www.businessinsider.com)
submitted 2 weeks ago by to c/technology@lemmy.world
78 comments fedilink hide all child comments
 

For years, tech giants have argued that if information is available on the internet, it can be used for AI model development and outputs. They call it fair use. Content owners have tried to prevent this, with no success.

Now Anthropic, OpenAI, and Google are discovering what the rest of the internet has already learned through painful experience: once you put something online, people will find ways to use it in ways you don't like and can't stop.

The latest flashpoint is something called "distillation," using the outputs of one AI model to improve another. Anthropic says competitors are harvesting its outputs at scale, turning billions of dollars of research into a shortcut for rivals. OpenAI and Google have made similar warnings recently.

Remove Paywalls Link

you are viewing a single comment's thread
view the rest of the comments
[–] 25 points 1 week ago* (1 child)

At least the models like Qwen have open weight versions. It’s the same concept as distributing compiled binaries, though, whereas we really need true FOSS models where all of the code and training data are available under a permissive license. All of these models were trained on copyleft-licensed content, so all of it should be FOSS if the licenses were actually being respected. From that perspective, distillation attacks shouldn’t even be necessary and I couldn’t give two shits that there is no honor among thieves when the real thievery is that these models are closed source.

  • permalink
  • fedilink
  • source
  • hideshow 2 child comments
    •
  • [–] 8 points 1 week ago (1 child)

    So far, we have exactly one of those

    https://en.wikipedia.org/wiki/Apertus_(LLM)

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 5 points 1 week ago (1 child)

    Oh, nice! I haven’t been able to find one because searching for “open source llm”, etc., always yields open weight LLMs instead of open source ones.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 4 points 1 week ago* (last edited 1 week ago)

    Honestly, I think that's the only llm of its kind. The next closest thing is this

    https://github.com/karpathy/nanochat

    If you don't mind something narrower (an expert system, sort of), I'm slowly coding something for myself here. It's not ready yet (about half way done to initial release).

    https://codeberg.org/BobbyLLM/picoGURU

    Public facing docs are a bit of a mess at the moment (sorry; writing docs is boring AF; will probably be the last thing I tidy before official 1.0.0) and it's not really runnable right now, but when done, the "source files" for this will be kiwix (off line Wikipedia), your own markdown notes and 30 or so trusted domains (user defined).

    So it's all either your own content or public domain aka open source fully. Code itself is AGPL-3.

    Anyway, that's the goal.

  • permalink
  • fedilink
  • source
  • parent