684
OpenAI confirms that AI writing detectors don’t work (arstechnica.com)
submitted 2 years ago by to c/technology@lemmy.world
113 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 18 points 2 years ago (1 child)

The output is pure text so you would have to hide the signature in the response itself. On top of being useless since most users slightly modify the text after receiving it, it would probably have a negative effect on the quality. It's also insanely complicated to train that kind of behavior into an llm.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 2 years ago (2 children)

    Your implementation of my concept might be useless, but that doesn’t mean the concept is.

    One possible solution would be to look at how responses are structured, letter frequencies, etc. The flexibility/ambiguous nature natural language is that you can word things in many many different ways which allows for some creative meta techniques to accomplish a fingerprint.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 3 points 2 years ago

    It is a valid idea, and not impossible. When generating text, a language model gives a list of possible tokens.. or more correctly it gives a weight to every possible token where most would be 0 weight. Then there's multiple ways to pick the next token, from always picking top one to select random from top X tokens to mirostat and so on. You could probably do some extra weighting to embed a sort of signature. At some quality loss

  • permalink
  • fedilink
  • source
  • parent
  • [–] 2 points 2 years ago (1 child)

    The idea itself is valid, but wouldn’t that just make it more dangerous when malicious agents use the technology without fingerprinting?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 years ago (1 child)

    Cats out of the bag my friend. Just like the nuke, the ideas are always out there. Once it’s been discovered and shared that’s that.

    We can huff and puff and come up with all the cute little laws we want but the fact of the matter is we know the recipe now. All we can do is dive deeper into the technology to understand it even better, make new findings and adapt as we always do.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 years ago (1 child)

    Not sure if you’re disagreeing or agreeing with me. What I mean is, if a LLM’s output is in practice indistinguishable from human output, fingerprinting some popular services just creates a false sense of security, since we know malicious agents will for sure not fingerprint it.

    Isn’t it just better to let humanity accept that a LLM’s output is identical to a person’s and always be skeptical?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 years ago

    To be honest with you I’m torn on the subject.

    I don’t think it’s fair to abandon the idea that it’s possible to get a reliable fingerprint to differentiate between some hypothetical LLM/NLP AI and humans. I haven’t been convinced it’s impossible to tweak things purposefully to make them inherently produce a fingerprint every single time to help differentiate.

    I just think we need more time, so I guess I’m abstaining?

  • permalink
  • fedilink
  • source
  • parent