you are viewing a single comment's thread
view the rest of the comments
[–] 3 points 2 years ago

It is a valid idea, and not impossible. When generating text, a language model gives a list of possible tokens.. or more correctly it gives a weight to every possible token where most would be 0 weight. Then there's multiple ways to pick the next token, from always picking top one to select random from top X tokens to mirostat and so on. You could probably do some extra weighting to embed a sort of signature. At some quality loss

  • source
  • parent