190
Developer Creates Infinite Maze That Traps AI Training Bots (www.404media.co)
submitted 2 years ago by to c/technology@lemmy.world
19 comments fedilink hide all child comments
 

A pseudonymous coder has created and released an open source “tar pit” to indefinitely trap AI training web crawlers in an infinitely, randomly-generating series of pages to waste their time and computing power. The program, called Nepenthes after the genus of carnivorous pitcher plants which trap and consume their prey, can be deployed by webpage owners to protect their own content from being scraped or can be deployed “offensively” as a honeypot trap to waste AI companies’ resources.

Registration bypass: https://archive.is/3tEl0

you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 2 years ago (1 child)

You can specifically target crawlers that ignore robots.txt, which will catch practically every LLM scraper.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 years ago

    Well, yeah, but obeying robots.txt is only a courtesy in the first place, so you can't guarantee it'll catch only LLM-related crawlers and no others, although it may lower the false positive rate.

  • permalink
  • fedilink
  • source
  • parent