you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 2 years ago

interesting project, thx for sharing! though:

There is not currently a way to differentiate between web crawlers that are indexing sites for search purposes, vs crawlers that are training AI models. ANY SITE THIS SOFTWARE IS APPLIED TO WILL LIKELY DISAPPEAR FROM ALL SEARCH RESULTS.

  • source
  • parent