A year of fighting scrapers on my 1.5 million-page website. What I tried, how I failed, and what's working now.

you are viewing a single comment's thread
view the rest of the comments
[–] 5 points 1 day ago

I thought Cloudflare's anti-AI defenses were meant to dynamically catch all of this. That's a lot of work to do to keep bot traffic away.

FWIW, I tried loading it with a VPN, it showed the content, but not all the images. Turned off VPN, it fully loaded. Maybe the blog has different rules.

  • source