logoalt Hacker News

Venn1today at 3:23 PM1 replyview on HN

I'm blocking the Amazon search crawler, anything coming from Googleusercontent, and limiting AI crawlers to search rather than allowing AI assistants. The residential proxy waves are something to behold, but Cloudflare does an okay job catching those in the AI labyrinth. Still, it's all a bit silly, and I can't imagine what large sites deal with when I'm tangoing with this much nonsense on a small tech blog.


Replies

inigyoutoday at 3:30 PM

Don't. Just serve the page unless it's an unusually expensive page to serve (like search).