logoalt Hacker News

nerdralph • yesterday at 10:19 PM • 0 replies • view on HN

Some useful information, though I'd like more details on headers sent like Sec-Fetch.

For passive blocking, TLSv1.3 filters out some bad bots and scrapers.

Most bad bots follow 302 redirect, but don't parse html and act on meta refresh.

Most bad bots don't store cookies, but I've seen a couple that do.