logoalt Hacker News

input_shyesterday at 10:03 PM1 replyview on HN

Yes, we know, its purpose is to guide the bots, not forcibly block them.

That said, one of the biggest websites in the world not respecting it is definitely a noteworthy story. Hopefully another one of the biggest websites in the world (formerly known as Twitter) eventually respects it as well instead of not even disclosing itself via a user agent and pretending to be Safari running on iOS.


Replies

namegulfyesterday at 10:17 PM

Why down vote a comment?

You're talking about one (yes, biggest) but millions of other bots don't follow must be a bigger story.