So they made a prior tweak to reduce fetches of favicon.ico ( by link rel="icon" href="data:," ) but why do they still serve the overlarge meaningless html page to requests for robots.txt ?!

Given that they have gotten spiked by automated requests lately, aka agents, wouldn't this at least be respected by the big players?

Also, as they are using Cloudflare, wouldn't a discussion (or a note from IANA or CF) about how they configured CF request rejection and how one should love CF's flat rate static serving be appropriate?

Am I missing something?