alt Hacker NewsI can definitively say llms.txt is not used by any AI players. I run a blogging platform with around 80k blogs and /llms.txt is not requested by anything (other than humans checking to see if there's an llms.txt path).
All regular pages are aggressively scraped to the extent it's a problem I have to consistently manage, but not llms.txt.
Replies
isaachinman • today at 11:29 AM
How is a static blog being scraped a problem? Do you not use a CDN?
➕ show 2 replies
sunshine-o • today at 12:17 PM
Amazing, I didn't know.
So it get even stranger, I am the only one reading those /llms.txt ...
0123456789ABCDE • today at 12:05 PM
> I can definitively say llms.txt is not used by any AI players.
https://developers.openai.com/llms.txt
https://docs.anthropic.com/llms.txt
https://geminicli.com/llms.txt
https://github.com/llms.txt
https://docs.aws.amazon.com/llms.txt
https://openrouter.ai/docs/llms.txt➕ show 1 reply
I'm seeing quite a bit of request for these on my work's GitBook documentation site.
But perhaps these are developers specifically targeting these pages to feed whatever LLM they are using.