logoalt Hacker News

ipdashctoday at 4:17 PM1 replyview on HN

> it's not actually obvious who's running these bots

This is fascinating to me. It's a large enough phenomenon that it's affecting the entire Internet and yet nobody seems to know yet who's actually doing it. Which isn't surprising, of course, it's hard to trace back to a source through all these proxies and it's probably a bunch of distinct groups anyways, but still!

Personally I have to wonder how much of it is "scrapers for training data" vs just tool-use LLMs. Even if you use chatgpt in thinking mode you can clearly see it searching and visiting a bunch of different websites to answer a question, presumably faster than any human would. That's got to add up. It's got me wondering why everyone seemingly discounts that as an option


Replies

tmp10423288442today at 4:26 PM

ChatGPT uses a distinct bot type for ChatGPT-initiated web search versus model training:

GPTBot → model improvement / training crawl

OAI-SearchBot → search index / ChatGPT Search

ChatGPT-User → fetch initiated by a user's ChatGPT interaction

OAI-AdsBot → advertising landing-page crawl