logoalt Hacker News

redox99today at 3:52 PM2 repliesview on HN

Search results suck because the web sucks these days. The big models from OpenAI/Anthropic have every book in existence baked into them


Replies

jamesharttoday at 4:23 PM

I don’t think that’s the right way to think about LLM ‘knowledge’. They don’t have absolute recall of everything in the training set. They have been trained so that they have weights that can predict what those books might say - that is, if they read them they would find the contents unsurprising. That doesn’t mean it wouldn’t be helpful to pull relevant passages of text directly into context for a particular task.

infp_arboristtoday at 6:55 PM

Does it really matter? What about including all relevant and up-to-date literature as skills for local models? I have no experience with this but I am pretty sure someone has already thought about it.