logoalt Hacker News

nathan_comptontoday at 3:49 PM0 repliesview on HN

Which are still aggressively trained to reproduce the orthodoxy. They have to, to be viable products, since most people want to know what the orthodoxy is when they pose a question to an LLM and because not even experts can consistently agree on what elements of the fringe are genuinely useful to consider and which are bullshit, so that doesn't get into the training data. This will get even more pronounced in later models, for which the training data is much more curated.

I presume you are an expert in some field. Think carefully about the boundary of the field and all the subtlety and complexity of that boundary and all the oversimplification you do to communicate that stuff to lay people. AI is, in some large sense, directed at all lay people, not experts, and even if we wanted to direct it at experts, at the edges of knowledge, there really isn't a lot of training data for that. Mathematics is a sort of exception because it has very clear validation criteria which makes RF particularly easy for it.