alt Hacker NewsLLMs are just really good search. Ask it to create something and it's searching within the pretrained weights. Ask it to find something and it's semantically searching within your codebase. Ask it to modify something and it will do both. Once you understand its just search, you can get really good results.
Replies
bhadass • today at 7:16 PM
better mental model: it's a lossy compression of human knowledge that can decompress and recombine in novel (sometimes useful, sometimes sloppy) ways.
classical search simply retrieves, llms can synthesize as well.
➕ show 4 replies
cultureulterior • today at 8:42 PM
This is not true.
johnisgood • today at 7:08 PM
Calling it "just search" is like calling a compiler "just string manipulation". Not false, but aggressively missing the point.
➕ show 3 replies
I agree somewhat, but more when it comes to its use of logic - it only gleans logic from human language which as we know is a fucking mess.
I've commented before on my belief that the majority of human activity is derivative. If you ask someone to think of a new kind of animal, alien or random object they will always base it off things that they have seen before. Truly original thoughts and things in this world are an absolute rarity and the majority of supposed original thought riffs on what we see others make, and those people look to nature and the natural world for inspiration.
We're very good at taking thing a and thing b and slapping them together and announcing we've made something new. Someone please reply with a wholly original concept. I had the same issue recently when trying to build a magic based physics system for a game I was thinking of prototyping.