alt Hacker NewsI have a theory that everything is search. Protein folding? Search for minimal energy. DNA evolution - search for ecological niche fit. Cognition? attention is search, memory is search, imagination is also search, problem solving - of course is search. Scientific progress? It is (re)search. Optimizing an AI model? Search for optimal parameters to fit the data. Reinforcement learning? Search for optimal behavior to maximize rewards. Even speaking is search - we output words in sequence, searching the next word like LLMs. Now I can add comedy to the list.
<rant>Search is a nice concept, it defines everything clearly - search space, goal space, action space. Compare it with fuzzy concepts like understanding, intelligence and consciousness. We can never define them, precisely because they gloss over their input-output domains and try to present a distributed process as centralized in the brain.
Search has a bunch of properties - it is compositional, hierarchical, recurrent (iterative in time) and recursive. This pattern holds across many fields, I think it is based on the fundamental properties of space-time which are also compositional, hierarchical and recurrent (object state at time t+1 depends on its state at time t)
Search can be personal, inter-personal, physical or information based. It can explain away much of the mystery of the three fuzzy concepts I mentioned. I describe cognition as two search loops - search externally by applying known behavior to collect experience, and search internally to compress experience and update behavior.</>
Replies
Lately I’ve come to think of science as a large-scale search algorithm so I think there’s truth to what you’re saying and it’s interesting to think about.
Strong agree. I'm on a similar path as you, travelling through related thoughts
I think this hypothesis goes a long way to explaining why the math of transformers (doing mathematical operations on language) create something that rhymes so much with intelligent thought. Though I should clarify that LLMs do not share the same processes or verbs of our intelligence, only the snapshot moment-in-time of a mind-like object ;)
I think another way to put that really good idea is simply to say that humans are innately wired to explore and discover!
I think a more specific denominator you might wanna look into is free energy minimization as you mention in your first example. I really liked reading Active Inference and What Is Life? on the subject.
You're spot on that all problems can be interpreted as a search problem. Similarly, all problems can be interpreted as a compression problem. Or parsing, boolean satisfiability, or halting, etc. It's helpful to keep them all in mind because sometimes a different problem domain has a tool that your preferred one doesn't, or just the mindset shift can be useful to unblock.