logoalt Hacker News

dsigntoday at 6:35 PM0 repliesview on HN

Up to five years ago[^1], we lived an age of pure speculation about intelligence and minds. That age brought about multiple theories of the mind, and to me, their finest legacy are arguments of the form "$X is not $concept because $theory", where any given day $X can be "LLM" or "LLM+harness+tool", "agent" and $concept can be "intelligent", "aware", "rational", "long-termed", , etc. Those arguments, in the context of $theory, are not wrong, and they are in fact quite useful. But there's only so much utility in a sieve.

We are going to need positive theories of the mind, and we will have to build them using terms and bits that avoid confrontation with the old guard. So, maybe instead of "mind" we will say "artilect"[^2][^3], and then move on to engineer desirable properties. For some of those properties ("alignment", "paucity", Kantian analysis and synthesis) we are already there or aiming for, but as we design to take on more and more challenging deeds, we may need to go back, steal and rebrand other old ideas. For example, "long term goals" and "self-preservation" are strongly linked in humans. Can we reify that relationship in the new artilects we make?

Other things we will need to make from whole cloth. For example, minds with an intuitive understanding of, and capability to manipulate: quantum mechanics, biological organisms, the workings of an ecosystem, the stock market, a stock company, or itself and other minds.

[^1]: YMMV, depending on where you draw the line between fine intelligence and stochastic parrots.

[^2]: Not the dictionary meaning, but the term as used by Alastair Reynolds in "Blue Remembered Earth (2012)"

[^3]: We may not use the term "LLM", because of the stochastic parrot crowd, because LLMs become one of many ingredients, or simply (but unlikely) because we find a completely different tool for the kit and discard LLMs entirely.