logoalt Hacker News

alecst • today at 2:11 PM • 1 reply • view on HN

I think this article gets it wrong, which is unfortunate:

> Vernor Vinge pointed out, rapidity in cognition is not the same as intelligence. Speeding up the mind of a dog, he argued, wouldn’t enable the animal to play chess.

True, he writes about this in his essay "On the Coming Technological Singularity." His point is that there's some hypothetical possibility that computers will get fast but not smart--i.e. a dog thinking a million times faster than a human is still a dog. But seeing as computers have solved Jacobi Conecture, quasi-Riemann, and Navier Stokes--and we're in the early stages of LLMs--I think this hypothesis is bunk. The machines are smart. Foolish in some ways, but still smart.

> Their construction prevents such an outcome: Instead of prompting an LLM to generate an open-ended strategy for pursuing a general goal, these AIs consider much more limited spaces of possible actions

Well this is what Yudkowsky has advocated for--models that are trained on limited subspaces, e.g. protein folding sequences and nothing else. So I don't think this is a counterpoint.

On a personal note: Has "alignment" even been defined in a consistent way? How can AI even be aligned if people are not aligned?

> Objection #1: Recursive Self-Improvement Is Speculative

I don't think it's speculative since it seems to already be happening. OpenAI wrote about how use of agents in research is climbing quickly, and the goalposts for intelligence move every day ("Image generation can't do hands", "video generation can't be consistent", "context length is too small", etc., I could go on.)

This article makes me feel worse. These are bad arguments.


Replies

keybored • today at 2:27 PM

Don’t feel bad. AI Doomers are the yin to the yang of AI Hypers. The HN machine needs both (for the sake of AI).