alt Hacker NewsHonestly, this article has too many words which don't mean anything.
'AI Experts', 'superintelligence', and hand-waving doom scenarios.
As an example-
> Within a couple of years, possibly much sooner, AI may achieve so-called closed-loop recursive self-improvement (RSI): the capacity to rewrite its own code to become more capable, without human intervention. Should that happen, the result could be an intelligence explosion of a kind for which there is no precedent and no map.
I've heard these same objections in my lifetime about the internet. And I've read similar arguments against TV, radio, the phonograph, and the printing press.
Honestly, this is getting extremely tiring. Every new invention that has happened has affected the world in both good and bad ways.
Ultimately, what counts is what we do with it.
Replies
There’s a meme in finance circles that by the time some trend appears on the economist front cover it’s a top signal and time to get out of the trade and this is why.
It’s a very normie magazine written by and for normies, and by the time an idea has got that far down the easy money has already been made
Yeah, big lol on the Recursive Self-Improvement.
I mean, firstly you get to have an "Agent" actually capable of really long horizon tasks without getting stuck in tools loops and having its context rot. Secondly, each trial (ie a model fully trained to convergence) costs millions and takes O(weeks). You can probably run 1 or 2 of those experiments in parallel even at big AI labs as the hardware is scarce and they are costly as mentioned before. Assuming this agent needs something like a hundred tries to just show some improvement, we are looking at years.
And you can't early stop training for candidates that are not promising due to "emerging capabilities". At some point you might get a big drop in loss even if the model has been plateuing for a while. And you can't really scale down models for running trials quickly either also due to these emerging capabilities. In fact you might create a model that is great at converging with in small trials (fewer params, fewer tokens), but that is uncapable of developing those unexpected traits. And this will likely happen as you created a sort of evolutionary pressure in this direction: if you are good at learning in the first few epochs you "survive" and get to pass down your traits to future trials.
All of this to say that recursive-self-improvement is waaaaaay out of our grasp as things stand right now. We need another one or two breakthroughs to get there (IMHO).
The Economist reflects the views and biases of economic and political elites.
AI has turned into something of a religion for them because the substitution of technology for labor is a reflection of one of their deepest desires. Whatever the reality of the situation, they want to believe that AGI employees are imminent.
>Honestly, this is getting extremely tiring. Every new invention that has happened has affected the world in both good and bad ways.
Common human trope is to rely on analogies and past patterns to predict the future. It blinds you reality.
This isn't some new "invention". What's happening is fundamentally more profound. the stakes are much much much more higher and the consequences much more grave. Thus a historical analogy can't save you when the situation is categorically different.
I agree it's getting "tiring" but this is another common human trope. You see the same thing too many times and you lose the sense of danger that comes with it. It's like sky diving all the time. You eventually lose the fear via repetition, but fundamentally speaking, you are still parachuting out of a fucking plane.
> Ultimately, what counts is what we do with it.
Looking at the people in charge of these companies, and the sheer lack of understanding by the broad consumer market using them, I have nothing but pessimism about the direction that this technology takes us.