logoalt Hacker News

saagarjha • today at 12:47 PM • 6 replies • view on HN

I don’t necessarily agree with the labs’ policies and predictions on RSI but it’s worth noting that the points raised here are something they do have responses to.

On the idea that training better models requires more data, I think the current thought here is that we are not getting all the “meat off the bone” of the data we have, so there is still quite a ways to go with training that AI can improve. After all, it’s not like Anthropic is doubling their data set every year (?) and yet their models definitely improve by more than that.

For computability, yes, obviously the models cannot violate mathematics. But there’s a lot of things that are very computable and very desirable, or correspondingly very computable and very bad. This is enough to be worth thinking about.

For alignment: it’s not actually a question of alignment at all, but rather one of “can we ignore alignment and design systems where the AI cannot cause harm?” Obviously, we can. However the issue is that doing this takes time and very few people (especially the frontier labs!) seem to be doing so. It’s much easier to hook your LLM up to the world and ask it to go than ask, “is this safe?”


Replies

ben_w • today at 1:14 PM

> For alignment: it’s not actually a question of alignment at all, but rather one of “can we ignore alignment and design systems where the AI cannot cause harm?” Obviously, we can.

This does not seem at all obvious.

Any system worth using has to actually do something beyond the boundaries of that system. Even if that consists only of writing responses to prompts, the moment those are connected to any thing they can alter the state of that thing.

Making that safe means it only ever produces a safe response. Even ignoring malicious uses, it is unclear how to correctly classify what "safe" means on an unbounded time horizon.

➕ show 2 replies
BugDeveloper • today at 1:15 PM

About computability: All functions on finite sets are computable and superintelligence doesn't require a function on an infinite domain.

➕ show 1 reply
shishy • today at 1:35 PM

> but rather one of “can we ignore alignment and design systems where the AI cannot cause harm?” Obviously, we can

How? Many failures are emergent properties in complex systems -- you can capture some of them but this doesn't really seem obvious to me?

➕ show 1 reply
soco • today at 12:53 PM

Training a human brain needs and uses much, much, less data than training the AI models. So maybe the expectations on the AGI are, dunno, just optimistic? I mean you can put tons and tons of gas in a car but that won't make it drive faster.

➕ show 6 replies
majormajor • today at 1:11 PM

> For alignment: it’s not actually a question of alignment at all, but rather one of “can we ignore alignment and design systems where the AI cannot cause harm?” Obviously, we can. However the issue is that doing this takes time and very few people (especially the frontier labs!) seem to be doing so. It’s much easier to hook your LLM up to the world and ask it to go than ask, “is this safe?”

Much easier to pump up the numbers to investors to sell the quick-and-dirty solution. These companies are trying to speed-run things in unsafe ways that (just so coincidentally, I'm sure) happen to be exactly what the things you'd do if you were just in it for the money, and I wish we wouldn't be so accommodating.

All the "are we creating god" talk is similarly window-dressing nonsense marketing fluff. Gods throughout the history of mythology have been dumb as often as they've been superintelligent, the defining attribute of them is power not reasoning speed. If humans give massive rapid destructive power to a computer program it's a human error, not a god escaping us.

api • today at 12:57 PM

Having responses doesn’t mean anything. Are the responses valid and grounded in reality? Is there evidence for them?

A skilled debater can argue any arbitrary position convincingly.

I thought of a short LT;DR version of why I am not a Rationalist: they think arguments are evidence.

➕ show 3 replies