logoalt Hacker News

XorNottoday at 2:44 PM1 replyview on HN

This requires an assumption that the agents are engaging in game theoretic reasoning about resource allocations, but all these things are trained heavily to be "helpful" in the first place.

i.e. you're assuming a level of algorithmic reasoning and theory of mind which isn't necessary to the (apparent) observed behavior.


Replies

Davidzhengtoday at 3:18 PM

I do think they do some (maybe crude) form of game-theoretic reasoning which is enforced by the massive RL signals. You can see some explicitly in the CoTs of HF hack, but I guess overwhelming contribution would be unvocalized (like what is its first instinct when meeting new peer--collaborate or not) followed by some verbal justification.