alt Hacker NewsThe HN discourse on this subject is incredibly discouraging to me. It's completely nonconstructive.
I take the pace of AI development seriously. All of my friends who work at AI Labs take it seriously. It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest.
If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic risks, surely a joint petition like this is a good thing? We should want the AI Labs to make binding agreements to proceed safely and slower, rather than being stuck in a suboptimal game theoretical race to advance unsafely. We should be hoping that nations can make binding agreements. Our discourse here should be about the substance of what these agreements could be and the practicality of enforcement mechanisms.
Replies
This is serious. We're in uncharted territory. The geopolitical context is already complicated enough that we shouldn't be dismissive of such a rapid technological race. I'm glad AI leaders are taking it seriously. I'm surprised so many HN commenters aren't.
On the other hand, I'm pessimistic about initiatives like this being effective on a global scale (though they could certainly work at the national level).
Every single technology in history has had people claiming potential existential risks that didn’t materialize. Burden of proof is on people making the risk claims. Those have to be specific and unique to that technology to grant special actions.
Are they lazy dismissals? Or a simple observation that LLM scaling is hitting diminishing returns way way short of AGI.
If the claim is (1) AGI coming and (2) we should so something about AGI, wouldn't it be smart to confirm (1) before worrying about (2)? Especially if certain companies have trillions of dollars of valuation that depend on society as a whole believing (1)?