alt Hacker NewsUnrelated to the thread of discussion, but thank you for posting it on your personal site, and not on Twitter/X.
I'd also be interested in how much your opinion on the existential threat of AI changes during your ten day backpacking trip, assuming it'll be a pretty offline experience.
Whilst I am involved in technology in my work and some of my hobbies, I do enough things 'away from the desk' that it feels to me that a lot of the doomerism is from people who have too much of their own personality and self-worth tangled up with the internet; the serially-online. I don't know where you naturally exist on that scale, but you already said you're outside the bubble and aren't the typical AI-rationalist-tech-bro (and surely the natural beauty of New Zealand is impossible to ignore).
I personally think that the old analogue world is big enough that some kind of AI-pocalypose wouldn't be anywhere near a civilisationally existential threat.
At risk of coming across an arrogant prick in telling you to do something: Consider your opinion now, and record it somehow. Then revisit it when you get back from your trip and see how far you've strayed from that as a baseline.
Thanks for the thoughtful comment. I go on this sort of trip pretty regularly so I don't expect this one to reorient me dramatically. They are good for my mental health though, for sure.
I think there are pretty clear mechanisms by which AI (a few model generations from now) could bring down civilization as we know it. One of them is a runaway cybersecurity scenario where hacking one set of systems quickly enables the AI to hack more systems (via supply chains, stolen credentials etc) and also host more instances of itself, and eventually everything Internet-connected that's not invulnerable just falls apart, which probably includes most electrical grids and a lot of other infrastructure. Yeah, the wilderness and countryside would still be there, but I for one can't live there without returning to the supermarket regularly for supplies. A lot of people would suddenly have to live (or not) without their meds. It would be bad.