logoalt Hacker News

jerftoday at 5:14 PM0 repliesview on HN

"unless you control the stack down to the hardware, it's basically a one way ticket to eternal slavery"

And you can't control the stack down to the hardware. Even if you are rich enough to fund a group of people to start from sand and end up at a simulation platform for you, you are also rich enough to have attracted enough attention that either one of those people could corrupt the platform, or someone may attack the organization to corrupt it with nation-state level resources.

No matter how you secure the computation, you can still corrupt the input and output streams. And as a $RICH_DYING_GUY it doesn't even matter if someone produces a proof of some software program somewhere's safety, because you have no way of knowing that it corresponds to the software that is running on the hardware.

Before you go with "yeah but I could...", I agree that a single individual could theoretically develop the skills that might permit validation of any particular small portion of the process. What I'm saying is no human can develop the skills to be able to validate the entire stack from top to bottom, hardware and software, in way that it couldn't be corrupted, especially in a world of more and vastly smarter AI agents. The probability that you will truly own and have full sovereignty over your computational substrate is zero.

(Whether I have full "sovereignty" over my current computational substrate would be a definitional matter, but I'm fairly confident that at least nobody else has the ability to just reach in and start mucking with neural weights directly or something.)