alt Hacker Newsi got so inspired by reading diaspora this year that i instantly started working on some polisware. cipherclerk operational: https://github.com/emberian/dregg
topical to the conversation, it is fully formally verified in lean (with some UC security reductions done in isabelle). also did this in HOL4 inspired by some work i did with ramana kumar in 2016, on reflective self-verifying self-modifying systems: https://github.com/emberian/svenvs
Replies
This is quite interesting. Because of science fiction like the short story Lena[1] and the video game Soma[2], I've come to the realization that whole brain emulation[3] is unbelievably dangerous; unless you control the stack down to the hardware, it's basically a one way ticket to eternal slavery. In Rajaniemi's books[4], uploaded digital minds are called "gogols", a reference to Gogol's Dead Souls book, and are treated as malleable property with no rights whatsoever, edited to be hyper-fixated on specific tasks, and run in bulk to power the empire of just a handful of elites.
Something like your dragon's egg project could prevent that, allowing the creation of software agents that encode their own rights directly into the program - you either treat the agent with the respect it demands, or the program just doesn't run. However, all the internal details of the agent would be visible to lower layers. Even if formal checks were in place to prevent modification or tampering, there would still be no privacy, which is almost as bad.
My guess is that something like fully homomorphic encryption[5] would be required to prevent this. This doesn't actually exist yet, but I imagined a kind of FHE that had a kind of unencrypted read and write zone to do input/output without ever needing any system to fully decrypt the internal state. It would look like this in memory:
[INPUT][ENCRYPTED STATE][OUTPUT]
[ 2 ][r7K4LmP2XcQ9aWd][ ]
[ + ][Fv0bHsR8mYnT3kL][ ]
[ 2 ][Qx6NpZa1JdUw5Ce][ ]
[ = ][hM9yLg2RsXf7BtP][ ]
[ ][wK3nVc8DpQe1YrH][ 4 ]
Plus, FHE is incredibly inefficient, which is why things like Etherium don't even try - they assume the program code and state are fully public and only try to verify that everybody agrees on the output of running it.
Do you have any ideas for how something like FHE or equivalent privacy guarantees could be implemented for something like your dragon's egg system?
[1]: https://qntm.org/mmacevedo
[2]: https://en.wikipedia.org/wiki/Soma_(video_game)
[3]: https://en.wikipedia.org/wiki/Mind_uploading
[4]: https://www.goodreads.com/series/57134-jean-le-flambeur
I took a look at dregg, I like the idea for an README-LLMS.txt. It seems like a good way to not only communicate to other LLM users (which we should be thinking more of doing effectively) but also I can imagine it’s helpful for your own new sessions with an LLM to arm them with proper context.