> For alignment: it’s not actually a question of alignment at all, but rather one of “can we ignore alignment and design systems where the AI cannot cause harm?” Obviously, we can.
This does not seem at all obvious.
Any system worth using has to actually do something beyond the boundaries of that system. Even if that consists only of writing responses to prompts, the moment those are connected to any thing they can alter the state of that thing.
Making that safe means it only ever produces a safe response. Even ignoring malicious uses, it is unclear how to correctly classify what "safe" means on an unbounded time horizon.
I think you’d need to broaden your scope too far as to be useful for something like that. If we hook AI up to LHC, well, the immediate impact is obvious in that some physicists can’t use the detector. But it’s hard to ascribe “we neutron bombed the Milky Way in 42069 AD” to it finding a theory of quantum gravity.
We need to stop using the word "Alignment" because it clearly has different meanings for different people. Some people consider it to be preventing the model from waking up and enslaving or destroying humanity a la paperclip maximizer. Some people mean it as preventing the model from assisting with bad things like hacking, cybersecurity exploits, building weapons, and other things that are aligned with that individual user's goals, but not those of society more broadly.