alt Hacker NewsThe paper's abstract itself says that behavior can be encrypted such that it remains opaque even with full access to model weights.
Replies
hackerman_fi • today at 5:36 AM
dataset != model weights
The paper's abstract itself says that behavior can be encrypted such that it remains opaque even with full access to model weights.
dataset != model weights
Right, but access to the training pipeline and intermediate checkpoints can allow us to catch the development of these structures in model weights before they become fully undetectable! It's a bit like monitoring agents communicating over noisy channels: it's interpretable for a while, but as soon as they come up with asymmetric key exchange, anything that looks to you like "noise" might actually be the agents scheming. So it's important to catch this _before_ it occurs!