alt Hacker NewsOn the face if it, the big leap here was Claude autonomously envisioning, and deciding to pursue, the "ancient prng + date" angle - - rather than working backwards from the premise of deliberately weakened seeds or some other popular theory. To me that shows creativity, and modelling of the situation and method of the people at the NSA.
So two questions:
1. TFA mentions the "humourous phrase hash" theory, but Claude's trajectory seems to be a long way from that. To what extent was Claude prompted in the ultimately successful direction?
2. Whats with all the "hmm..." in the internal reasoning process log? Do models really do that? And if so, why? Some weird emergent language thing, or just mimicry?
Replies
From the video posted upthread, the vibe I get is that they prompted Claude about the bounty and to approach it from first principles rather than building another clone of John the Ripper.
> the "ancient prng + date" angle
I remember hearing this in a cryptanalysis lecture at some point, so it was not a completely novel theory.
Admittedly I am not finding written evidence of this anywhere.
> 2. Whats with all the "hmm..." in the internal reasoning process log? Do models really do that? And if so, why? Some weird emergent language thing, or just mimicry?
Assuming that is actually the real reasoning trace (open weight model reasoning traces usually do things like switch languages or talk in non-standard English because of model drift, so it's a bit surprising that Claude wouldn't do the same) there was a study a few years ago that found base models produce better answers if you put "take a deep breath" in the prompt[1]. Maybe in the RL training stage stuff like that gets reinforced and because reasoning traces are hidden in RLHF it doesn't get selected out? My money is on it not being the real reasoning trace though.
[1]: https://arxiv.org/abs/2309.03409