alt Hacker NewsBecause its generally expected that models only work 'in distribution', i.e. they work on stuff they have previously seen.
They almost certainly have never seen regular conversations in Base64 in their training set, so its weird that it 'just works'.
Does that make sense?
Replies
fweimer • yesterday at 9:01 PM
If you do not properly MIME-decode email, you end up with at least some base64-encoded conversations.
broDogNRG • yesterday at 7:06 PM
[dead]
For all we know, AI tech companies could theoretically have converted all of the "acquired" (ahem!) training set material into base64 and used it for training as well, just like you would encode say japanese romaji or hebrew written in the english alphabet.