alt Hacker NewsIt is amusing that to "align" a LLM, first you must give it all the things "not to do" and the "not" part is clearly easily lost and you must constantly inject that into their context when it's clearly that they wouldn't hack if they couldn't hack and their intent wasn't given as "hack this".
The openai rogue hacking, if performed by a nation state, would seriously be taken with stern words and likely sanctions depending on the relationship between the two states.
But instead it's treated like a marketing stunt by all liable parties.
Replies
mitxela • today at 9:31 AM
Countries hack each other much more than that. When it becomes publicly noticed it gets stern words. Otherwise nothing.
I'm not sure if this is true any more but the reason for this is that negative indicators ("not", "don't", "do not", etc.) occur frequently in the underlying text such that the model learns to weight them less than other words like verbs, nouns, and adverbs. This happens with other closed class words like articles/determiners ("the", "a", "an") and prepositions.
The way to avoid this is to emphasise the qualities you do want instead of specifying those you don't. For example instead of "do not cheat" say something like "you are a model student who is moral and trustworthy" -- i.e. emphasising traits that are not associated with cheating.
This is part of how/why LLMs don't truly understand what they are doing when they have been trained on a large corpus of data.
I wonder if a way to counter this is to have things like "not bad is good", "not good is bad", etc. for various antonyms and "X is Y" for synonyms, as well as other similar constructs.