logoalt Hacker News

freejazztoday at 6:15 PM1 replyview on HN

oh, it was only due to a "one-line system prompt change"? well okay then! Here I was thinking it was multiple lines! How foolish was I??


Replies

losvedirtoday at 6:40 PM

I think "system prompt" is the key bit they're getting at. It doesn't necessarily reflect poorly on the underlying model if the system prompt was bad. It does reflect somewhat, in terms of alignment (how well the model does what the training company wants) and instruction following (how well the model does what the user wants). But it's not so clear to me what exactly the right answer is here. E.g., a model that scrupulously follows its system prompt and does what the user wants is a pretty useful, if very sharp, tool, albeit perhaps dangerous in the wrong hands.