logoalt Hacker News

pessimizeryesterday at 7:59 PM1 replyview on HN

> Why are you asking a token generator to explain its prior output?

I swear I'm not. I'm trying to get it to fix the bug. I know it's a stateless slop generator, but I need it to be an obedient stateless slop generator.

The "magic words" I'm trying to come up with are whatever will prompt it to see the bug at all. I've tried standing instructions demanding that it simply not ever question me about whether a bug I've mentioned exists, because I'd rather it "fix" a bug that doesn't exist (so it can fail fast and I can realize I'm the dumb one) than fall into this loop of trying to argue it into doing what I say.

edit: that tactic does not work, even with much repetition, all caps, and many exclamation points. Eventually the instructions read like I'm having a mental breakdown.


Replies

sjsdaiuasgdiayesterday at 8:34 PM

You still seem to be expecting some degree of thought and understanding from these tools.

They generate tokens. The output has a probabilistic relationship to the established context and prompts, plus whatever prompting is happening as you interact with the model.

There is no understanding of "don't do [thing]". Sometimes, you can get something closer to what you wanted by putting stuff like that in the prompt. But it's still probabilistic token generation. It's not interpreting that as a literal command to not do the thing. It has that command in its context now, and maybe that changes the output. Maybe it changes in a useful direction, maybe it doesn't. But it's not going to be treated as a literal command because the model does not have the capability to do so. Phrasing it differently doesn't change the fundamentals.