logoalt Hacker News

Der_Einzigeyesterday at 10:32 PM1 replyview on HN

Models can't reliably follow instructions involving their own logprobs unless they can take agentic control and use quite sophisticated dynamic grammars/structures/constraints to force this behavior in one shot (which can be slow and the dynamic grammar modification feature isn't supported in closed model APIs for safety reasons) or repeated attempts at rewriting which is expensive/slow.

Yes they can do this, but it's more likely closer to the original "red token, green token" paper: https://arxiv.org/abs/2301.10226

i.e. take half of your LLMs vocabulary, and upweight its probabilities by ~55% to the other half's ~45%, and scan for overuse of this half of all tokens. You can even choose a different half/slice for every individual user, for every individual action. You can implement this under the hood cheaply with logit-biasing.


Replies

Terrettatoday at 11:48 AM

Considering how weirdly detuned tokens selections have become in Anthropic's LLM prose in recent models, there is a chance this goes unnoticed in everyday use.