alt Hacker NewsYou don't even bother text after the [a] at first place in this case
Your question is something like
anwser only a,b,c,d for following question a. b. c. d....
the model output possibility of next character a: 0.8 b: 0.7 c: 0.3 f: 0.2 d: 0.1
If the list contains option you did not provide. The model is confused anyway, it don't matter if you use grammer to filter out the bad option or not, the answer is screwed already.
Yes, agreed. I was speaking in general, of course. This particular topic is of interest to me, so thinking of the edge cases and confounds vs Jev.
In your example, I would expect an LLM to do fine and if you have access to the raw logits you can measure whether or not it was confused and assign a confidence to the answer it gave.
I do think that Jev handles more than this though and, in my early testing, does things that are not easily accomplished with guided decoding techniques.