alt Hacker NewsIt really isn't necessary when using constrained decoding (aka structured outputs) which guarantees that you'll get JSON output in the correct structure.
It really isn't necessary when using constrained decoding (aka structured outputs) which guarantees that you'll get JSON output in the correct structure.
This is not true at all. Just because you can force the logits to give syntactically valid outputs, doesn't mean you're going to get a useful result.
Constrained generation, without a proper understanding of the model's natural response tendencies, can give horrible results.