logoalt Hacker News

xyzsparetimexyztoday at 5:52 AM1 replyview on HN

This is just json right? One issue with llms right now it's that if you give them a json specification, they're not always amazing at following it. I think it makes sense to have an agent tool that takes llm json, a file name and a spec path and only writes out the file if it conforms.


Replies

mirashiitoday at 6:47 AM

> give them a json specification, they're not always amazing at following it

Used correctly, this hasn't been true for a quite a while. Most inference engines have a form of grammar constrained decoding. See, for example: https://vllm.ai/blog/2025-01-14-struct-decode-intro

show 1 reply