alt Hacker NewsI'm one of the authors of this blog post. Figuring out how to integrate the LLM, the inference and serving engine, and the constrained decoding implementation such that you get the intended output and overall quality definitely requires careful engineering and testing. Please feel free to AMA. :)
Do you know where <||> syntax is coming from? I find it very odd that existing markup wasn't used and there is very little info online about it.