alt Hacker NewsRight but my point was that "given classes of templated in-context examples" is either
a) a game of roulette where you hope the LLM provider has RLHFed something very close to your use case, or
b) trying to few-shot it with in-context examples requires more engineering (and is still less reliable) than simply doing it yourself
In particular it's not just "the lack of a succinct, exhaustive text description," it also a lack of English->Prolog "translations."
It seems like the LLM-Prolog community is well aware of all this (https://swi-prolog.discourse.group/t/llm-and-prolog-a-marria...) but I don't see anything in Universalis that solves the problem. Instead it's just magically invoking the LLM.
That link you're citing is old news and also contained/discussed in the Quantum Prolog article. Those observations were made with respect to translating problem descriptions into PDDL, a frame-based, LISPish specification language for AI competitions encoding planning "domains" in a tight a priori taxonomical framework rather than logic or any other Turing-complete language. As such, contrary to what's speculated in that link, the expectation is that results do not carry over to the case of Prolog, which is much more expressive. I actually considered customizing an LLM using an English/Prolog corpus, which should be relatively straightforward given Prolog's NLP roots, but the in-context techniques turned out so impressive already (using 2025 SoTA open weight models) that indeed the bottleneck was the lack of text descriptions for really challenging real-world problems, as mentioned in the article. The reason may lie in the fact that English-to-Prolog mapping examples and/or English Prolog code documentation examples are sufficiently common in the latent space/in foundation training data.
I can ensure you Prolog prompting for at least the class of robotic planning problems (and similar discrete problems, plus potentially more advanced classes such as scheduling and financial/investment allocation planning requiring objective function optimization) works well, and you can easily check it out yourself with the prompting guide or even online if you have a capable endpoint you're willing to enter [1].
[1]: https://quantumprolog.sgml.net/llm-demo/part2.html