alt Hacker NewsThe modern state of the art is inherently not verifiable. Which way you give it input is really secondary to that fact. When you don't see weights or know anything else about the system, any idea of verifiability is an illusion.
Replies
hu3 • yesterday at 12:16 PM
At least MCPs can be unit tested.
With Skills however, you just selectively append more text to prompt and pray.
Sure. Verifiability is far-fetched. But say I want to produce a statistically significant evaluation result from this – essentially testing a piece of prose. How do I go about this, short of relying on a vague LLM-as-a-judge metric? What are the parameters?