logoalt Hacker News

crustycodertoday at 12:20 PM1 replyview on HN

Is that based on your "deep understanding" of how LLMs work or have you actually tried it? If you watch the execution trace of a Skill in action, you can see that it's doing exactly this inspection when the skill runs - how could it possibly work any other way?

Skills are just textual instructions, LLMs are perfectly capable of spotting inconsistencies, gaps and contradictions in them. Is that sufficient to create a good skill? No, of course not, you need to actually test them. To use an analogy, asking a LLM to critique a skill is like running lint on C code first to pick up egregious problems, running testcases is vital.


Replies

hansmayertoday at 1:10 PM

> you can see that it's doing exactly this inspection when the skill runs

I mean how do you know what does it exactly do? Because of the text it outputs?

show 1 reply