logoalt Hacker News

fg137today at 1:17 PM1 replyview on HN

Glad to see that more and more people realize these are just snake oils. Without objective metrics like benchmarks, none of the claims mean anything.

That's also how I feel about skills/plugins. While some provide important context for specific projects/environments, I am very skeptical about (over)generalized skills like "writing JS tests" or "creating a spec". There are dozens of these skills internally at my company, but I haven't seen a single benchmark that shows any of those are better than just plain, single sentence prompts in a meaningful way (aka statistically significant).


Replies

jasonjmcgheetoday at 1:34 PM

I see the same thing and have effectively the same philosophy. If I'm using something like figma or glean or playwright/chrome dev tools, plugin/skill/mcp - likely very useful.

But so many of the weird collections of skills that people on YouTube get viral followings for - I just don't get it.

People excitedly ask me what skills I use and I feel bad just saying only things we've directly authored for some express purpose. None of the "hot" ones.

I've written a large handful of skills, but they aren't like vim plugins. I don't just leave them "on".

This has been my experience at least- curious if I'm just behind the times.

I also effectively didn't leave the IDE+ChatGPT copy/paste workflow until the first release of Claude code. So maybe I'm slow to adopt.