It definitely was a real skill; there was a time when you could get impressive results out of frontier LLMs if you knew the most advanced tricks for prompting them, but not if you just told them "hey solve this problem for me".

However, it's not a sustainable skill, because the labs care a lot about making "hey solve this problem for me" work well, and so put out models that are better and better at working with unsophisticated prompts over time.

There's an excellent piece about this, but it's unfortunately paywalled: https://www.theargumentmag.com/p/can-you-tinker-your-way-out...