logoalt Hacker News

Razenganyesterday at 11:07 AM1 replyview on HN

> You can write the "prompt tuning" down in AGENTS.md and then you only need to do it once.

Yeah, I just mean: I know how to "fix" the AI for things that I already know about.

But how would I know if it's wrong or right about the stuff I DON"T know?? I'd have to go Google shit anyway to verify it.

This is me asking ChatGPT 5 about ChatGPT 5: https://i.imgur.com/aT8C3qs.png

Asking about Nintendo Switch 2: https://i.imgur.com/OqmB9jG.png

Imagine if AI was somebody's first stop for asking about those things. They'd be led to believe they weren't out when they in fact were!


Replies

theshrike79yesterday at 2:23 PM

There's your problem right there.

Don't use it as a knowledge machine, use it as a tool.

Agentic LLMs are the ones that work. The ones that "use tools in a loop to achieve a goal"[0]. I just asked Claude to "add a release action that releases the project as a binary for every supported Go platform" to one of my Github projects. I can see it worked because the binaries appeared as a release. It didn't "hallucinate" anything nor was it a "stohastic parrot". It applied a well known pattern to a situation perfectly. (OK, it didn't use a build matrix, but that's jsut me nitpicking)

In your cases the LLM should've seen that you're asking about current events or news and used a tool that fetches information about it. Now it just defaulted to whatever built-in training data was in its context and failed spectacularly

AIs have a branding issue, because AI != AI which isn't AI. There are so many flavours that it's hard to figure out what people are talking about when they say "AI slop is crap" when I can see every day how "AI" makes my life easier by automating away the mundane crap.

[0] https://simonwillison.net/2025/Sep/18/agents/