logoalt Hacker News

nekusarlast Sunday at 12:50 PM2 repliesview on HN

Makes sense.

LLMs only have 'knowledge' that was encoded by scanning as much data as possible. (Same way a search engine only can find data that was indexed)

American LLMs are by and large closed. Chinese are open to a point. And research how these things work is still a big mystery.

So yeah, was comprehensive data about saving tokens scanned and indexed? Likely no. So engaging about token saving is going to generate a lot of verbose and useless slop.


Replies

bkotryslast Sunday at 12:56 PM

Yeah, that was pretty much my experience. The models weren’t lacking knowledge as much as discipline. Without a good workflow, they will most likely spend thousands of tokens exploring dead ends.

show 2 replies
jscottbeelast Sunday at 2:25 PM

What most AI model users don't understand is (most here will know) that they are probabilistic (statistically based) and not deterministic (static/fixed). So answers to the same prompts can vary (wildly sometimes)

show 1 reply