logoalt Hacker News

steve_adams_86today at 1:54 AM1 replyview on HN

> I'm surprised how strange that idea sounds to some people.

I think a lot of people using the models genuinely feel like the models are more capable than they are now, and they're content to relinquish a lot of trust and agency. The worrying thing is that the models are superficially hyper-capable, but from more granular perspectives, you can see a lot of holes in their abilities. This is incredibly important, but very difficult to convey concisely to people. It's a classic example of nuance seeming too complicated because not caring is so much more gratifying. People love using these models.


Replies

niyikizatoday at 3:57 AM

Yeah, people calibrate trust to the median behaviour of the model and get burned by the tail. What makes it harder is that even people who do see the holes often respond with better prompts and more elaborate context. Same trust-the-model move one level up. Hyperscalers aren't incentivized to fight that instinct either. Every "fix" routes more tokens through their meter.