alt Hacker NewsI similarly noticed Gemini absolutely refuses to look at a url when I give it one and will instead just hallucinate based on what it thinks the url is. Here I am assuming Google will have the best web capabilities in its AI.
Replies
I have one google account where gemini constantly confidently hallucinates crap, and another where it doesn't (at least to the extent that other modern LLMs don't).
I take this to mean that my one account has been mistaken for a competitor and they're trying to poison its data. But who knows.
Where ever would you get that impression from the company that made its fortune by being the best at searching the web?
Literally every experience I've had with Gemini / Google Search AI answers followed this exact pattern, often repeated several times more if I remained persistent instead of just giving up.
Typical example:
"Where can I buy <thing I'm looking for that I can't find anywhere using normal search terms>?"
> You're looking for <related but different and widely available thing>. It is sold by <sites I never heard of>.
"No, that's different. I'm looking for <that thing but with the exact differences spelled out again>."
> Ah, my mistake. You're looking for <thing I described>. It is sold by <sites I never heard of but which don't actually sell it>.
"I've checked your links and none of those sites actually sell it, one doesn't even sell products and instead only offers manufacturing - but also not for what I asked you for."
> I'm sorry, my bad. Those sites don't sell what you are looking for. Instead you should check out <more sites I've never heard of>.
"Those sites sell the thing you initially thought I was asking about but not the thing I described."
> I'm sorry for the misunderstanding. You can find the thing you actually described at <yet more sites including some of the same>.
"No. None of these sites sell anything close to what I asked you for and two of them don't actually exist."
> Oh, sorry about that. You're completely right. The thing you asked me about isn't actually being sold by anyone. However you could buy <thing it first thought I meant and that wouldn't bring me any closer to solving my problem>.
(ad nauseam)
Claude also has very arbitrary and confusing rules about what web pages it allows itself to look at, and how much of the page it can read. Did you know for example you’ll get a much deeper analysis if you download a PDF yourself and upload it to Claude instead of giving it the url?
This is a key reason why I actually like Grok for factual queries based on web grounding. It’s fast and reliable. Maybe it’s ignoring robots.txt? Dunno. But it works well.
I searched for something, it told me that according to a YouTube video, the entire point of my search was wrong. I asked it for the source, I watched the video, it never made the claims Gemini hallucinated. I asked again and it claimed it scrubbed the video and found the point it made multiple times. I said those timecodes were wrong and it admitted it couldn't actually parse videos and just guessed.
What the fuck?