alt Hacker NewsI have a searxng instances but stopped using it after doing a comparison vs actual search provider APIs.
searxng gets rate limited fast and the results are inconsistent depending on what providers it can reach at any given time / isn't captcha'd out.
Meanwhile there are multiple providers with generous free tiers so no point in feeding LLMs inferior data
Also did a couple of side by side tests asking LLMs directly which feed results it prefer for some sample technical searches. Exa and tavily #1, brave #2, searx last
> searxng gets rate limited fast and the results are inconsistent depending on what providers it can reach at any given time / isn't captcha'd out.
Exactly. I really wonder what the people going on about using SearXNG are actually doing, because in practice I've seen just what you see. Even from a residential IP, you might get a handful of Google searches before getting backed off. If you ever have multiple searches going on at once, even Bing and Brave are going to back you off pretty quick. My home IP is apparently permanently banned from using Brave Search through SearXNG.
And when you aren't getting results from Google/Bing/Brave, it gets bad really quick for your agent. As much as I wanted to go fully self-hosted for agent search, it just doesn't work well.