logoalt Hacker News

gvfsa • today at 8:09 AM • 2 replies • view on HN

Can someone explain why fetch costs money. I thought it was your locally-running harness doing it. How can they charge you for that?


Replies

BlackRabbit1 • today at 8:11 AM

Most of your local requests will fail (with Python/Curl/whatever) if you are not using a remote-controlled Firefox/Chrome. And even then you will be bitten brutally by captchas, rendering them basically unusable for mass requests. So it's better to outsource your web search to an external service with a stable API.

The existing search machines that offer free APIs for local harnesses are super bad with low quality.

This is worth the invest as LLMs can be guided closer to existing information/code to adapt it.

hobofan • today at 9:42 AM

> I thought it was your locally-running harness doing it.

That is rarely the case (see sibling comment for reasons). Even open source harnesses will usually guide you to connect a hosted provider for that.