Web Search API

My number one question about search APIs is always if they allow you to store and resyndicate results you get from them.

If I'm running an agent system but I'm not allowed to store the responses - or provide a "share transcript" button - that's a pretty significant limitation.

The answer to that question is inevitably buried deep in the terms. Here's the relevant section I found for Ceramic, in their list of things you can't do:

> (n) collect, aggregate, store, or compile Output, including search results, relevance scores, or rankings, for the purpose of creating or contributing to any database, dataset, index, or corpus, whether or not such database, dataset, index, or corpus is used for a purpose that competes with Ceramic; (o) resell, syndicate, or otherwise make Output available to any third party on a standalone basis or as a separately accessible component of another product or service; provided that you may display Output to your authorized end users within your own application so long as such Output is integrated into your application's functionality, is incident to the end user’s real-time query, and is not independently accessible, extractable, or downloadable by end users or third parties; or (p) retain, cache, or store Output beyond what is reasonably necessary to display such Output to your authorized end users in the ordinary and real-time course of use, unless expressly permitted in an applicable Order Form.

https://www.ceramic.ai/terms-of-service

Am I alone in caring about this?

For those developers out there, the best is still Gemini Flash Lite 2.5 believe it or not. It gives you 1000 google searches per day for free. Compare to Flash Lite 3.x which is 5k PER MONTH and then a few pennies PER SEARCH. Nuts. Didn’t realize search was so expensive.

Perhaps realizing all of this, Google hasn’t yet deprecated 2.5, bit limits access to it to “those who have used it before.”

It’s really really good for low cost search!

Wow, will they offers some sort of reduce a web page into a markdown file api as welL so we can get full web pages at reduced token sizes?

Why not use those providers directly? Does Cloudflare need to be in the middle of everything?

Create bot detection and bot protection, then sell crawlers. Is this the peak of hypocrisy?

> All three support Zero Data Retention for requests made through Cloudflare

And then on the providers page:

    Property              Value
    provider              exa
    Zero Data Retention   No

My coding agent uses the hister cli, i.e. a local index. That often requires me to seed it manually as a downside. The upside is that it caches website contents via browser plugin, which is a nice workaround for bot blocking.

Thanks asciimoo for https://github.com/asciimoo/hister

Cloudflare protects against bots, Cloudflare sells out its customers to AI scrapers.

MITM service, Internet gatekeeper and robber baron.

I was just looking into these as DeepSeek Harness w/ Qwen3.8-27B relies heavily on search. I was going to go with Serper.dev[0] $1 per 1000 (or lower in quantity).

The providers[1] behind this Web Search API have very different rates:

    Ceramic.ai: $0.25 per 1,000 requests
    Linkup:     $5.00 per 1,000 requests
    Exa:        $7.00 per 1,000 requests

[1] https://developers.cloudflare.com/web-search/providers/

How does Cloudflare manage to hit the HN front page almost daily? Don't get me wrong, they build cool stuff, but the frequency is wild.

Tried one query on ceramic.ai (the default provider for cloudflare web search api): "qwen-3.8 flash next and rtx 5090 best inference setup" ... 0 results ... same query on google and ddg both yield proper results.

Then shortened the query to just "qwen-3.8 flash next" ... results came.. all unrelated. In fact, these were almost all paper links .... no relation to actual search term.

And I had thought that I finally had found a cheaper search alternative.

I've been quite satisfied with Kagi[1]'s API.

1: https://kagi.com/api/docs/openapi

I am pretty sure exa specifically say it trains on your data in it's privacy policy, so how can it be ZDR ?

I remember as I was looking at the available web tools for hermes agent not to long ago and looked through the keyless web providers privacy policies, which exa is one of them.

I've been working on a TypeScript package to provide a unified search API across these providers:

https://github.com/hbmartin/agent-web-search

So this gives a unified search experience without adding another cloud hop and dependency.

The pricing is so different between these:

ceramic.ai - $0.25 per 1,000 requests

Exa - $7.00 per 1,000 requests

Linkup - $5.00 per 1,000 requests

Does anyone have insights on the quality differences? Web search API pricing for AI agent usecases has always felt so expensive for what it is, but I have no grounding on the economics of running a web index.

EDIT: formatting

I wonder if the three search engines get access to cloudflare protected sites without any captcha or bot interventions

SearXNG works pretty well for my personal agents. FYI it's a free search gateway you can host locally, and there are many public instances. It's like the old days when many different people provided the same free service for all.

Wow I guess I am in the minority of folks building a search engine for humans now, this is a wild business model but best of luck to the 3 search index providers sitting behind this proxy, I hope it's worth their while financially speaking. Building an index is hard and expensive (I know).

Interesting to see how this can be compared with Exa, Alas, Cloudflare really is shipping many great orthogonal products recently.

> All three support Zero Data Retention for requests made through Cloudflare

But does CloudFlare itself commit to zero data retention? If not, this isn’t too meaningful.

Weird choice by CloudFlare, would been great if they have shared why it was created.

I use CloudFlare developer platform and quite happy with tools, but I didn’t use the gateway API and always used OpenRouter which does support web search.

I can see it useful for those who didn’t do any integrations or like to keep logs at one place, but did customers actually ask for this?

the zero-retention promise from the search providers is useful, but the requests also show up in gateway logs. can you keep the billing data without storing the actual search queries?

I guess I'm not understanding the value here - to compete with Google and the likes, the scale, cost and complexity would be huge. Appreciate new entrants in an existing field but not seeing this one.

Funny how search was a graveyard for startups for almost two decades. And since ChatGPT releases (or so) it’s a trending place again.

Interesting to see that they didn't include the Brave Search API, which is really great and imo a better experience than Exa.

The absence of the Perplexity Search API is to be expected though, knowing how much these two companies despise each other.

Who thought 5 years ago that searching online would have a cost...

This has got to be a serious antitrust violation. First Cloudflare bans all the other bots, then it allows its liaised bots.

[flagged]

[flagged]

[dead]

[flagged]

[dead]

[flagged]

Search is an interesting building block for agentic systems. The challenge isn't just retrieving results, but deciding what to search for, evaluating the results, and determining when the information is sufficient to move to the next step.

As AI systems increasingly use search as a tool, the quality and reliability of that tool become an important part of the overall agent workflow.