SiteOne Crawler – Single-binary CI/CD gate for web quality regressions
Rust-powered crawler with CI/CD gates where Screaming Frog costs money.

Independent search index in Rust when Tavily and Serper wrap Google.
AI agent developers, LLM application builders
Tavily · Serper · Jina AI
Seltz is a web search API built for AI agents. We wrote the crawler, the index, and the retrieval models ourselves, in Rust, by a team that's spent years building web search at scale. In our tests, queries come back in under 200ms.
Efficiency was the first design principle. Search sits on the critical path: agents can't generate their first tokens or kick off the next tool call until results come back. When you run tens or hundreds of queries in parallel, every millisecond of tail latency compounds.
Most search APIs for agents are wrappers around Google or Bing. If your agent already has a Google tool, a second call to a Google-wrapped API returns the same ten documents. We run our own independent index, so you get different results and different rankings.
Coverage starts with US news. More verticals are coming.
There's a free tier with $100 in credits at the link. I'd love feedback from anyone building agents: how does it compare to what you're using, and where does it fall over?
Rust-powered crawler with CI/CD gates where Screaming Frog costs money.
Competent Rust crawler when Scrapy and crawlee already solve this.
Coding agents miss research knowledge; this surfaces 2M+ papers with benchmarks.
Checks if GPTBot and ClaudeBot are blocked before Google even sees your page.
Referenced element indexing cuts token spend 3-10x versus DOM-dumping AI browsers.
40x token reduction vs Playwright by returning tree deltas instead of DOM.