Back to browse
GitHub Repository

Is a domain genuinely dead, or just blocking your bot? A passive, local, MIT Go CLI + library that classifies domain reachability (alive/redirect/blocked/dead). The open methodology behind the Crawlora Dead-Web Index.

0 starsGo

Crawlora-deadweb – tell if a domain is dead or just blocking your bot

by tonywangcn·Jun 19, 2026·2 points·0 comments

AI Analysis

●●SolidBig BrainNiche Gem

Distinguishes blocked from dead domains when most link checkers conflate them.

Strengths
  • Open methodology backs the Dead-Web Index research with measurable 14% vs 27% findings.
  • NDJSON output with parallel probing fits directly into data pipelines.
  • Dependency-free Go binary with prebuilt releases for all major platforms.
Weaknesses
  • Browser-fingerprint arm requires their hosted engine, not fully local or open.
  • Niche audience limits broader adoption beyond web crawling use cases.
Target Audience

Web crawlers, link checkers, SEO tools, researchers

Similar To

LinkChecker · W3C Link Checker · Screaming Frog

Similar Projects

Security●●Solid

Laravel middleware that logs attacks-injection, XSS, bots, never blocks

Passive threat detection middleware for Laravel—zero risk since it never blocks, 130+ patterns logged locally.

Solve My ProblemDark Horse
jay123anta
213mo ago