Developer Tools●●Solid
CLI for crawling documentation sites into Markdown with defuddle
No-browser doc crawler when JinaAI and Firecrawl already dominate this space.
Solve My ProblemNiche Gem
nistuley
205d ago
Back to browse
GitHub Repository
Lightweight CLI for crawling documentation sites into Markdown with defuddle
3 starsTypeScript
CLI for crawling documentation sites into Markdown with defuddle
by nistuley·Jun 3, 2026·5 points·0 comments
AI Analysis
●●SolidShip ItSolve My Problem
No-browser docs crawler using defuddle when Firecrawl and JinaAI already exist.
Strengths
- •No browser dependency means lighter weight than Puppeteer-based scrapers
- •Defuddle integration handles messy docs HTML cleanly
- •Manifest.json with content hashes enables incremental updates
Weaknesses
- •Doesn't execute JavaScript, so dynamic docs sites won't render
- •RAG docs ingestion already served by Firecrawl, JinaAI Reader
Category
Target Audience
Developers building RAG pipelines or local knowledge bases
Similar To
Firecrawl · JinaAI Reader · Crawlee
Similar Projects
Other●●Solid
The Crawl Times – Newspaper-style front pages for tech news sites
Cloudflare /crawl API powers nostalgic newspaper layouts for Hacker News and friends.
Eye CandyCozy
g_br_l
503mo ago
Developer Tools●●Solid
Dw2md – Compile all DeepWiki pages into a single, LLM-friendly file
DeepWiki scraper for Claude, but Jina and Firecrawl already do this better.
Solve My ProblemNiche Gem
nwyin
104mo ago
Developer Tools●●●Banger
DocMCP – Index any docs site locally, search it from Claude via MCP
Hybrid BM25 + vector search via MCP beats pure keyword or pure semantic for API docs.
Big BrainSolve My Problem
pieeee
203mo ago
Developer Tools●Mid
Qapir – Generate API tests automatically from docs
Auto-generates API tests from OpenAPI specs when Schemathesis and Postman already exist.
Ship ItSolve My Problem
gradnichkovski
322mo ago
Developer Tools●Mid
DocOCR – Convert document images to Markdown locally on macOS
Local Markdown OCR via CLI and HTTP, though macOS Live Text overlaps heavily.
Solve My ProblemCozy
riddleling
4021d ago