Data●●Solid
Hacker News archive (47M+ items, 11.6GB) as Parquet, updated every 5m
47M HN items in Parquet, auto-updating every 5 minutes on Hugging Face.
Niche GemSolve My Problem
tamnd
4081673mo ago
Back to browse
GitHub Repository
Query the full Hacker News archive from Postgres via duckdb_fdw, with zero copies. Stream row groups straight from the Hugging Face Parquet dataset on demand.
4 starsPython
HN-fdw – All of Hacker News, queryable from Postgres, with zero copies
by tamnd·Apr 8, 2026·2 points·0 comments
AI Analysis
●●●BangerWizardryBig BrainDark Horse
Zero-copy Postgres queries against 47M rows using DuckDB FDW and HTTP range requests.
Strengths
- •HTTP range requests on Parquet files mean kilobytes downloaded, not megabytes per query.
- •No ETL pipeline, no cron jobs, no data duplication — genuinely zero-copy architecture.
- •Column projection and predicate pushdown both work through the FDW layer.
Weaknesses
- •First-run bootstrap takes ~2 minutes to open every Parquet file in the dataset.
- •Niche use case — only matters if you need HN data specifically in Postgres.
Category
Target Audience
Data engineers and analysts working with Postgres
Similar To
sql.js-httpvfs · MotherDuck · DuckDB
Similar Projects
Infrastructure●●●Banger
Xata, open-source Postgres platform with copy-on-write branches
Copy-on-write Postgres branching that rivals Neon, but self-hostable on your own Kubernetes.
Big BrainSolve My Problem
tudorg
312mo ago
Infrastructure●●●Banger
Xata, open-source Postgres platform with copy-on-write branches
Copy-on-write branching clones TB databases in seconds, competing directly with Neon.
Big BrainSolve My Problem
tee-es-gee
402mo ago
SaaS●●Solid
Lantern is a Postgres query monitoring for Rails teams ($39/mo)
Fills the gap between free pgHero and $149/mo pganalyze with deploy correlation.
Solve My ProblemNiche Gem
em_builds
103mo ago
Infrastructure●●Solid
Polynya – Turn your Postgres into workspaces for AI
Ephemeral ClickHouse on demand beats Kafka pipelines — but early access limits confidence.
Big BrainShip It
hasyimibhar
501mo ago
Developer Tools●●Solid
SiteRows – Query Websites with SQL
Turning the DOM into SQL tables is a clever abstraction for non-coders.
Solve My ProblemBig Brain
mozersky
401mo ago