Data Projects

Data tools and analytics from Show HN — visualization, ETL pipelines, databases, and data science utilities.

Sort:

389 projects

Data

●●●●Gem

I analyzed 227M Medicaid billing records to flag potential fraud

Public fraud detection across 227M Medicaid claims—1,860 flagged providers, zero false positives yet.

Zero to OneSolve My ProblemBig Brain

kianoconnor

303mo ago

Data

●●●●Gem

I analyzed 1.3M federal cases for racial sentencing bias

1.3M case analysis proves systemic sentencing bias with rigorous controls; visceral real-time cost counter.

Big BrainSolve My ProblemZero to One

justiceindex

213mo ago

GitHub

High-Performance GPU Cuckoo Filter

39Cuda

Data

●●●●Gem

Cuckoo-GPU – A 350x faster Bloom filter alternative for GPUs

350x faster GPU Bloom filter with academic paper backing the performance claims.

WizardryBig BrainDark Horse

tdortman

112mo ago

Data

●●●Banger

Govbase – Follow a bill from source text to news bias to social posts

Bills + AI summaries + bias-rated news + politician posts all in one.

Solve My ProblemZero to One

foxfoxx

219893mo ago

GitHub

PostgreSQL extension for BM25 relevance-ranked full-text search. Postgres OSS licensed.

3,783C

Data

●●●Banger

How This Graybeard Built the Fastest and Freest Postgres BM25 Search

Native Postgres BM25 with ORDER BY content <@> 'query' operator syntax.

Big BrainSolve My Problem

tjgreen

203562mo ago

Data

●●●Banger

What was the world listening to? Music charts, 20 countries (1940–2025)

Explore 230 global music charts from 1940–2025 with a Back to the Future UI.

Rabbit HoleEye CandyZero to One

matteocantiello

114513mo ago

GitHub

ACORN-1 pre-filtered HNSW search for DuckDB

62C++

Data

●●●Banger

DuckDB community extension for prefiltered HNSW using ACORN-1

ACORN-1 pushes WHERE filters into HNSW traversal, fixing pgvector's post-filter limitation.

Big BrainDark HorseSolve My Problem

cigrainger

9052mo ago

Data

●●●Banger

Large Scale Article Extract of Newspapers 1730s-1960s

Finally, a history archive that lets you search by meaning instead of just keywords.

Rabbit HoleBig Brain

brettnbutter

57201mo ago

Data

●●●Banger

Ismcpdead.com – Live dashboard tracking MCP adoption and sentiment

Real-time MCP sentiment tracker when everyone's debating if the protocol is dead or thriving.

Niche GemShip It

sagirodin

39232mo ago

Data

●●●Banger

GeoTraceroute – Traceroutes on a 3D globe and submarine cables

Infers invisible submarine cable paths using A* pathfinding between ocean-crossing hops.

WizardryRabbit Hole

Himred

2111mo ago

Data

●●●Banger

A search engine for deleted YouTube videos (1.5B+ indexed since 2005)

Recovers metadata for deleted videos across 1.5B indexed entries since 2005.

Dark HorseRabbit Hole

archivarix

14121mo ago

Data

●●●Banger

Stratum – SQL that branches and beats DuckDB on 35/46 1T benchmarks

Git-like branching for columnar data with DuckDB-beating benchmarks from pure JVM.

Big BrainWizardry

whilo

1233mo ago

Data

●●●Banger

Wikigraph – an interactive visualization of all of English Wikipedia

GPU-accelerated layout renders 7M articles with shortest-path finding between any two.

Rabbit HoleWizardryEye Candy

tfpgh

12210d ago

Data

●●●Banger

DeepTable – an API that converts messy Excel files into structured data

Agent-guided compilation handles merged cells and multi-level headers LLMs choke on.

Solve My ProblemBig Brain

francisrafal

802mo ago

Data

●●●Banger

I indexed the academic papers buried in the DOJ Epstein Files

207 papers buried in government records, now searchable with full context extraction.

Zero to OneBig BrainNiche Gem

am-seo

703mo ago

GitHub

Distributed DataFrames for Elixir powered by DuckDB

77Elixir

Data

●●●Banger

Distributed DuckDB on the Beam

Distributes DuckDB queries across BEAM nodes when Polars maintenance became unsustainable.

Big BrainNiche Gem

cigrainger

702mo ago

GitHub

Open-source streaming SQL engine written in Rust using Apache Arrow and DataFusion. Supports continuous queries, temporal stream joins, tumbling/session windows, and CDC/Kafka connectors. Lightweight, embeddable, and sub-microsecond latency

37Rust

Data

●●●Banger

LaminarDB – Streaming SQL database in Rust, zero-alloc hot path

Sub-microsecond streaming SQL via zero-alloc hot path; genuine advancement over SQLite+DataFusion.

WizardryNiche GemSolve My Problem

sujitn

613mo ago

Data

●●●Banger

DAGraph – local-first reactive graph from arithmetic to OLAP SQL

Reactive SQL graph in the browser beats Tableau for local prototyping.

Zero to OneCozySolve My Problem

notfirstpost

611mo ago

Data

●●●Banger

Visualizing How Books Reference Each Other Across 3k Years

3,000 years of literary citation networks visualized with WebGPU; LLM pipeline resolves messy bibliographic data.

WizardryRabbit HoleEye Candy

farcaster

534mo ago

Data

●●●Banger

We benchmarked 18 LLMs on OCR (7K+ calls) – cheaper models win

7,560 runs proving cheaper models beat expensive ones on production OCR tasks.

Big BrainSolve My Problem

TimoKerr

511mo ago

Data

●●●Banger

Flop Map – a 3D globe of every known AI compute cluster

Stunning 3D globe tracking 725 AI clusters and 80M H100 equivalents globally.

Eye CandyRabbit Hole

kitwebster

531mo ago

Data

●●●Banger

Skymap – a 3D galaxy catalog explorer in the browser via WebGPU

WebGPU renders millions of galaxies in-browser with real-time density correction filters.

Rabbit HoleEye CandyWizardry

thereallex

591mo ago

Data

●●●Banger

OrcaSheets, local first analytics engine to process billions of rows

Runs billions of rows locally on your machine, bypassing cloud warehouse costs entirely.

Big BrainSolve My Problem

ydgandhi

5428d ago

Data

●●●Banger

Searchable aggregator of 24M London council spending transactions

24M spending rows searchable instantly; missing data but fills a real transparency gap.

Dark HorseSolve My Problem

ashfn

453mo ago

Data

●●●Banger

Open load forecasts that beat US grid operators on 6 of 7 RTOs

Beats utility forecasts on 6 of 7 RTOs using only public EIA data and open models.

Big BrainDark HorseZero to One

tylergibbs1

401mo ago

GitHub

Probabilistic Spreadsheets

1TypeScript

Data

●●●Banger

Spreadsheets Cells with Uncertainty Distributions

Monte Carlo sampling inside a spreadsheet beats collapsing uncertainty into single scalars.

Big BrainNiche Gem

tobiadefami

401mo ago

GitHub

Column-oriented analytics for SQLite, designed for fast local analytical queries with benchmarked speedups of up to 130,000x over standard SQLite scans on selected workloads.

Data

●●●Banger

SQLite-Columnar – Columnar Storage for SQLite

Columnar storage inside SQLite delivers 130,000x speedup on aggregation scans.

WizardryBig Brain

marcobambini

401mo ago

GitHub

The First Real-Time Search Analytics Database

534C++

Data

●●●Banger

Zero-ETL elastic-grade search over remote data (Parquet, Iceberg, JSON)

BM25 search directly on S3 Parquet files without ingesting data into Elasticsearch.

Big BrainBold Bet

gnusi

4028d ago

GitHub

Local-first hybrid vector database. HNSW + BM25 + fusion search. Single binary, no cloud.

9Rust

Data

●●●Banger

Vecdb – local-first hybrid vector database in Rust (HNSW and BM25)

Single-binary Rust DB fusing HNSW and BM25 without cloud dependencies or API keys.

Big BrainZero to One

zaydmulani

4024d ago

Data

●●●Banger

Metaxy – versioning for multimodal data pipelines

Sample-level versioning for ML pipelines—solves real compute-waste problem with clever metadata design.

Niche GemWizardry

danielgafni

313mo ago

GitHub

A command-line tool to extract plain text from Wikipedia dumps with category and section filtering

194Ruby

Data

●●●Banger

WP2TXT – Wikipedia dump text extractor with category/section filtering

Category-aware Wikipedia text extraction with 20-year maintenance history and parallel M4 speed.

Niche GemDark Horse

yohasebe

303mo ago

GitHub

Rasteret is a library for 20x+ faster reads of GeoTIFF than Rasterio/GDAL. Interops with TorchGeo, Xarray, DuckDB, Polars

132Python

Data

●●●Banger

Rasteret-library for fast table like access to Satellite Imagery

Parquet-cached COG metadata cuts satellite image cold starts from minutes to seconds.

Niche GemSolve My ProblemWizardry

sid_tf

303mo ago

Data

●●●Banger

Conflicts.app, Iran conflict dashboard better then alternatives

Geospatial conflict dashboard beats raw-feed competitors with spatial intel and AI signal curation.

Big BrainNiche GemDark Horse

juliusolsson

303mo ago

Data

●●●Banger

ColumnLens – Query millions of rows in milliseconds on your Mac

DuckDB-powered desktop app queries 5GB CSVs locally in seconds without cloud uploads.

SlickSolve My ProblemWizardry

hilti

313mo ago

GitHub

Bankstatemently Open Benchmark for bank statement parsing. 15 synthetic statements, 40 parsing challenges, automated scoring via API.

1Shell

Data

●●●Banger

Open-source synthetic bank statements for testing parsers

First shared benchmark for bank statement parsers when real data is legally off-limits.

Solve My ProblemZero to OneDark Horse

Maesh

302mo ago

Data

●●●Banger

Live global gas station inventory

Live fuel levels from 4k exposed industrial sensors—basically FlightRadar24 for gas stations.

Dark HorseZero to One

t819291

302mo ago

Data

●●●Banger

Sixteen year trends in AI doom on HN

Scores 16 years of HN comments for AI pessimism, showing a doubling trend since ChatGPT.

Rabbit HoleBig BrainNiche Gem

easygenes

302mo ago

Data

●●●Banger

I scraped 17 months of "Who is hiring?" – what 6,539 jobs require

Finally proves Rust has overtaken Java in HN hiring with hard numbers, not vibes.

Dark HorseBig Brain

aminedhuibi

301mo ago

Data

●●●Banger

All the government's 330K UFO files so far as 1 12-mile long microfilm

Stitched 330K files into a 12-mile scroll that actually scrolls smoothly in the browser.

Rabbit HoleZero to OneBig Brain

keepamovin

3020d ago

Data

●●●Banger

I built a better way to track regulatory action

Automates regulatory compliance tracking—links proposed rules to final text with diffs and extracted obligations.

Solve My ProblemZero to One

tldrthelaw

213mo ago

Data

●●●Banger

I built an outdoors data API for developers

Replaces scraping dozens of inconsistent government APIs with one normalized REST endpoint.

Solve My ProblemBig Brain

caderosche

202mo ago

Data

●●●Banger

CDMO Signal- FDA, EMA, and clinical trial data across 190 pharma CDMOs

Independent CDMO scoring prevents pay-to-play bias in vendor selection.

Niche GemSolve My Problem

spanjwani3

212mo ago

Data

●●●Banger

WorldSim – Monte Carlo simulation for 195 countries to 2050

100+ structural coupling rules enforce causal economics across 195 countries with probabilistic outputs.

Big BrainBold BetWizardry

Yiannis91

202mo ago

GitHub

A new search paradigm where documents have gravity, queries converge into basins, and multi-signal scoring uses interference instead of linear fusion.

36Rust

Data

●●●Banger

I rebuilt search using physics instead of statistics. +18.5% NDCG 10

Gravity-based search with interference scoring beats linear fusion by 18.5% NDCG.

Big BrainWizardryZero to One

KendallCBooker

202mo ago

GitHub

Query the full Hacker News archive from Postgres via duckdb_fdw, with zero copies. Stream row groups straight from the Hugging Face Parquet dataset on demand.

4Python

Data

●●●Banger

HN-fdw – All of Hacker News, queryable from Postgres, with zero copies

Zero-copy Postgres queries against 47M rows using DuckDB FDW and HTTP range requests.

WizardryBig BrainDark Horse

tamnd

202mo ago

Data

●●●Banger

NRC nuclear licensing RAG pipeline and regulatory embeddings dataset

First public NRC regulatory embeddings dataset—37K chunks ready for ChromaDB and Pinecone.

Niche GemSolve My Problem

davenporten

202mo ago

GitHub

🚀 GizmoSQL — High-Performance SQL Server

324C++

Data

●●●Banger

DuckDB / DuckLake Server (With Arrow Flight SQL) for iOS

TPC-H 1GB in 2 seconds on iPhone—Arrow Flight SQL running locally.

WizardryZero to One

philbe77

201mo ago

GitHub

An experimental embedded SQL engine in C++20. Query Parquet, CSV, JSON, Arrow, Avro, SQLite, and Excel files directly with SQL, in-process. Early-stage.

493C++

Data

●●●Banger

SlothDB is a super fast embedded SQL database

1.3MB WASM database beats DuckDB 5x on 10M row analytics, runs in Cloudflare Workers.

WizardryBig BrainZero to One

souravroy78

221mo ago