GitHub Repository

Open-source event streaming platform built on S3. Kafka-compatible APIs, built-in SQL engine, schema registry — one Rust binary replaces Kafka + ZooKeeper + KSQL. Retention costs pennies, not thousands

64 starsRust

StreamHouse – S3-native Kafka alternative written in Rust

Name: StreamHouse – S3-native Kafka alternative written in Rust
Availability: InStock
Author: gbram

by gbram·Feb 25, 2026·10 points·8 comments

Visit Project View on HN

AI Analysis

●●●BangerBig BrainWizardryZero to One

Removes broker disk complexity entirely—S3 as durable log cuts Kafka ops burden and cost dramatically.

Strengths

•Architectural insight: stateless brokers + S3 durability eliminates replication tuning and disk management entirely.
•Kafka-compatible API (Producer, Consumer, consumer groups) means drop-in replacement for existing integrations.
•Throughput verified: 62K writes/sec, 30K+ reads/sec with <10ms metadata latency—production-grade performance metrics.

Weaknesses

•Early-stage (0 stars, fresh repo)—no evidence of production deployment or long-term reliability under load.
•S3 API costs and latency can compound at scale; claimed 10x savings assumes bulk pricing and ignores network costs.

Post Description

Hey HN,

I built StreamHouse, an open-source streaming platform that replaces Kafka's broker-managed storage with direct S3 writes. The goal: same semantics, fraction of the cost.

How it works: Producers batch and compress records, a stateless server manages partition routing and metadata (SQLite for dev, PostgreSQL for prod), and segments land directly in S3. Consumers read from S3 with a local segment cache. No broker disks to manage, no replication factor to tune — S3 gives you 11 nines of durability out of the box.

What's there today: - Producer API with batching, LZ4 compression, and offset tracking (62K records/sec) - Consumer API with consumer groups, auto-commit, and multi-partition fanout (30K+ records/sec) - Kafka-compatible protocol (works with existing Kafka clients) - REST API, gRPC API, CLI, and a web UI - Docker Compose setup for trying it locally in 5 minutes

The cost model is what motivated this. Kafka's storage costs scale with replication factor × retention × volume. With S3 at $0.023/GB/month, storing a TB of events costs ~$23/month instead of hundreds on broker EBS volumes.

Written in Rust, ~50K lines across 15 crates. Apache 2.0 licensed.

GitHub: https://github.com/gbram1/streamhouse

Happy to answer questions about the architecture, tradeoffs, or what I learned building this.