Infrastructure●●Solid
I embedded 685M public texts in 32 minutes (on 8x A100, Rust, TensorRT)
3.6x faster than Hugging Face TEI on same hardware with zero Python overhead at runtime.
WizardryBig Brain
ddayanov
7013d ago
Back to browse
GitHub Repository
Fast self-hosted embedding engine for search, RAG, and reindexing workloads on NVIDIA GPUs. Built in Rust + TensorRT for teams that care about scale, cost, and control.
7 starsRust
IgniteMS – batch text embeddings at 253K msg/s on 8x A100
by ddayanov·May 20, 2026·3 points·0 comments
AI Analysis
●●●BangerWizardrySolve My Problem
Beats Hugging Face TEI by 3x with raw TensorRT and zero Python runtime overhead.
Strengths
- •Bucketed batching reduces padding waste by grouping texts of similar token lengths.
- •Production proof: sustained 357K msg/s embedding 685M social media events.
- •Cost efficiency drops embedding price to $0.01 per million messages on spot instances.
Weaknesses
- •Requires NVIDIA GPUs and ONNX export compatibility, excluding CPU or Mac users.
- •Niche utility for batch reindexing rather than real-time inference serving.
Category
Target Audience
ML engineers building RAG pipelines or vector search indexes
Similar To
Hugging Face Text Embeddings Inference · NVIDIA Triton Inference Server
Similar Projects
AI/ML●Mid
Fastembed-rs – Rust library for generating vector embeddings, reranking
Rust port of Qdrant's fastembed when the Python original already works fine.
Slick
thoughtfullyso
202d ago
Infrastructure●Mid
ApexStore – An embedded LSM-Tree storage engine written in Rust
Rust LSM-Tree engine, but RocksDB and Redb already dominate this space.
Niche Gem
texuguito
113mo ago
Security●●●Banger
Zero-allocation embedded security in Rust (fits in 256KB Flash)
Sub-microsecond CAN frame detection with zero heap allocation in 122K lines of Rust.
WizardryBig BrainNiche Gem
victor-craton
401mo ago
Data●●Solid
IssunDB – a new embedded graph database with vector and text search
Sparse matrix graph operations with MCP server integration for AI agents.
Big BrainShip It
habedi0
404d ago
Data●●●Banger
Clark Hash, 32x smaller searchable sketches for embeddings
32x embedding compression without calibration beats product quantization's training overhead.
Big BrainWizardry
stan_kirdey
1021d ago