AI/ML●●Solid
Nexa-Gauge – LLM eval framework, now with self-hosted model support
Cache-aware LLM eval with self-hosted model support beats Ragas on flexibility.
Solve My ProblemSlick
Sardhendu
201mo ago
Back to browse
GitHub Repository
An graph-eval framework for LLM's
38 starsPython
Nexa-gauge – Cache/cost-aware graph-based eval for LLM and RAG
by Sardhendu·May 9, 2026·3 points·0 comments
AI Analysis
●●SolidSolve My ProblemSlick
Cache-aware execution cuts eval costs while tracking grounding and relevance metrics.
Strengths
- •Graph-based evaluation allows selective node execution instead of running full suites.
- •Built-in cost estimation helps teams budget large-scale regression testing runs.
- •Supports Hugging Face datasets and uv for modern Python workflow integration.
Weaknesses
- •LLM evaluation space is crowded with Arize, LangSmith, and Ragas already established.
- •Graph abstraction may add complexity for teams needing simple pass/fail metrics.
Category
Target Audience
ML engineers, LLM application developers, QA teams
Similar To
Ragas · LangSmith · Arize Phoenix
Similar Projects
Infrastructure●Mid
Nexus Gateway – Reduce LLM API Costs Using Semantic Caching
Semantic caching for LLM APIs exists (Anthropic prompt caching, Langchain, Miniplex, vLLM); gateway routing is table stakes.
Ship ItSolve My Problem
Sunnyanand_dev
213mo ago
AI/ML●●Solid
Replaced Neo4j with pure vector search for Graph RAG
Graph RAG without Neo4j — pure vector search beats HippoRAG on multi-hop benchmarks.
Big BrainDark Horse
zhangchen
202mo ago
Infrastructure●●Solid
AI Cost Firewall – OpenAI-compatible gateway with semantic caching
LLM gateway with Redis + Qdrant caching, but LiteLLM does this.
SlickShip It
vcaluser
112mo ago
AI/ML●●Solid
An agent that tunes its own cache
Two-tier caching saves real money, shown live on the dashboard.
Ship ItBig Brain
kaliades
701mo ago
Developer Tools●●Solid
Argmin AI, system level LLM cost optimization for agents and RAG
LLM cost optimizer, but Anthropic's batch API and local quantization solve this cheaper.
Solve My ProblemBig Brain
konyrevdmitriy
203mo ago