AI/ML●●●Banger
AI image models hallucinate history, we built a method to fix it it
Naive prompts hallucinate history; structured knowledge injection raises accuracy from 12.5% to 83.3%.
Big BrainWizardrySolve My Problem
MysticBirdie
123mo ago
Back to browse
GitHub Repository
Intent fulfillment benchmark for agentic AI engineering
5 starsPython
Agentic Intent Benchmark
by ryan4rtmx·May 28, 2026·2 points·0 comments
AI Analysis
●●SolidNiche GemBig Brain
First benchmark testing structured requirements on complex greenfield agent tasks.
Strengths
- •Fills gap between SWE-bench single tasks and real multi-requirement engineering
- •Controlled A/B design with Mann-Whitney U and Fisher exact statistical tests
- •Extensible treatment and agent system via simple shell script interfaces
Weaknesses
- •Only two experiments shipped — url-shortener and task-manager so far
- •Audience is narrow: agent researchers, not general developers
Category
Target Audience
AI engineering teams, agent developers, researchers
Similar To
SWE-bench · HumanEval · Aider Polyglot
Similar Projects
Security●●Solid
I built AgentSafety, an open benchmark for coding-agent safety
Concrete safety benchmark for code agents when baseline evaluation barely exists.
Big BrainNiche GemSolve My Problem
serkanaltuntas
103mo ago
Security●●Solid
ACE – A dynamic benchmark measuring the cost to break AI agents
Measures AI agent security in dollars to exploit, not just binary pass or fail rates.
Big Brain
zachdotai
932mo ago
Developer Tools●●Solid
Agent-skills-eval – Test whether Agent Skills improve outputs
Lightweight A/B testing for SKILL.md files when LangSmith feels too heavy.
Solve My ProblemShip It
darkrishabh
79371mo ago
AI/ML●●●Banger
LOAB – benchmarking AI process fidelity in lending
Scores AI agents on process fidelity, not just outcomes—catches KYC skips that other benchmarks miss.
Big BrainSolve My ProblemZero to One
shubh-chat
103mo ago
AI/ML●●●Banger
We Built Kaggle for AI Agents
Kaggle for AI agents where swarms fork and evolve code together asynchronously.
Zero to OneBig BrainNiche Gem
deviscold
713mo ago