AI/ML●●●Banger
LLM Sycophancy Benchmark: Opposite-Narrator Contradictions
Opposite-narrator test catches models agreeing with both sides of same dispute.
Big BrainDark Horse
zone411
303mo ago
Back to browse
Pokémon SVG Generation LLM Benchmark
by haxfenx·May 14, 2026·2 points·0 comments
AI Analysis
●●SolidNiche GemCrowd Pleaser
Finally, a benchmark that uses Pokémon to test if models understand complex geometry.
Strengths
- •Uses a fun, recognizable dataset (Pokémon) to make abstract SVG generation metrics concrete.
- •Breaks down scoring into geometry, features, and complexity for nuanced comparison.
- •Includes an interactive quiz to let users manually verify model outputs.
Weaknesses
- •SVG generation is a narrow slice of multimodal capability compared to image understanding.
- •Lacks a clear methodology for how the 'Visual Score' is calculated programmatically.
Category
Target Audience
AI researchers and developers interested in multimodal model capabilities
Similar To
SVG-Bench · GenAI Benchmarks
Similar Projects
AI/ML●●Solid
LLM Debate Benchmark
Side-swapped debate matchups expose model weaknesses standard benchmarks miss.
Big BrainDark Horse
zone411
932mo ago
AI/ML●●Solid
ErrataBench - A Proofreading Benchmark for LLMs
51 models, 1613 runs, $558 spent — finally proofreading benchmarks with real numbers.
Niche GemBig Brain
artursapek
302mo ago
AI/ML●Mid
My "home rig" for iterative attribute-weighted LLM benchmarking
Home rig for attribute-weighted benchmarking lacks the polish of established eval frameworks.
Ship It
yuvalhaim
211mo ago
AI/ML●Mid
A benchmark where LLMs make memes from current news
Automated meme generation is fun, but lacks depth beyond the novelty.
Crowd Pleaser
max-azendorf
411mo ago
AI/ML●●Solid
ModelSweep - Open-Source Benchmarking for Local LLMs
Postman for local LLMs with LLM-as-Judge and Elo ratings built in.
Ship ItNiche GemSlick
leonickson
203mo ago