Back to browse
GitHub Repository

Key-Pool-Rotation inspired by OpenClaw

4 starsTypeScript

Key rotation for LLM agents shouldn't require a proxy

by EmptyDrum·Mar 6, 2026·5 points·2 comments

AI Analysis

●●●BangerSolve My ProblemBig BrainShip It

In-process key rotation with state machine simplicity instead of LiteLLM/Redis overhead.

Strengths
  • Clear architectural argument: rate-limit/auth/billing failures are state-machine problems, not distributed systems problems. Eliminates proxy/queue overhead.
  • Specific, tuned exponential backoff: 1min→5min→25min→1h for transient errors; 5h→10h→20h→24h for permanent failures. Production-tested.
  • Error classification built-in (HTTP status codes, message patterns, timeout detection) — you don't write custom error handlers for each provider.
Weaknesses
  • No persistent state demo or real-world examples showing restart behavior. Storage path exists but feels underbaked.
  • Only supports TypeScript/Node; no Python, Go, or language-agnostic REST API despite multi-agent claim.
Target Audience

Backend engineers and AI teams running multi-key LLM agents or services

Similar To

LiteLLM · vLLM's key manager · Langchain's LLM router

Post Description

I think in-process key management is the right abstraction for multi-key LLM setups. Not LiteLLM, not a Redis queue, not a custom load balancer. Why? Because the failure modes are well-understood. A key gets rate-limited. You wait. You try the next one. Billing errors need a longer cooldown than rate limits. When all keys for a provider are exhausted, you fall back to another provider. This is not a distributed systems problem — it's a state machine that fits in a library. The problem is everyone keeps solving it with infrastructure instead. You spin up a LiteLLM proxy, now you have a Python service to deploy and monitor. You reach for a Redis-backed queue, now you have a database for a problem that doesn't need one. You write a custom rotation script, now it lives in one repo and your three other agent projects don't have it. key-carousel gives each pool a set of API key profiles with exponential-backoff cooldowns. Rate limit errors cool down at 1min → 5min → 25min → 1hr. Billing errors cool down at 5hr → 10hr → 20hr → 24hr. Fallback to OpenAI or Gemini when all Anthropic keys are exhausted. Optional file-based persistence so cooldown state survives restarts. Zero dependencies.

Similar Projects

Infrastructure●●Solid

VoidLLM – privacy-first LLM proxy (Go, self-hosted)

Zero-knowledge architecture means prompts never touch disk — unlike LiteLLM.

Solve My ProblemSlick
chrisremo85
342mo ago
Developer Tools●●Solid

Agent Firewall – Go proxy to kill LLM death spirals

Wire-protocol circuit breaker for agents when LangSmith costs too much.

Solve My ProblemShip It
wuweiaxin
213mo ago