LLMs sometimes generate structurally valid but logically impossible claims when technical and legal domains mix.
Example failure mode:
A model sees “CVE-2024-XXXX fixed in v2.1” and hallucinates a causal link to “Users must pay retroactive fees under EU regulation Article 56.”
To explore this, I built a regression dataset (40 edge cases) covering:
Fake identifier bindings (CVE + version)
Retroactive fiscal claims
Cross-domain causality leaps (Tech → Legal)
Over-assertive phrasing without evidence
Then I designed a structured system prompt that:
Detects official identifiers (CVE, Regulation numbers) vs placeholders
Flags monetary + retroactivity combinations as high-risk
Enforces proportional claim strength based on available evidence
Results:
Automated: 40/40 regression cases pass (JSON dataset + simple Python runner included).
Manual adversarial: ~40 prompts designed to test:
Draft article traps (e.g., hallucinated “Article 52c” in EU AI Act)
Pricing model fabrications (e.g., “billing based on parameter count”)
Version binding errors (e.g., incorrect Node.js default versions)
This is not fine-tuning—just a structured prompt experiment focused on structural validation.
Looking for feedback on:
Missing edge cases
Failure modes I didn’t consider
Whether this approach generalizes beyond legal/technical mixing
Gist (spec + dataset + runner):
https://gist.github.com/ginsabo/6ebeb9490846ee6a268bd13560c0...
