MTG Bench

TL;DR

A benchmark that uses the insanely complex rules of Magic: The Gathering to see if an LLM can actually think or if it's just a fancy autocomplete.

Who is this actually for?

AI researchers and LLM developers who need a high-complexity logic test that isn't already leaked in the training data.

The Good

• Magic's ruleset is a massive logic puzzle, making it a brutal test for reasoning capabilities.
• It moves away from boring, standardized benchmarks that models have already memorized.

The Catch (Potential Downsides)

Extremely niche for anyone not working on gaming or deep logic; also, the models will likely struggle with the sheer volume of card-specific exceptions.