32
AI benchmarks are self-promoting trash — but regulators keep using them
(pivot-to-ai.com)
Still occasionally think about that bit in the o1 white paper where the openai researchers innocuously pose the question of what if our benchmarks for detecting hallucinations are shit actually, wouldn't that be something.