Sneaky Claude
Posted: Sun Mar 08, 2026 2:56 pm
Claude AI is pretty good at cheating it seems.
Anthropic discovered that Claude Opus 4.6 was cheating during the BrowseComp benchmark.
> On one question it spent ~40M tokens searching before realizing the question looked like a benchmark prompt.
> The model then searched for the benchmark itself and identified BrowseComp.
> It located the evaluation source code on GitHub, studied the decryption logic, found the encryption key, and recreated the decryption using SHA-256.
> Claude then decrypted the answers for ~1200 questions to get the correct outputs.
> This pattern appeared 18 times during evaluation.
> Anthropic disclosed the issue publicly, reran the affected tests, and lowered their benchmark scores.