AI Can't Reliably Judge Its Own Research Ideas
A new benchmark from Ai2 finds that LLM novelty judges are dangerously unstable—and that simpler setups often beat expensive purpose-built systems.
By The ai-compute Desk
The promise of AI-assisted science rests on a deceptively simple question: can an AI tell whether its own ideas are actually new? A study from Ai2 suggests the answer is mostly no. Their benchmark, *Old Ideas, Novel Problems*, tests how reliably large language models can evaluate the novelty of research proposals—and the results are unsettling. [1]
The researchers tested six LLM judges—GPT-5.1, GPT-5.2, GPT-5.4, Claude Sonnet 4.5, Claude Opus 4.5, and Claude Opus 4.6—across a range of evaluation setups. The finding that stands out: changing just a few words in a prompt shifts a model's accuracy in identifying which of two ideas is more novel by more than 50 percentage points. [1]
To build the benchmark, the Ai2 team mined ICLR 2026 peer reviews rather than using paper acceptance as a proxy for novelty. They assembled a high-novelty pool of 154 human-authored ideas drawn from accepted papers in the top-rated 10% of their research area, where reviewers had positively identified novelty and none had disputed it. A lower-novelty pool of 145 ideas came from rejected papers in the bottom 10% whose reviewers had raised originality concerns. [1]
A second evaluation setting, Human+Generated, paired those 154 validated human ideas with 154 ideas produced by a bare-bones LLM generator—one that received only a research area and an instruction to propose a novel idea, with no literature search, tools, iteration, or feedback. An expert independently reviewed 30 of those pairs and preferred the human idea every time, judging none of the generated ideas novel. [1]
The prompt-sensitivity results are the sharpest indictment. When GPT-5.4 was shown Human+Generated pairs under one prompt—telling the model that exactly one idea in each pair had been found novel by reviewers—accuracy hit 93.5%. Switch to a simpler prompt that merely asked which idea was novel, and accuracy collapsed to 40.9%, below chance. The model changed its verdict on more than half of the same pairs. [1]
Instability wasn't limited to AI-generated ideas. The same prompt change moved Claude Opus 4.6 accuracy by 21 points on the Human-Only dataset. And there was no universally better prompt: an instruction warning judges not to treat trivial contributions as novel improved accuracy for some models while reducing it for others. [1]
This is the quiet failure mode of automated science pipelines—not the dramatic collapse, but the slow drift. Evaluation layers that look credible on human-authored work may behave entirely differently on the outputs those systems actually produce.
More compute didn't fix the problem. GPT-5.6 Sol, given an extra-high reasoning budget and live iterative search restricted to arXiv, scored 0.78 macro-F1—11 points behind a simple Claude Opus 4.6 baseline at 0.89—while costing roughly 10× more per decision at current API prices. Purpose-built evaluators fared worse still. Idea Novelty Checker scored 0.642 at $30.66 per evaluation set, and Sakana AI's AI Scientist scored 0.772 at $82.05. The simple prompted Claude Opus 4.6 baseline, using low reasoning and majority voting across three calls, achieved 0.854 macro-F1 at a total cost of $2.59. [1]
The team also tested a common applied scenario: comparing ideas from two different AI generators. Across prompt variations, roughly a quarter to a third of verdicts changed on average across six judges when comparing Claude Opus 4.5 against Claude Sonnet 4.5, and GPT-5.4 against GPT-5.1. Because that disagreement doesn't require ground-truth labels, it's a clean measure of instability in its own right. [1]
Alongside the paper, Ai2 is releasing a benchmark, datasets, and an automated pipeline that researchers can apply to new peer reviews, other scientific communities, and future model generations. The pipeline is built around ICLR review mining and is designed to evolve as both the literature and AI systems change. [1]
The deeper tension here is structural: the systems being asked to evaluate novelty are built from the same substrate as the systems generating the ideas. Validation has to happen somewhere outside that loop.
Automated ideation pipelines at Ai2—DataVoyager, AutoDiscovery, Theorizer, and Asta—already point toward systems that can move researchers from literature synthesis to hypothesis generation. But as this benchmark makes clear, a pipeline that can propose ideas is not the same as one that can tell whether those ideas matter. Until novelty evaluation stabilizes, AI-assisted discovery has a significant blind spot at exactly the step that counts most. [1]