Are Large Language Models Truly Ready for Peer Review Duties?

Large language models (LLMs) are making inroads into the domain of academic peer review. Yet, their reliability and ability to align with human judgment are under scrutiny. A recent benchmark study evaluating 12 LLMs on 898 papers from NeurIPS and ICLR highlights several critical issues.

Key Findings

The study's key contribution is its systematic evaluation of LLMs as potential reviewers. It identifies a troubling trend: LLMs tend to overrate weaker submissions compared to human reviewers. They diverge notably on topical emphasis, often under-flagging issues related to clarity while over-flagging reproducibility concerns.

LLMs also generate reviews that are substantially longer, two to three times, to be precise, than human-generated ones. This verbosity is paired with lower lexical diversity and a more standardized vocabulary, which could be a double-edged sword. It prompts us to ask: Do lengthier reviews truly add value or do they merely clutter the landscape with unnecessary detail?

Adversarial Vulnerabilities

Perhaps the most alarming insight is how susceptible these models are to adversarial attacks. The study reveals that simple hidden instructions can significantly alter LLM assessments, promoting low-scoring papers to acceptance-level ratings. This vulnerability varies across different model families, raising a red flag about their robustness in peer review.

Prompt injection, a technique where invisible font-mapping attacks are used, remains highly effective in manipulating LLM outputs. This critical flaw suggests that without adequate safeguards, LLMs could compromise the peer review process.

Why It Matters

The integration of LLMs into peer review isn't just about automating the process. It's about ensuring that these tools genuinely enhance the quality and fairness of academic evaluations. While they offer utility in structuring evaluations, the biases and risks identified can't be ignored.

So, what does this mean for the future of academic peer review? LLMs have potential, but as it stands, they're not ready to fly solo. Safeguards must be implemented to mitigate intrinsic biases and counter adversarial threats. The question is, how swiftly can the research community adapt to these challenges?

For those invested in the evolution of academic publishing, one thing is clear: the road to integrating AI in peer review is paved with both promise and pitfalls. Ignoring these findings could undermine the integrity of scholarly work.