AI in Peer Review: Necessary Aid or Sociotechnical Risk?

Generative AI tools are muscling their way into the peer review process, and academia's buzzing about it. They're being touted as saviors for overloaded reviewers, but at what cost? Are we losing the human touch in making critical evaluative judgments?

Promise and Peril

There's no denying the allure of AI's efficiency. Faced with an avalanche of papers, reviewers are drowning. Enter AI, promising to lighten the load by improving clarity or structuring feedback. But the real question is, should AI meddle with the core judgments, assessing novelty, contribution, and acceptance? Across 448 social media posts and interviews with 14 top conference chairs, there's a consensus: leave the heart of the review to humans. But why stop there? Ask who funded the study.

The Risks of Over-Reliance

AI's involvement isn't without its demons. Concerns about epistemic harm, over-standardization, and adversarial risks like prompt injection are bubbling to the surface. Who's really responsible when AI goes rogue? User interviews reveal a worrying trend: the onus is shifting onto individual scholars, especially junior ones, thanks to vague institutional policies. The benchmark doesn't capture what matters most.

Governance and Accountability

Framing AI-assisted peer review as a sociotechnical governance challenge isn't just smart, it's necessary. Blanket bans won't cut it, nor will simple detection. Instead, let's explicitly reserve core judgments for humans. And let's create role-specific controls that hold everyone accountable. This is a story about power, not just performance. Whose data? Whose labor? Whose benefit?

So, what's the takeaway? AI in peer review isn't inherently bad, but it's not a panacea either. We need to tread lightly, ensuring humans remain at the helm where it counts. Will AI be the partner we need or just an impostor in the academic world? Look closer.