The Argument Collapse: How AI's Predictable Persuasion is Limiting Debate

Large language models (LLMs) are increasingly stepping into the arena of public dialogue. However, their contributions might not be as diverse as once hoped. Instead of enriching debates, they could be compressing them into predictable patterns. The AI-AI Venn diagram is getting thicker, but is it becoming more insightful?

Predictable Patterns

By analyzing 1,039 human responses from 195 New York Times debates, the study contrasts these with 23,384 essays generated by LLMs. The results reveal a stark difference: 65.3% of human arguments are unique, yet only 3.4% of LLM arguments share this trait. This isn't a partnership announcement. It's a convergence towards uniformity.

Even when we nudge LLMs to generate diverse responses, the variety falls short. Typically, models recover just half of the distinct arguments humans present. Much of the added variation doesn't even align with human argumentative space. A striking similarity emerges in sub-arguments too. With 41% uniqueness in human sub-arguments versus 9.1% from LLMs, it's clear that AI-generated content tends to reuse hedged, generalized claims.

Structural Stagnation

Not only are the arguments repetitive, but the structure of LLM essays also lacks creativity. They typically kick off with a direct claim and swiftly transition to solutions, adhering to a fixed narrative arc. This pattern holds even in longer forum pieces, as evidenced by analysis from 61 Boston Review debates. Argument collapse seems pervasive, extending beyond just short responses.

Why does this matter? If LLMs are drafting public-facing content, the richness of human debate could be at risk. Repeatedly encountering similar arguments can dull the public's critical engagement. If agents have wallets, who holds the keys to open debates?

Implications for Public Discourse

This convergence suggests a need to question the reliance on LLMs for generating public arguments. Is it wise to depend on models that inadvertently narrow discussions? The compute layer needs a payment rail, but it also requires diversity in thought.

Machines might be efficient, but argumentation, they lack the depth and creativity human minds offer. If AI is to play a role in public discourse, it'll need to evolve beyond the predictable and embrace the unpredictability that fuels meaningful debate. We're building the financial plumbing for machines, yet we mustn't let them dictate the script of public dialogue.