Can LLM Agents Break Free from Cognitive Monocultures?

As large language model (LLM) agents become more prevalent in prediction markets and collective decision-making, a significant risk emerges: cognitive monoculture. Essentially, when agents are built upon the same foundational models, their forecasts tend to correlate. Recent measurements show a staggering correlation of errors at around 0.77 among frontier models. This raises a critical question: Can we recover human cognitive diversity and infuse it into these LLM agents?

The Nous Experiment

Enter Nous, a project that aims to extract an eight-dimension behavioral profile from real trading activity on Polymarket and inject it into LLM agents through prompts. The researchers found some success with the extraction process. Profiles from 100 wallets showed temporal stability in eight out of 14 parameters, with the contrarian score achieving an impressive intra-class correlation (ICC) of about 0.9. Additionally, these profiles were identifiable above random chance, with top-1 retrieval rates between 17-22% compared to a 1% chance.

However, the prompt-level injection of this diversity into LLM models, the results were underwhelming. The structured injection showed no measurable improvement over a length-matched control on any model, neither reducing ensemble error correlation nor enhancing Brier scores. Essentially, the intended diversity didn't translate into better performance or less correlation.

Why This Matters

Strip away the marketing and you get to the heart of the problem: while measuring cognitive monoculture is achievable, addressing it via prompt-level intervention remains elusive. The diversity extracted at the profile level isn't effectively transmitted during the prompting process. The reality is, the narrative compression occurring before the model even processes the prompts results in uniformity that doesn't reflect the intended diversity.

What's next for Nous and similar projects? The authors suggest moving beyond prompt-level interventions to techniques like fine-tuning or activation steering. Essentially, deeper integration rather than surface-level tweaks might hold the key. But here's a pointed question: Will these deeper methods actually address the core issue, or just layer more complexity on top?

The numbers tell a different story: attempts to instill diversity need a more solid approach than just prompt engineering. As the field grows, the architecture matters more than the parameter count. Whether Nous can pave the way for a more diverse cognitive landscape in AI remains to be seen, but one thing's clear: innovation in methodology is key.