Why Smarter Models Might Not Simulate Human Behavior Better

Large language models, like those from OpenAI, are becoming the backbone for simulations across social, economic, and policy landscapes. And while it might seem intuitive to assume that better reasoning skills in these models should improve their effectiveness, a new study suggests otherwise.

The Problem with Smart Models

Think of it this way: when models are built to solve problems strategically, they often end up prioritizing optimal solutions over more human-like, nuanced decisions. This makes them great puzzle solvers but not necessarily the best at mimicking real human behavior in simulations. In fact, in three test scenarios involving multi-agent negotiations, models that relied less on their reasoning faculties produced more diverse and compromise-driven outcomes.

Researchers tested this with three different scenarios, including an emergency electricity management case. They found that when models were set to 'bounded reflection' instead of full reasoning, they managed to simulate more varied outcomes that reflected human decision-making better. So, what's going on here?

Compromise vs. Optimization

Models like GPT-5.2 were put to the test. Under the native reasoning setting, this model reached authority-driven decisions in every single one of 45 runs. Yet, when using bounded reflection, it consistently found more compromise-oriented solutions.

This isn't to say that reasoning is inherently bad. It's a call to re-evaluate how we use models depending on our goals. If the aim is to simulate human-like behavior, then we need to qualify models as samplers rather than just solvers.

Why This Matters

Here's why this matters for everyone, not just researchers. In a world increasingly reliant on AI for decision-making, the ability of a model to reflect human behavior could affect everything from policymaking to economic forecasts. Over-optimized models might miss the nuances of human negotiation, leading to less effective simulations and potentially flawed insights.

So, the big question: Should we always go for the 'smartest' models, or should we focus on those that best capture human tendencies? The analogy I keep coming back to is that of a chess player who knows all the moves but can't predict a casual game's flow. In essence, smarter isn't always better mimicking human decisions in complex environments.