How Lifetime Learning Transforms Chess Agents Over Time

The question of whether lifetime learning can truly expand behavioral diversity over evolutionary periods, rather than stifling it, has long intrigued researchers. Conventional wisdom suggests that plasticity might compress diversity by buffering against environmental variability. However, new findings competitive chess agents suggest otherwise.

Chess Agents and Hebbian Learning

In a fascinating study, chess agents were equipped with eight neural modules developed through NeuroEvolution of Augmenting Topologies (NEAT) and integrated with Hebbian learning principles. This setup enabled the agents to adapt within games via a process akin to imagination. Across ten experimental seeds per Hebbian condition, a striking pattern emerged. Initially, agents with Hebbian learning exhibited lower cross-seed variance compared to their non-Hebbian counterparts. Yet, by the 34th generation, this variance not only increased but surpassed that of the non-Hebbian agents.

What does this tell us about Hebbian learning's role in behavioral variance? The evidence points to a reversal over evolutionary time. Initially compressing diversity, plasticity eventually amplifies differences, driven by evolving perception and imagination feedback loops. This transformation results in a structured behavioral divergence among agents.

Structured Behavioral Divergence

The study observed that evolved chess agents began to diverge significantly in their decision-making processes. A striking 62% disagreement arose over move selection from identical board positions. Moreover, distinct opening repertoires, piece preferences, and game lengths emerged. These differences weren't mere sampling variations. Rather, they represented reproducible behavioral signatures with interpretable configurations.

Consider this: in a world dominated by AI systems that often converge towards uniform strategies, could fostering diversity in agent behavior be the key to overcoming stagnation in AI development?

Three Regimes and Their Implications

The research identified three distinct regimes influenced by the type of opponent: exploration (Hebbian ON with a heterogeneous opponent), lottery (Hebbian OFF with elitism lock-in), and transparent (same-model opponent leading to brain self-erasure). Notably, the transparent regime offers a falsifiable prediction. It suggests self-play systems might suppress behavioral diversity by eliminating the heterogeneity important for personality development.

Before discussing advancements, it's vital to address the long-term implications of these findings. Could self-play systems inadvertently curtail the very diversity that could propel AI towards more human-like intelligence? Fiduciary obligations demand more than conviction. They demand process. As such, this new understanding of plasticity and diversity calls for a reevaluation of how we develop and deploy AI agents in competitive domains.