Bio-inspired Optimisation Surpasses Traditional Models in Neural Network Performance

Reservoir computing, a method that leverages the fixed dynamics of a recurrent network, has long depended on a trained linear readout for temporal processing. Traditionally, random reservoirs have served as the backbone of this approach. But what if millions of years of evolution have already encoded something more sophisticated in biological neural connectomes?

Unleashing Evolutionary Potential

Recent work applies four gradient-free, bio-inspired optimisers to the edge weights of biologically-based echo-state networks. These optimisers, Particle Swarm Optimisation, Differential Evolution, Grey Wolf Optimiser, and Whale Optimisation Algorithm, were tested across species ranging from the humble C. elegans with 279 neurons to humans with 83 parcels of structural MRI connectivity.

Each connectome was evaluated against four standard reservoir computing benchmarks: Memory Capacity (MC), Lorenz attractor prediction, NARMA-10 system identification, and Mackey-Glass chaotic time-series prediction. The key finding? All optimisers consistently outperformed their unoptimised biological counterparts, demonstrating that the evolutionary structure could be further enhanced.

Impressive Gains Across the Board

The Whale Optimisation Algorithm (WOA) stood out with the largest improvements, achieving up to a 17-fold increase in Memory Capacity for C. elegans and an impressive 89% reduction in Normalized Root Mean Square Error (NRMSE) for the Mackey-Glass task in humans. Across all species and tasks, an average improvement of 214% was observed. The ablation study reveals that initialising from biological weights, rather than random values, is important for these gains.

Such results beg the question: Are we underestimating the potential of bio-inspired methods to revolutionize neural network performance? While traditional methods rely heavily on random initialisation, these findings highlight biological weights as an essential inductive bias that random topology alone can't replicate.

The Road Ahead

This study builds on prior work from the reservoir computing domain by showing that biologically-initialised optimisation strategies aren't just competitive, they're superior across diverse neural complexities. The implications for AI and machine learning are significant. If the future of computing lies in mimicking biological processes, then the integration of bio-inspired, biologically-initialised methods could reshape our approach to developing more efficient and capable networks.

Code and data are available at the respective repositories, ensuring that these insights aren't only reproducible but also a foundation for further exploration. The question remains: How far can we push the boundaries of what's possible when we blend nature with technology?