Why Informal Learning Could Be the Key to Evolving LLMs

Large Language Models (LLMs) have been making headlines with their prowess in formal tasks like mathematical reasoning and code generation. But planning, creativity, and social intelligence, they're still playing catch-up. So, what's the next frontier for these titans of text? Informal learning may hold the answer.

The Game Plan

Inspired by how humans learn, through a blend of formal instruction and informal experience, researchers are introducing game-based environments into LLM training. Games like TicTacToe, Matrix Games, and Who's the Spy provide feedback-driven, annotation-free settings that can foster a wider range of abilities. Here, the AI can experiment, make mistakes, and learn dynamically.

But there's a snag when you try to mix these tasks under a single reinforcement learning (RL) objective: it muddies the learning signals. Task-gradient directions become a tangled mess, offering little guidance. That's why Coordinated Subtask Training (CST) has emerged as a potential big deal. It separates RL signals into sequential subtask-specific updates, ensuring that the AI can focus on different skills without interference.

Benchmarking Brilliance

In experiments using ability-oriented benchmarks, game-based informal learning improved generalization beyond what traditional formal training could achieve. CST not only preserved performance on in-domain tasks but also enhanced broader general abilities. The results suggest that informal learning environments might just be the catalyst for LLMs to break new ground.

So, what does this mean for the future of AI? For starters, it challenges the notion that formal tasks are the be-all and end-all of AI competency. Slapping a model on a GPU rental isn't a convergence thesis. If these LLMs are to navigate more complex social and creative landscapes, informal learning is an avenue worth exploring.

Why Should We Care?

Why should we care about LLMs playing games? The crux is adaptability. In a world where AI is increasingly interacting with humans, having machines that can mimic human thought processes is invaluable. Creativity, planning, and social intelligence aren't just buzzwords. they're essential for real-world applications.

If the AI can hold a wallet, who writes the risk model? The intersection is real. Ninety percent of the projects aren't. But the ones that are, could redefine how we perceive and interact with technology. So, will informal learning be the missing puzzle piece? Show me the inference costs. Then we'll talk.