The Surprising Social Skills of Large Language Models

In the evolving world of AI, large language models (LLMs) are increasingly taking the reins in email communication. A recent study involving over 600 emails, crafted by both humans and LLMs, puts these models under the microscope to see just how well they handle the nuances of social goals in workplace scenarios.

Communication Games: A New Frontier

Enter the HR Simulator, a game designed to dig into the mechanics of communication. Players assume the role of a Human Resources officer, drafting emails to navigate tricky workplace situations. Here's where it gets interesting: LLMs, acting as judges, often rated their peers, other LLMs, as superior to human-written emails. We're talking about LLMs scoring a 48-54% success rate versus humans at a mere 23.5%.

But don't write off humans just yet. When humans team up with LLMs, the success rate skyrockets, hitting nearly 100% in some scenarios. That's a staggering leap, suggesting that the sweet spot might be a collaboration rather than competition.

Understanding Tact and Tone

So, what's behind the divergence in email styles between humans and LLMs? It turns out, the answer lies in tact. Weaker models prefer blunt strategies, while their stronger counterparts lean towards more tactful approaches. It's kind of like training wheels for social niceties, models learn to ride smoothly with experience.

As for tone, LLMs tend towards formality and empathy, a stark contrast to the varied styles of human writers. Sure, LLMs can rewrite human emails to sound more polished, but they still can't quite capture the raw, informal essence humans sometimes bring to the table. It's a limitation that highlights the current boundaries of AI's post-training finesse.

Why It Matters

Think of it this way: if you've ever trained a model, you know the thrill of watching it improve with each iteration. This study suggests something bigger at play. Communication games like HR Simulator aren't just fun. they might be the tool we need to truly measure AI's communication prowess. And here's why this matters for everyone, not just researchers: effective email communication is key in today's digital workspaces. If we can harness the best of both human intuition and LLM efficiency, we're looking at a future of enhanced communication.

So, is this the dawn of AI-powered communication, or just another tech trend that’ll fade away? Given the numbers and the potential, my bet is on the former. But the real question is, are we ready to embrace a future where our emails might be co-authored by an AI partner?