Why AI Struggles to Tell a Good Story

AI's foray into creative writing has hit a snag. Despite the hype, large language models (LLMs) struggle to craft compelling stories. On the EQ-Bench, a leading benchmark for creative writing, AI judges sometimes rank their own zero-shot stories higher than New Yorker short stories. That's a problem.

The Missing Ingredient: Narrative Tension

So, what's the real issue? Current evaluation rubrics for story quality often overlook narrative tension, a important element that keeps human readers hooked. Enter the 100-Endings metric. This innovative approach analyzes a story sentence by sentence, predicting 100 different endings at each step based solely on the text so far. Tension gets measured by how often these predictions don't match the actual ending.

The numbers don't lie. Unlike standard rubric-based evaluations, the 100-Endings metric ranks New Yorker stories far above what AI generates. It's a stark reminder that AI still lacks the narrative nuance that makes human stories compelling.

A Roadmap for Better AI Storytelling

What's next? Researchers suggest a new story-generation pipeline that leans on structural constraints and narratological principles. They propose using story templates, idea formulation, and narrative scaffolding to boost narrative tension without sacrificing performance on the EQ-Bench leaderboard.

But let's be real. Can AI ever truly match the storytelling prowess of a human with a knack for narrative twists and turns? The gap between our expectations and AI's current capabilities is enormous. As much as we'd like to automate creativity, the results show we're far from that reality.

Why Should You Care?

Why does this matter? Because storytelling is more than just a sequence of events. It's about crafting an experience that resonates on a human level. If AI can learn to tell stories with genuine tension, it could revolutionize fields like entertainment and education. But until then, the magic of storytelling remains a uniquely human talent.