The Estonian AI Experiment: Can GPT-5 Outdo Humans?

This week in 60 seconds: Estonian researchers have dropped a fresh dataset to explore document-level subjectivity. They've got 1,000 documents, mixing 300 news articles with 700 web texts. The goal? Rate subjectivity on a 0 to 100 scale. Think of it like a Rotten Tomatoes score, but for objectivity.

Human vs. AI: The Showdown

Four human annotators took a stab at this, but the results were all over the map. Some documents had scores as split as a family at Thanksgiving dinner. So, they went back and re-annotated the contentious texts. Turns out, practice makes perfect. The scores aligned better the second time around.

But here's the kicker: they also threw GPT-5 into the ring. This large language model gave its own subjectivity scores. Surprisingly, GPT-5 didn't just flail around. Its scores lined up pretty closely with the humans. Yet, some gaps emerged, suggesting AI isn't ready to completely replace us. At least not yet.

Why This Matters

Why should you care? Because automatic subjectivity analysis could change the way we process language. It's a big deal for media, tech, and anyone who deals in shades of gray rather than black and white. But here's the rub: AI can't yet capture the subtle nuances of human judgment. So, while it's a neat tool, it's not a standalone solution.

The one thing to remember from this week: AI's potential in subjectivity analysis is tantalizing but still not foolproof. The decision on when to rely on AI versus human insight hinges on the task at hand. Are we ready to let AI decide what's subjective? Or do we still trust humans to call it?

That's the week. See you Monday.