Why Small Models Can Outshine Giants in Automated Fact-Checking

automated fact-checking, the common perception is that bigger is better. Large language models (LLMs) have dominated the scene, but not everyone can afford them, especially organizations dealing with languages that don't have vast resources backing them.

The Problem with Large Models

LLMs are powerful, but they're also expensive. For many low-resource groups, these models are simply out of reach. This is where the new multilingual Citation Needed Detection (CND) corpus, named MCN, steps into the spotlight. Spanning 18 languages, MCN doesn't just pay lip service to these communities, it actively empowers them.

Here's where it gets practical. By using smaller, decoder-based language models (SLMs), MCN shows that you don't need a giant to do the heavy lifting in fact-checking. In fact, when fine-tuned with an encoder-style objective, these SLMs outperform their larger counterparts across different languages.

Cross-Lingual Success Story

The MCN corpus also shines a light on cross-lingual capabilities. SLMs trained only on English claims managed to surpass LLMs, even with minimal adaptation to the target language. That's a big deal. In production, this looks different because it means less time and fewer resources spent on adapting models for each individual language.

So why should we care? For one, this approach democratizes access to reliable fact-checking tools. It suggests that compact, task-specific models can be more effective for Citation Needed Detection than their large, unwieldy cousins.

What's at Stake?

But here's the catch: while the demo is impressive, the deployment story is messier. Getting these models into the hands of those who need them most isn't just a technical challenge. It's also about changing perceptions that bigger isn't necessarily better.

Isn't it time we question the almost blind faith in LLMs? With the release of all data and code on GitHub, the ball's in the community's court to adapt and implement these findings. The real test is always the edge cases, those tricky scenarios that weren't considered in initial designs.

So, while the tech world often swoons over the latest and greatest in LLMs, maybe it's time we give the smaller players a closer look. They might just have more to offer than we think.