Why Local Language Might Be Key to Unlocking AI's Cultural Knowledge

AI's growing role in our lives isn't just about its technical capabilities, it's about how these machines engage with the world around us. A fascinating study shows that large language models, used for culturally grounded questions across various languages, may function better in local tongues than in English, despite what raw accuracy suggests.

The English Proficiency Problem

Here's the deal. Across 13 locales and roughly 80 models, English showed a clear edge when tackling culture-agnostic questions. It's not that English is the ultimate medium for cultural insights, but rather, English proficiency shines through. However, raw accuracy often conflates this proficiency with genuine cultural understanding.

Think about it. If a language model can answer a question accurately in English, does that mean it truly grasps the cultural nuances? Or is it just flexing its English muscles? The question isn't just academic. It affects how we deploy these technologies globally. The gap between the keynote and the cubicle is enormous language capabilities.

Local Languages: An Untapped Advantage

Once you factor out the English proficiency, the picture changes. Local languages show a clear advantage in accessing cultural knowledge. It's like peeling back layers to reveal what these models can really do. It's a bit of a plot twist, one that suggests the future of AI might be more multilingual than we thought. But how do we convince companies to invest in local language models when English seems to be the go-to?

The results are masked by limited proficiency in local languages. But when models are regionally aligned or language-adapted, they shine. So, why aren't more companies leaning into this? Management bought the licenses. Nobody told the team.

A Call to Rethink AI Evaluations

AI's cultural competence isn't just a technical question, it's about reshaping how we think about technology in a global context. Companies need to move beyond raw accuracy as the ultimate measure. It's time to consider localized knowledge access. What would happen if we took this more seriously?

Imagine the productivity boost if AI could navigate cultural questions in the local language, without being tripped up by English proficiency. It'd be a big deal for companies expanding internationally. I talked to the people who actually use these tools. Their frustration is palpable when models miss the mark due to language barriers.

So, let's rethink how we evaluate AI models. The real story is that local knowledge is often hidden behind a language wall. It's time to scale it.