LLMs: Lost in Translation of Jurisdictions?

In a world where language models (LLMs) tackle everything from healthcare to taxes, an interesting pattern emerges. Language can skew the legal context these models use, often leading to jurisdictional mismatches. Particularly in Asia, where multilingual users abound, this raises critical questions about the accuracy and relevance of AI-provided advice.

The Language-Jurisdiction Dilemma

Recent audits of LLMs developed in the U.S. and China, using 60 legal prompts in both English and Mandarin, reveal a significant pattern. English prompts typically result in U.S.-based answers, while Mandarin prompts often produce China-centric responses. Specifically, 74.5% of English responses adopt a U.S. framework and 53.3% of Mandarin responses default to a Chinese framework. This is concerning because Asia moves first in adopting multilingual AI tools.

Why This Matters

Imagine a user in Tokyo asking about local labor laws but receiving advice based on U.S. regulations simply because they used English. Or a multilingual user in Hong Kong getting China-specific guidance due to Mandarin input. The licensing race in Hong Kong is accelerating, and clear jurisdictional advice is important for regulatory clarity. When location is absent, LLMs should request it or clarify the jurisdictional scope of their answers. Failure to do so could result in costly legal missteps for users who rely on these models for decision-making.

The Call for Change

So, what needs to happen? LLM interfaces shouldn't route institutional advice based on input language alone. They should prompt for location or make their jurisdictional assumptions clear. This wouldn't only enhance accuracy but also foster trust in these AI systems. After all, what's the point of sophisticated AI if it can't deliver the right advice where it's needed most?