Cracking the Indic Code: The Challenges of Multilingual Text-to-SQL

Large Language Models have undeniably pushed the boundaries of Text-to-SQL performance, but there's a catch. Most benchmarks stay rooted in Western contexts, often simplifying schemas that don't reflect the complexity of real-world applications. This is where IndicDB steps in, tackling these limitations by introducing a multilingual benchmark that evaluates cross-lingual semantic parsing across various Indic languages.

Why IndicDB?

IndicDB isn't just another benchmark. It's a tool for addressing the Western-centric bias in AI models. By sourcing relational schemas from platforms like the National Data and Analytics Platform and India Data Portal, IndicDB mirrors the intricate nature of real administrative data. It includes 20 databases spread across 237 tables, with a dense relational structure boasting up to six join depths.

These numbers aren't just for show. They represent a significant step towards creating benchmarks that truly reflect global diversity. The pipeline used to build IndicDB is value-aware, difficulty-calibrated, and join-enforced. This ensures that the 15,617 tasks generated are both challenging and realistic, spanning English, Hindi, and five other Indic languages.

The Indic Gap

Here's what the benchmarks actually show: State-of-the-art models like DeepSeek v3.2, MiniMax 2.7, LLaMA 3.3, and Qwen3 experience a 9% drop in performance when shifting from English to Indic languages. This 'Indic Gap' isn't just a number. It's a reflection of the deep-rooted challenges in schema linking, structural ambiguity, and limited external knowledge that these models face when dealing with non-Western languages.

Why does this matter? Because it highlights a significant bias in how models are trained and evaluated. If AI models can't handle the complexity of non-Western languages, are they truly ready for the global stage?

The Bigger Picture

Strip away the marketing and you get to the heart of the issue. The architecture matters more than the parameter count. It's not just about building bigger models. it's about building smarter ones that can adapt to diverse linguistic and cultural contexts.

IndicDB sets a new standard for multilingual Text-to-SQL benchmarks. It challenges the AI community to step up and address these inherent biases. The numbers tell a different story, and it's one that demands attention. As AI continues to evolve, the reality is that we need benchmarks like IndicDB to ensure we're not leaving large swathes of the global population behind.

The question remains: Are AI developers ready to embrace this challenge and create truly inclusive models?