Cracking Language Barriers: Basque Dataset Elevates Essay Scoring

In a groundbreaking move for low-resource language processing, a new dataset for Automatic Essay Scoring (AES) has been introduced, focusing on essays written in Basque. This isn’t just an ordinary dataset. It targets the CEFR C1 proficiency level, offering a significant leap for educational resources in non-dominant languages.

Raising the Bar with Basque

Comprising 3,200 essays, each meticulously annotated by seasoned evaluators, this dataset sets itself apart. It covers a spectrum of criteria, including correctness, richness, coherence, cohesion, and task alignment. The dataset not only scores these essays but also provides detailed feedback and error examples, essential for both learners and educators.

The use of this dataset transforms the capabilities of open-source models like RoBERTa-EusCrawl and Latxa 8B/70B. Fine-tuning these models has shown that they can surpass even the mighty closed-source systems like GPT-5 and Claude Sonnet 4.5 in both scoring consistency and feedback quality. Now, that’s a wake-up call for the giants relying on proprietary software!

Breaking New Ground with Latxa

Fine-tuning Latxa models proves to be a breakthrough in this context. Not only does it enhance performance, but it also ensures the feedback generated is criterion-aligned and pedagogically valuable. The big question is, can proprietary models even keep up with this level of educational significance?

An innovative evaluation methodology was also proposed, blending automatic consistency metrics with expert validation of learner errors. The results? The fine-tuned Latxa model doesn’t just match but surpasses proprietary models in identifying a broader array of errors. This is more than just a technical achievement. it’s a step forward for transparency and reproducibility in NLP research for low-resource languages.

Implications for the Future

The introduction of this dataset isn’t just about the essays or the models. It’s a statement that the intersection is real, and the opportunities for low-resource language processing are growing. Slapping a model on a GPU rental isn't a convergence thesis. Real progress happens when educational tools become accessible and meaningful in diverse linguistic contexts.

If the AI can hold a wallet, who writes the risk model? As we move forward, we must question the reliance on proprietary models that often overshadow open-source initiatives. This Basque dataset is a testament to the potential of transparent, reproducible research that can redefine educational NLP, especially for languages that have long been sidelined.