AI Tutors: Smarter, But Not Yet Human

In a recent study on math tutoring, researchers examined how large language models (LLMs) stack up against human tutors in delivering educational support. The focus wasn't just on accuracy, but on whether these models could mimic the nuanced instructional behaviors of expert tutors. The paper, published in Japanese, reveals an intriguing gap between AI and human experts.

The Human Touch

Analyzing dialogues with students, the study compared responses from expert human tutors, novice tutors, and seven LLMs of varying sizes. Notably, the larger models showed significant promise, often approaching expert-level performance. But, here's what Western coverage has largely overlooked: the subtleties in teaching strategies that LLMs miss.

Experts excelled in qualities like restating student ideas and encouraging deeper reasoning. These aren't just metrics, but key components of effective teaching. The benchmark results speak for themselves. Yet, LLMs lag in employing these discursive strategies, though they produce responses that are longer, more polite, and lexically diverse.

The AI Advantage?

So, why does this matter? While LLMs are improving, they still lack essential human teaching nuances. Regression analyses highlighted that strategies such as pressing for accuracy and revoicing were positively associated with quality, whereas AI's tendency towards politeness and verbosity didn’t impress as much.

Crucially, this isn't just a technical exercise. If you're a parent, educator, or student relying on AI for educational support, you might wonder: Are these AI tutors competent enough? The data shows they're getting there, but they're not ready to fully replace seasoned human educators yet.

Room for Growth

The industry's push to develop AI tutors is fueled by potential cost savings and scalability. But, let's be clear, despite the progress, these AI systems should complement, not replace, human expertise. The underuse of effective teaching strategies is where AI needs improvement to truly revolutionize education.

Are AI tutors the future? Perhaps. The trajectory suggests they could redefine educational support. However, until they master the art of teaching, the human touch remains indispensable.