Teachers Editing AI Feedback: A Look at How and Why

Large language models are stepping into classrooms, crafting feedback for students. Yet, what's fascinating is how teachers decide to tweak or leave this AI-generated feedback untouched. A recent study sifted through 1,349 pieces of AI feedback and found that teachers make changes in only 20% of the cases. So why is this worth a deeper look?

Exploring Teacher Edits

Think of it this way: If you've ever trained a model, you know that output is just one part of the equation. The study revealed that when teachers do edit AI feedback, they tend to trim down verbose explanations, opting for more concise, corrective guidance instead. It turns out, about half the teachers never touch the AI-generated suggestions, but a small 10% are frequent editors, revising over two-thirds of the time.

Predicting the Edits

Here's where it gets interesting. Using machine learning models to predict when a teacher will change feedback based solely on the AI text shows moderate success, with an AUC of 0.75. Essentially, while we can make an educated guess, it's far from a crystal ball. This highlights a potential gap between AI's understanding of what makes feedback effective and the human touch teachers bring.

Why It Matters

Here's why this matters for everyone, not just researchers. The analogy I keep coming back to is the recipe and the chef. AI can provide the ingredients, but teachers are the ones who know what works best for their students. This dynamic raises the question: Are our AI tools aligned with educational priorities? If teachers are mostly accepting the AI output as-is, maybe the AI is doing something right. Or maybe, just maybe, it's because the effort to edit isn't worth it.

Ultimately, this study highlights a significant opportunity. By understanding how and why teachers edit AI feedback, we can design systems that require less teacher intervention, optimizing both the AI's suggestions and educators' valuable time. Let's face it, if AI can better cater to teachers' needs, it can truly enhance the learning experience rather than just automate it.