LLMs in Education: Striking a Balance Between Aid and Ownership

Exploring how large language models (LLMs) impact university teaching and learning offers insights into integrating AI in a way that supports educational outcomes. A recent study involving 24 college students examined the effects of LLM assistance on writing performance, engagement, and perceived authorship.

The Study Setup

Participants were randomly assigned to three groups: no LLM access, limited access with up to three prompts and a 100-word cap on responses, and unlimited access. Surprisingly, the overall essay quality across these groups was statistically indistinguishable. Yet, the approach to writing and perceptions of authorship varied significantly.

Ownership and Creativity

Students with limited LLM access reported a higher sense of ownership over their work. About 62.5% said they'd submit the essay as independent work, starkly contrasting with only 25% in the unlimited access group. These students also noted stronger organizational gains and used the AI for more strategic, revision-focused prompting.

Conversely, those with unlimited access spent more time on their essays and produced work more closely resembling LLM outputs. This group also experienced diminished creative expression. One might ask, does unfettered AI access stifle students' creative capacities?

Implications for Educational Institutions

The key finding: constraining LLM access could preserve students' confidence in authorship while still offering the scaffolding benefits of AI. Universities need to consider these results when integrating AI tools in curricula. Is it time to redefine the role of AI in education as a supportive tool, rather than a crutch?

The study sheds light on a key balance educational institutions must strike. While AI can enhance learning, unrestrained access may dilute personal growth and creativity. The ablation study reveals that a limited, strategic use of LLMs fosters better educational outcomes. Code and data are available at the institution's repository for those interested in further exploration.