Why AI Models Aren't Moral Agents and Never Will Be

Everyone loves to hype up AI. But before we crown large language models (LLMs) as the next moral judges, let's take a closer look. These models might spit out impressive and coherent text, but don't be fooled. The idea that they possess agency or qualify as moral agents is a misunderstanding of what these terms actually mean.

The Mirage of Intentionality

At their core, LLMs are statistical machines. They don't act out of intention or choice. they follow probabilistic input-output mappings learned from vast datasets. It's like a parrot repeating phrases, it doesn't understand, it just echoes. The 'intentionality' in LLMs is nothing but derived. It's a byproduct of clever programming and extensive data training. Not a reflection of true agency.

Consider this: would you say a calculator 'wants' to find the sum of two numbers? Of course not. It simply follows the rules it's been programmed with. Why should LLMs be any different? Their apparent decisions are pre-scripted by algorithms, not the result of conscious choice.

Commitment and Ownership: Missing in Action

One could argue that LLMs demonstrate some form of moral reasoning in their outputs. But here's the kicker: such reasoning isn't based on self-attributed action or intrinsic intentionality. These models don't own their words. They're akin to actors following a script without understanding the plot.

Without the ability to commit to an action or decision, claiming LLMs have moral agency is like saying a puppet controls itself. It's simply not possible. True moral responsibility demands a level of commitment and self-attribution that LLMs can't offer. Their responses are as random as a roll of the dice, introducing variability without genuine choice or authorship.

Don't Buy the Hype

Some argue that the presence of moral reasoning, as output by these models, suggests a semblance of agency. But let’s not get ahead of ourselves. Their 'reasoning' is no different from a well-crafted magic trick designed to deceive the eye. It’s impressive, but don’t mistake it for reality.

Ask yourself: would you trust a machine to make moral judgments affecting your life? The answer should be a resounding no. Machines can mimic, simulate, and predict, but they don’t possess the intrinsic qualities that underlie true moral agency.

Despite the AI gold rush, let’s stay grounded. LLMs might be the shiny new toy, but attributing moral responsibility to them is just another case of being bullish on hopium and bearish on math. The data already knows it, this ends badly if we don’t check our optimism at the door.