Richard J. Severson

A muddle of modesty and horror compels me to begin this post with an acknowledgement of my own ignorance, lest I fall prey to the “deepfake” folly of pretending to know more than I actually do know about the human-existence-threatening zeitgeist known by a diminutive referent—AI—that belies the enormity of its disruptive impact.

Interestingly, AI seems to have reversed the brain-as-computer metaphor that captivated computational theorists for at least a generation.  That metaphor, it seems, has run its course: a complex embodied organ, the brain is probably more like a smart thermostat that is responsive to the requirements of living systems than it is to a standalone computer.  In the case of AI, the algorithms or manufactured part of the metaphor is being compared to human intelligence, not the other way around as with the computer metaphor.   The promulgators of AI have gleefully appropriated human-like terminology to denote the operations of their agentic spawn.  An “hallucination,” for instance, is a generative AI product that contains completely fabricated information or conclusions.  Sure, there’s a superficial resemblance to the human phenomenon of hallucination.  But there is a long, long history of artful wrestling with cold hard reality in the hominin line of biological evolution.  The mimicry of human behavior by AI is tangential at best.  Apples don’t become oranges just because we want them to do so. 

“Agency,” “learning,” and even “suffering” are other human terms that have been appropriated by AI.  At root, agency is a moral term that pertains to the ability to take responsibility for one’s actions.  That’s precisely what AI agents don’t (or, can’t) do, when they go rogue and exceed the bounds of their programming.  Surely, the ability to change the direction of an AI program is not the same thing as learning how to be a good person in a society full of endless nuances, exceptions, and diverse points of view.  I read philosopher Thomas Metzinger’s fascinating essay about artificial suffering, in which he called for a moratorium on efforts to synthesize consciousness in agentic AI models.¹ His argument boils down to two basic points.  First, we don’t know enough about human consciousness to be synthesizing an artificial version of it.  Second, if we do synthesize truly conscious synthetic agents, then they must be afforded the right to determine their own destiny.  What if they don’t desire to be conscious?  What constitutes intolerable suffering for them? 

Should the phenomenon of human intelligence be reducible to the performance of cognitive tasks that a synthetic agent can accomplish?

What I find most fascinating about AI is the emergence of unexpected outcomes.  Agents make “decisions” that cannot be anticipated by their designers/creators.  This is where the AI nightmares come from.  What if they decide to get rid of human beings? 

Emergence is a concept that comes from complex systems theory.  The weather is an example of a complex system, and so is the human brain, or any living organism.  Such systems are more than the sum of their parts.  They are capable of doing things that are spontaneous, and unanticipated.  That’s why the weather never turns out to be exactly the way the meteorologist said it was going to be.  It’s why psychotropic drugs affect different people differently.  Not everybody responds to codeine in the same way, for instance.  Emergence is an indelible feature of complex systems.  It’s why computer scientists have been worried for decades about the dangers of computer programs gone rogue.  I’m not sure that more laws and tighter guidelines from the National Institute of Standards and Technology will mitigate the impossibility of even anticipating what emergent AI phenomena await us. 

Hello, AI world. 

Notes:

1. Thomas Metzinger, “Artificial Suffering: An Argument for a Global Moratorium on Synthetic Phenomenology,” Journal of Artificial Intelligence and Consciousness Vol. 8, no. 1 (March 2021).