Become a TechRadar Insider
Artificial intelligence has become far more than just a productivity aid – for many, it’s now a friend, mentor and therapist. Millions of users now interact with chatbots daily, with tools capable of remembering past conversations and adapting to personal context.
With that in mind, a growing debate is now emerging around whether AI is being designed to help users, or whether it’s actually being built to keep them engaged for as long as possible.
And it’s a concern that’s oh so familiar, with similar worries already raised about social media platforms. Infinite scrolling and algorithmic recommendations have already kept millions hooked, and now, critics argue AI companions may be doing exactly the same thing.
Recent reports have emerged linking AI companion use to emotional dependency, poor mental health and, in some of the rarest cases, more tragic outcomes.
Dr Ayelet Gordon-Tapiero from the Benin School of Computer Science and Engineering at the Hebrew University of Jerusalem believes that AI products should be treated just like any other consumer products – rather than seeing emotional dependency, addiction and manipulative behavior as unfortunate side effects, she suggests courts may eventually need to consider whether those outcomes are actually product defects, which could expose developers to legal liability.
But this begs the question of how much responsibility companies should bear for the emotional wellbeing of users, and how regulation could shape that responsibility.
To better understand these side effects, I spoke with Gordon-Tapiero about the psychology behind AI addition, how it compares to social media and the role of regulation.
- In one of our email exchanges, you agreed that AI friends could become as addictive as drugs. Could you expand that further?
The design of AI companions reflects the result of decades of research in the fields of Human-Computer-Interaction (HCI) and Human-Robot-Interaction (HRI).
Researchers in these fields spent years finding ways, through experimentation, to make computer systems more appealing, engaging and empathetic. These design choices have been successful in eliciting addictive responses from users.
In terms of the outcome of such addiction, unfortunately we are already seeing numerous cases of harms stemming from the use of AI companions – a feedback loop where users turn to AI companions to help them deal with anxiety and depression but end up exacerbating these conditions, children who spend hours and hours interacting with their AI companion instead of playing outdoors, interacting with friends or even sleeping.
Every few weeks we read about cases of children taking their lives after prolonged ‘relationships’ with AI companions. These concerns are likely to increase as AI companions gain popularity, particularly with children and other vulnerable individuals.
- It's not just the small players though. Internal documents from Microsoft reveal plans to "make people addicted" to its new AI assistant, so is it just the playbook, the usual MO of everything in our world that goes viral?
It is tempting to view AI companions as a new phenomenon, perhaps part of the larger trend of AI-doomerism. I would resist this framing.
Instead, I see a direct line connecting the purposely addictive nature of social media to the harms we are encountering in the use of AI companions. The same financial incentives seem to drive the design of both types of technology.
What is different is the type of interaction – whereas social media facilitated interactions with other users, AI companions create a technological bubble with only one human actor.
- All this is driven by cold capitalism though. The drive for efficiency, maximizing ARPU and usage. That's what gave rise to infinite scrolls and short form videos. That is not intrinsically bad. How can this sort of behaviour be discouraged without impacting the bottom line.
It is true that maximizing profits is not intrinsically bad. At the same time, as a society we do not believe that profit maximization is the only goal that should be promoted or protected.
In certain settings we may limit profit maximization to promote other interests – particularly where vulnerable individuals are concerned.
For example, we limit advertising and selling cigarettes to children even though such sales could drive revenue for tobacco companies.
We even have a common-law legal doctrine that is particularly aimed at dealing with cases in which we believe that the enrichment by a corporation comes at the expense of individuals and allows us, under certain circumstances, to disgorge the profits generated unjustly. The challenge is identifying when corporate practices cross the line of legitimate profit maximizing.
- I am sceptical about your call to regulate by litigation. Other than US, the rest of the world seems to be less inclined to use courts in that manner. Don't you think so?
Regulation by litigation is often overlooked as a regulatory tool. In the context of technology, it offers a nimbleness that command-and-control regulation simply cannot keep up with. Regulation by litigation is an approach used in many fields in common law countries, such as the United Kingdom.
One of the tools that common law litigation offers are broad doctrinal categories that are flexible and can be applied and adapted by courts in particular settings.
As I mentioned in the previous question, the doctrine of unjust enrichment is one such tool, and products liability is another one. For example, the question of whether software is a product was recently discussed in an American court litigating an AI companion case.
In Garcia v. Character Technologies, the Court ruled that AI companions should be viewed as a product, allowing plaintiffs to sue based on products liability. The question of whether software should be treated as a product, which is clearly defined in the EU through a directive, was shaped in the US by a court.
- Is it too late to change the way things are done? Is regulation useless as the market is moving so fast? What else can be done?
It is not too late, but the more we wait the harder it will be to make meaningful structural changes to technology. Regulation is not useless, but neither is it a panacea.
Thus, while regulation can and should set important boundaries, we should solely rely on it, nor should we wait for it idly. Instead, education is still the most important tool we have.
Teaching adults and children about the promise and perils of technology, AI and AI companions has the power to incentivize the most change in the companies developing these tools.
If we refuse to use technology that is harmful for us, if we demand technology that encourages deliberation and brings humans closer together, the benefits will extend beyond the bottom line of companies to individuals and society at large.
Follow TechRadar on Google News and add us as a preferred source to get our expert news, reviews, and opinion in your feeds.
