At this year’s annual re:Mars conference, Amazon revealed perhaps the creepiest tech I’ve ever heard: an Alexa skill that can have it mimic other people’s voices.
On the surface, it doesn’t seem that bad right? The nerd in me is already thinking about the ways I could feel like Tony Stark or Luke Skywalker with my own personal Paul Bettany or Anthony Daniels-voiced Alexa speaker (the voices of the MCU's J.A.R.V.I.S. and Star Wars' C3-P0 respectively).
But rather than going down this celebrity route, Amazon’s Alexa senior vice-president Rohit Prasad presented the idea alongside a clip of a child being read to by Alexa using the voice of their recently-deceased grandparent. According to Engadget, an Amazon spokesperson explained this clip could be recreated using “as little as one minute of audio” of the person Alexa is mimicking.
The scene was likely meant to inspire heartwarming feelings as you see how Amazon’s tech helps a child process their grief, but it could just as easily have been a scene from Black Mirror. In fact, the episode Be Right Back has a pretty much identical premise to this.
Be Right Back follows Domhnall Gleeson's Ash, who is killed in a car accident, and, after discovering she is pregnant, Hayley Atwell's Martha needs to try new technology which allows her to communicate with an artificial intelligence imitating Ash. It's terrifying and definitely not meant to serve as a preview of anything good.
In his explanation of the Alexa tool, Rohit Prasad said that while it wouldn’t eliminate the pain of loss “it can definitely make memories last” and could help ease any heartache.
This is a sentiment that I can definitely understand; grief is a difficult emotion to process, especially at a young age like the child in the clip Amazon shared. But blurring the lines between life and death doesn’t seem like the healthiest way to deal with loss.
Based on Prasad’s comment it’s clear Amazon sees this as an evolution of reminiscing about loved ones using old photos and videos recorded before they passed away, but this isn’t the same. A photo or a video was taken with their consent and is of something the person actually did before they died; this AI-led feature isn’t playing a recording of a book reading, it’s using their voice to make a fabricated memory.
If someone wants their voice to live on through Alexa after they die then power to them, I’m sure they’ll love this feature. Personally, I would want to be left to rest in peace.
More than just digital-undeath
Outside of the nightmare-inducing uses above, this copycat feature could also pave the way for new scams. While many of us know to ignore and hang up on robotic voices on the other end of the line, we might be more inclined to listen to the pleas for money from our grandmother or son.
Coupled with sophisticated video deepfaking tools, we could eventually find it impossible to believe any video we watch – anything could have been constructed from realistic fake audio and visuals.
Amazon won’t be the only company developing AI voice tools like this mind. We suspect Apple, Google, and every other voice assistant maker is looking to make them sound more realistic and personal. But these sophisticated mimicking features are very much a pandora’s box, something to be opened with care.
To that end, it does seem like Amazon is aware of the need to be cautious. It hasn’t outlined a timeline of when it will get this feature out to existing Alexa devices, and it hasn’t even confirmed if it will even be publicly available.
But now that its audio immortality has been revealed to the world, it’s likely only a matter of time before Alexa gains the ability to speak as the dead.
