MIT engineers have developed a wrist-mounted artificial intelligence that can detect a conversation's tone, paving the way for a "social coach" that could help people with anxiety or Asperger's.
The team, from MIT's Computer Science and Artificial Intelligence Laboratory and Institute of Medical Engineering and Science, used a Samsung Simband as a platform for a wearable deep-learning system that can predict if a conversation is happy, sad or neutral based on speech patterns and the person's vital signs.
“Imagine if, at the end of a conversation, you could rewind it and see the moments when the people around you felt the most anxious,” says graduate student Tuka Alhanai, who co-authored a paper on the subject. “Our work is a step in this direction, suggesting that we may not be that far away from a world where people can have an AI social coach right in their pocket.”
In Real Time
The prototype that they've built analyses audio, text transcriptions and physiological signals to measure the mood of a story with 83 percent accuracy. It can also provide a "sentiment score" for every five-second interval in a conversation.
“As far as we know, this is the first experiment that collects both physical data and speech data in a passive but robust way, even while subjects are having natural, unstructured interactions,” says Mohammad Ghassemi, who also worked on the project.
“Our results show that it’s possible to classify the emotional tone of conversations in real-time.”
Privacy Fears
All this might sound scary in an age of pervasive surveillance, but the team emphasises that the algorithm runs locally on the device, and that a consumer version would need clear protocols for getting consent from others involved in conversations.
“Our next step is to improve the algorithm’s emotional granularity so that it is more accurate at calling out boring, tense, and excited moments, rather than just labeling interactions as ‘positive’ or ‘negative,'” says Alhanai.
“Developing technology that can take the pulse of human emotions has the potential to dramatically improve how we communicate with each other.”
