This software could soon be captioning your drunk Facebook photos

'Vomiting aggressively while grandma looks upset. Also, there's a dog'

Facebook

Two researchers at Idiap Research Institute, Switzerland, have developed an algorithm that can automatically caption photos.

They're not the first to come up with software capable of doing this, but what makes this software superior to existing auto-captioning techniques is that it's less predictive.

"Those other systems propose the first word based on the photo and then use that word to predict subsequent ones," said Pedro Oliveira Pinheiro, one of the project researchers.

Microsoft's own deep learning solution predicts after learning the first word. The Idiap solution instead creates vector representations of images and compares them to a range of possible words.

The whole image is broken into elements - nouns, verbs, prepositions - which are then put together to create the caption.

The technique is quite simple by comparison to others, and it's already got Facebook's attention. The researchers behind the project did an internship at the Zuckerberg's company, and Facebook is said to be drawing on this technology for its auto-captioning artificial intelligence, some of which will be dedicated to helping the visually impaired.

Article continues below