An AI is turning paintings into realistic photos

(Image credit: Jun-Yan Zhu et al)

Since the birth of Google's DeepDream neural network two years ago, the ability of artificial intelligence to manipulate images has grown leaps and bounds.

Now, instead of turning normal photos into creepy, artistic images, a deep learning system called CycleGAN has been built with the ability to turn paintings into photorealistic images. Well, almost.

It's the creation of AI researchers at the University of California Berkeley, and as well as reconstructing photos from paintings it can also turn horses into zebras, winter into summer and more.

CycleGAN works by finding the commonalities among sets of images, then applying those commonalities to other images. It was trained with a selection of paintings by Monet and a selection of landscape photos from Flickr.

Lingering Gap

Algorithms were then applied to let the system make the photos appear as Monet paintings (which has been done before) and the Monet paintings appear as photos (which has not). They then applied the same system to other artist's work - Cezanne, van Gogh and Renoir.

It does quite well in tests where a painting is transferred to a photo and back again - only really losing a bit of pixel resolution. But it's not perfect - you can see some typical failure cases here

"We also observe a lingering gap between the results achievable with paired training data and those achieved by our unpaired method," the researchers wrote in the description of the project on GitHub. There's also a paper with more details available on the arXiv preprint server.

"In some cases, this gap may be very hard -- or even impossible -- to close."

Duncan Geere
Duncan Geere is TechRadar's science writer. Every day he finds the most interesting science news and explains why you should care. You can read more of his stories here, and you can find him on Twitter under the handle @duncangeere.