Presented by
I re-imagined these iconic film classics with Autodesk Flow Studio
If you’ve been following me on this journey of CGI exploration, welcome back. In my previous venture, I had a good look at how Autodesk’s Flow Studio changes VFX workflows, and used it to transform myself into various CGI characters - some of them were even the screen at the same time, even though the basic free tier doesn’t technically allow you to do that (there’s always a workaround).
All this was a lot of fun, and helped me understand what’s possible to do with this impressive online service. That’s right: you don’t have to download and install any software to use it - everything is done from your favourite web browser, which means it’s not reliant on your computer’s CPU. You can access it from any machine you happen to be in front of, and carry on working wherever you are.
But experimenting with Flow Studio got me thinking: why rely on my own footage? How would it handle clips taken from actual movies? So, I set out to find out. I dug up footage from cinematic classics, and re-AI-magined them, by replacing characters with CGI variants! Let’s see how well that went…
Legal and copyright issues
Now of course, I don’t want the offices to TechRadar Pro to get a visit from some not-so-friendly lawyers, so in order to avoid that, I’ll be looking at some really old feature films that have been out of around a century - if not more - and are therefore in the public domain, and out of copyright.
That in itself brought along some challenges I had not originally anticipated, and in so doing, my experiments helped me understand some limits to Flow Studio’s capabilities (because like any service, it has limitations). Now, let's get those servers warmed up, as we’ve got some rendering to do.
Le Voyage dans la Lune
Le Voyage dans la Lune, shot in France in 1902 by Georges Méliès, is widely regarded as the very first sci-fi movie ever made. It’s a classic of the silent era, filled with amazing effects for the time, and impressive creativity.
The process is exactly the same as with any other clip you’d care to use. The only difference is being able to source a copy of that iconic film, convert it into a digital file, and extract the few seconds you’re interested in working on. Although Flow Studio’s free tier can help you work wonders, you are restricted to only being able to work with 15 seconds of footage per month. It’s not much if you’re working on a complex project, but it’s enough to explore its capabilities and have a little fun.
Once you’ve uploaded the clip you wish to alter, you need to scroll through the footage until you’ve found the actor you wish to modify. You then click on ‘Scan frame for actors’. All this is standard fare in Flow Studio, and despite the clip’s age and quality, it still managed to isolate various characters, of which I selected one.
And that’s where I encountered an issue, which has to be due to the clip’s resolution and frame rate. I didn’t encounter this with all the personal clips I used previously, since they were all in either HD or 4K, playing at 30fps minimum. But footage that’s over a hundred years old isn’t any of these things, and sadly, Flow Studio struggled, and struggled hard.
The result was pretty surreal, which in a way was fitting for the movie in question, with the CGI character appearing at times, floating around in the most unnatural way, while the character it was supposed to replace appearing and disappearing at random intervals, sometimes next to his digital counterpart. This was definitely not a good first attempt.
But I was undeterred. I think the lesson here is to work with a good high-resolution copy of the footage you’d like to work with, and make sure the frame rate is at least 30fps.
Nosferatu
There are a lot of good long shots with slow moving characters in Nosferatu - especially the very creepy Count Orlock, so I decided to see what the platform could do with him.
Flow Studio comes with a handful of alien/monster CGI characters which you can use for free. Any of them could act as a suitable replacement for Orlock. There’s Bob, an odd fellow that looks like walking bacon in human form; Meera is some kind of underwater creature that wouldn’t look out of place as part of some Lovecraftian nightmare; and Sam, an innocuous name, for a harmless-looking pink alien… Sam feels like a good vampire alternative.
Once again, the process is no different to any other clip I’ve worked with in Flow Studio: create a new project, start by adding the clip to the left sidebar, then drag it from there onto the centre of the interface.
Use the timeline at the bottom to scroll through the footage until you find the character you’re after, then click on ‘Next’ and ‘Scan frame for actors’ for Flow Studio to analyse the scene and find the actors doing their stuff. Once Orlock has a rectangular highlight around him, drag Sam from the list of CGI characters on the right, onto said outline.
Move on to the export process, and wait a few minutes to appreciate the results (there’s no preview available - you have to trust Flow Studio to do its magic and use up your precious free seconds).
The output was so much better than my original disastrous attempt with Méliès’ masterpiece. The movement of Sam slowly walking through the doorway doesn’t feel that realistic - but then again we’re talking about a weird vampire that almost glided as he walked, so I decided to give that a pass.
The character blended very well with the original lighting, with Flow Studio subduing his colours to make him look darker - although there was a little glitch which I’m sure the clever engineers at Autodesk never considered: just like with Méliès' trip to the moon, our CGI character remained in full color.
This wasn’t a dealbreaker in the slightest as it’s easy to import the clip into a video editor and desaturate the result, but I thought it was worth mentioning. As it stands, our new horror icon merged very well into his century-old surroundings.
Metropolis
As a big fan of sci-fi, I’d be remiss if I didn’t try to tinker with Metropolis - one of the most well known early works of sci-fi, complete with its Terminator-like robot, released nearly 60 years before James Cameron ever put his idea to film.
Rather than try to replace the robot itself, which I thought would be too obvious, I decided to try and alter close ups and mid shots from a single scene, when Freder, the protagonist, meets Maria for the first time.
Looking at the roster of available CGI characters, I decided to use Nick as a suitable replacement for Freder, and for Maria, there’s a CGI character named Maria, so that decision was easy.
The results were not as good as Nosferatu, but acceptable. I ended up with glitches I’d encountered before, when an actor wasn’t immediately replaced by their CGI counterpart - it just took a few frames for it to do so. Annoying but easy to fix. This happened with the close up of Maria, except the altered clip started with CGI-Maria, then went back to the original Maria, before fully converting to CGI-Maria for the rest of the clip.
What was more disappointing was the Freder/Nick substitution. There were two issues with that one: Freder was clutching at his heart to show how struck with love he was at the sight of Maria (acting was a much simpler affair back in the day), and the other was the fact another woman was holding on to him.
Clutching at one’s heart proved unbelievably difficult for Flow Studio to handle, and the output ended up with both hands merging, rather than remaining distinct, with one on top of the other. Turns out AI still has issues with hands.
When it comes to the clinging woman, I ended up with a weird blurry, furry boundary between the two characters. I wasn’t expecting the platform to separate the two to the extent she would be seen holding on to the CGI character’s waist - that would’ve been beyond impressive. Instead, she now appears to hold on to him from behind. It’s not perfect, but acceptable.
That furry boundary was more of a problem. I wondered if this might be because CGI-Nick is a very thin character. How would Flow Studio handle the situation with a larger person? So I tried again, this time taking Bob off the bench.
This made the clinging woman almost disappear from the shot, but in all fairness, it actually looked better, with that furry boundary being completely gone. Bob still had issues with his hands - this time one actually disappeared into his own chest - was Flow Studio making him literally hold his heart? Maybe. So, despite Bob’s disturbing appearance, he actually yielded better results.
Post production
Flow Studio isn’t a video editor. In order to put the shots of Maria and Freder into one continuous clip, as it was in the original, I had to bring the exported footage into separate software.
I took the liberty of removing the handful of frames of actual-Maria, so only CGI-Maria remained within Metropolis, and after desaturating the lot, the result was a fun CGI-alteration of a classic movie. I did the same with my Nosferatu replacement, and desaturating him helped blend him into the scene so much better.
Overall, the results helped me understand the current limitation of this impressive service. What it does isn’t easy, and the fact the free tier lets Flow Studio work without human interaction is very impressive. And the paid tiers grant you access to changeable parameters to fine-tune the output further - you’re also able to export the data to dedicated apps like Blender or Maya to tweak them some more.
The only real major downside I could see was how prohibitively expensive it would be to do such conversations for an entire movie (especially for one like Metropolis, which is around 2.5 hours long).
Despite some glitches due to the extreme age of these classic movies, I thought the outcome was mostly pretty good, all things considered. Le Voyage dans la Lune was a failure, and Metropolis was acceptable, with Nosferatu yielding the best results.
It’s clear you can have a lot of fun with Flow Studio, whether you feed it your own clips, or classic footage.
