Artificial intelligence (AI) generative tools such as text-to-video and text-to-3D are about to crash headfirst into the creative industries. While many are skeptical that these will overtake the likes of Disney or Pixar anytime soon, there are already examples of AI tools enabling independent creatives to produce astoundingly high-quality animations at a fraction of the time and cost of blockbuster studios.
Take the YouTube show The Corridor Crew, for example. The channel, which is run by an independent production studio that specializes in visual effects, released a video in February titled ‘Did we just change animation forever’ that revealed an animation technique for turning live action into animation.
The video provides a behind the scenes look at how the team created an automated animation process by using a technique found in software such as Stable Diffusion, an AI generator similar to DALL-E that is owned by the startup Stability AI. The technique solves the issue of visual noise (a blurry or flickering motion around frames) to produce a smooth sequence.
They shot a video on their phones against a green screen, and uploaded it to a model trained on a particular anime style (in this case it was Vampire Hunter D: Bloodlust). Instead of having to reduce noise by painstakingly going through each frame, the team were able to auto-generate this using a ‘deflicker’ effect in one of the best free video editing software tools, DaVinci Resolve.
The result is a slick animated short film that was produced by an AI-powered open-source program at a fraction of the cost and time that would be required to make Hollywood-style content. Of course, the video isn’t as polished as some of the best animation series out there, but it shows a far more smoother and more complex result compared to most home productions.
Imagine what even a small, dedicated team could make with tools like this – creating animated series with a huge sense of scale and expression that could never have been achieved without the budget of one of the best streaming services in the past.
While the team had put forward the project as a way to showcase a technique that smaller studios or enthusiasts could use to reduce the overwhelming amount of work needed to make a full-length animation film, industry professionals are outraged.
The director of Netflix’s Castlevania series Samuel Deats, for example, took to Twitter to disagree with Corridor Crew’s notion that AI could “democratize” the animation industry. The backlash is part of a growing debate over AI’s impact on copyright, intellectual property and jobs.
The ethical concerns of AI in animation
The animation industry has come a long way from using hand drawn images, with processes such as computer-based rotoscoping (tracking over live-action footage) and motion capture (recording the movement of real-life objects or people having 3D models mimic the movement) since becoming common animation forms.
However, while AI stands to be the next big leap in technology for animators, the industry sees it as controversial. When Netflix’s Anime Creators’ Base in Japan used AI-generated images to create the background artwork in an anime short film titled Dog and Boy earlier this year, it prompted calls of outrage. The streaming studio said in a Tweet that it released the experimental project in an effort to help the anime industry’s labor shortage. However, Motherboard reports that many took to Twitter to accuse the platform of simply avoiding costs.
Animation is a notoriously time-consuming and expensive process. Loving Vincent, for example, was created by 125 professional artists who studied Vincent Van Gogh’s paintings to produce 65,000 oil painted frames. The artistic merit required to create such productions are down to the creative minds behind them, argue animators such as Guillermo del Toro and Hayao Miyaziki (pictured below). Del Toro told Decider in an interview that he agreed with Miyaziki’s opinion that AI is “an insult to life itself” because it cuts artists out of the process.
As AI’s potential in animation production continues to be explored though, there are some that argue there are still limitations, especially at the major studio production level. Vox released a video in April exploring whether AI could kill the green screen. It determines that while green screens have limitations, they provide a certain level of background separation benefits that can’t yet be replicated by AI.
However, for independent animators, AI generative tools offer a way to explore the art form that makes it accessible to a larger group of creatives. Essentially, what the team at Corridor Crew showcased in their video titled ‘Anime rock, paper, scissors’ was an advanced form of rotoscoping. Some of the best Netflix anime and best Disney movies are known for using this technique (think Richard Linklater’s Apollo 10 ½ and the 1937 film of Snow White and the Seven Dwarfs).
While there are many challenges ahead, the recent acceleration in generative AI is nothing short of mind blowing. As the technology becomes more sophisticated and accessible, the automation of major CGI and VFX software processes could help create even more feature length animations by independent creatives on video sharing platforms like YouTube.
The rise of amateur animation productions won’t likely compete with Disney, Pixar or Netflix. Instead, it could be more akin to what we’ve seen in self-publishing or Kickstarter projects, which have grown off the back of niche audiences as fans support the projects they like. The potential here isn't that Pixar-level animation starts costing pennies to make, but that some small series from a couple of people with a creative vision gets to exist at all. And that could be exactly where your next obsession comes from.
We mentioned at the top that the image we used to lead this story was created by AI to match the theme of the story. Without that tool, we would have had to use something like a stock photo of something working at a desk – but AI gave us the option to do something more engaging within the resources and time we had. If it can unlock the same options for creative minds we could see great things in the future.
