Tablet menu

How to make a stunning anime using Neural network if you can't draw?


The YouTube channel Corridor Crew has been researching how neural networks can help in the creation of anime for a year, and the authors have managed to achieve impressive results. The new method, if it doesn’t change the idea of ​​animation, will definitely help studios save a lot of money, and novice authors to create high-quality work without sky-high budget.

In the 1990s, anime director Yoshiaki Kawajiri shared an insider in an interview. The Ninja Manuscript has scene wasp swarm attacks: hundreds of insects attack the main characters. The team had to sweat a lot, drawing them - even the master himself painted several wasps with his own hand, distracting from directing and other more important tasks. The Ninja Manuscript was released in 1993 - and already in that year (but not a year earlier!) It became possible to propagate the notorious wasps on a computer by pressing ctrl + c and ctrl + v, and not torturing the entire team and the director with routine work. This is an exemplary story that clearly demonstrates the production processes in the anime. Progress is always one step ahead of the work of artists, but modern technological solutions often save teams energy and studios money, so it makes sense to chase after them. Thirty years ago, computer programs changed classical animation forever. Now neural networks are in a hurry to help them. How Stable Diffusion Neural Network Works

Stable Diffusion - neural network,which generates images from noise. Imagine spot from thousands of points. If you wish, you can see anything in it. If you configure the neural network to a certain style, then it will “see” in the noise exactly what you require. For example, a popular query for Stable Diffusion is − "John Wick Pixar Style" .

It would seem, why not make a whole film in a given style instead of one frame, but then a problem was discovered. The principle by which the neural network works is that it selects each image from a monstrous number of options, so all frames are slightly different. Arisesripples. It's not that bad, it even has its own charm. Here, for example, is a clip by Thomas Mraz, in which the described effect is purposefully played up:


How to bring to mind the material issued by the neural network

The special effects masters from Corridor Crew were not satisfied with this result and tried to bring the system to perfection, completely removing the flicker. To begin with, they recorded on video all the scenes from the upcoming anime against the backdrop of a green screen with themselves in the lead roles.

Then, instead of an abstract text query, the neural network was purposefully stuffed with pictures from Vampire Hunter Dee: Bloodlust, and then important frames with characters from the anime were correlated with photographs themselves. The robot immediately became easier to navigate, the flicker was not so obvious. At the post-processing stage, several filters were applied on top, for example, Remove Flickering White, and the ripples completely disappeared.

The resulting short film titled "Anime Rock Paper Scissors" was uploaded to YouTube. Accustomed to criticizing everything in the world, the public immediately scolded the novelty, ignoring the technical breakthrough, but accusing the Corridor specialists of leaving honest artists without bread and stealing the art style from the aforementioned Yoshiaki Kawajiri (he is the director of Ninja Manuscript and The Hunter on Vampire Dee: Bloodlust"). In "Corridor" they did not take a back seat and released a continuation of the anime. In the new series, they used the same method, only instead of the works of Kawajiri, they saturated the neural network with exclusive art, which was drawn by the artist on their order. Thanks to several new filters, the image has become even more accurate. Here is the final result:


Technology won?

So, if you have a cell phone with a camera, a green background and a computer with access to a neural network, then you can film yourself on video, then run the resulting material through the neural network, and you will get a not perfect, but quite competent likeness of a drawn anime. If you have a team of artists, or at least you can use Photoshop yourself, then the resulting material will become a blank for subsequent work, which can be manually brought to mind. In any case, this method is an order of magnitude less expensive than drawing everything from scratch. Making anime just got a little easier and cheaper.

Most of these articles usually end with the idea that fine art is all about telling compelling stories and carving out emotions. How exactly these two processes are carried out is not so important, and progress never stands still ...

Or is it worth it?

Here "Akira" Katsuhiro Otomo 1988 release, and here "Steamboy" by the same director, only released sixteen years later. The effort put into each frame is subconsciously felt - you somehow feel the greatness of the work done on Akira. In Steamboy, the characters emasculated in digital programs seem faded, and even three-dimensional models of equipment on top of 2D art are completely perceived as kitsch. Probably, animation technologies will someday be able to compete with human labor and talent, but so far this is very far away.

Last modified on
back to top