AI is now applied in all film-making procedures, from creating digital characters to analyzing movie viewers’ reactions.

Operation monitoring

A task often assumed by AI at Industrial Light & Magic studio is monitoring the manufacturing of SFX and graphic effects. “We often use AI to schedule graphic renders for the film every night”, said Rob Bredow, Senior Vice-President, Creative Director of the studio. Industrial Light & Magic had recently joined the crew of Star Wars: The Rise of Skywalker.

Video editing

Harry B. Miller, an expert in video editing had shared that AI is used in some particular tasks in editing programs. According to him, Avid’s Media Composer has a feature that allows comparison of movie dialogues in dailies and scripts, which enables editors to quickly detect videos of the same scene and speed up the editing process. Meanwhile, Adobe Premier Pro can use AI to adjust videos to different frame ratios.

These softwares, however, have their own limits. “The computer cannot understand jokes nor get humor. At the moment, Stanford University is researching a software which can film animated dailies, then merge them together according to the script. However, this software cannot identify which are the pivots of the films, and it all comes back to human,” said Miller.

Making special effects

Digital Domain Studio uses an AI system to create Masquerade faces for Thanos in Avengers: Infinity War. “AI research has had huge breakthroughs recently.  And AI-based graphic rendering in real time, along with real-time face building, is accelerating the time needed for creating charaters. Soon, we will have realistic scenes as if the real Thanos is in front of the camera”, said Darren Hendler, the studio’s Director of Digital Characters. He also added that, deep learning technologies hold great potential in helping film-makers to turn quickly transform substitutes into real actors, or even changing the dialogue on the screen.

Digital characters

Christopher Nichols, Head of Chaos Group Lab and a core member of a group that specializes in researching and developing digital characters, had shared that AI is creating a strong wave in the field. He expressed: “The large problem for artists and tools is finding the fine features that make digital characters realistic, and this can now be solved much easier with deep learning.”

Recent improvements in deep learning had also gave birth to Deepfake. “Many want this technology to be stopped and made illegal. Yet it has this huge potential that is begging for further discovery,” Nichols said. According to him, many people used to be afraid that Photoshop will be abused to make false images, but now it is basically commonplace, and Deepfake should be the same. He also thinks that people need to delve deeper into the field, and have better tools for detecting false images.


Movio, a company which specializes in marketing data research for movies, is using AI to analyze the audience’s reaction, as well as ticketing trends before movie releases. It has worked with large customers in the field, including the crew of Hobbs & Shaw, and Once Upon a Time in Hollywood. According to Will Palmer – Movie’s founder and CEO, AI helps the company to predict popular movies, provide useful information for distribution and marketing campaigns, and assist in creative decision-making. The most important thing, he believes, is that the AI constantly learns of the behavior of movie viewers, without making the problem overtly complex.

Making a whole movie

“From now till decades to go, AI algorithms can help make films just from scripts. Here, all commands and features are just binary codes on computer. Should we provide them with a large enough neural network, and train them on merging films with scripts, however, they will be able to make whole movies out of completely new scripts,” Paul Debevec, a Senior Scientist at Google said.

He predicts that human can even command computers to make completely new movies, or transform old movies to specific styles of different directors, like Kubrick, Tarantino, or Scorsese. And all they need, is finding the relevance between their films and scripts.

Source: VnExpress

Related posts: