Meta is pushing the boundaries of artificial intelligence with the introduction of Meta Movie Gen, its latest model for media-based AI applications. Movie Gen enables the creation of videos, sounds and personalized multimedia experiences from simple text input. These capabilities position Meta Movie Gen as a tool that offers many different content creation functions, rather than being a pure text-to-video model.
🎥 Today we’re premiering Meta Movie Gen: the most advanced media foundation models to-date.
— AI at Meta (@AIatMeta) October 4, 2024
Developed by AI research teams at Meta, Movie Gen delivers state-of-the-art results across a range of capabilities. We’re excited for the potential of this line of research to usher in… pic.twitter.com/NDOnyKOOyq
The core of Movie Gen
Movie Gen was developed to create high-quality, high-definition videos from text. It allows users to create videos from scratch or modify existing material with just a few inputs. This new model offers a variety of features that take creative possibilities to a new level, including video editing, personalized video creation, and synchronized soundtracks.
text-to-video generation
At the core of Movie Gen is the 30B Parameter Transformer, a powerful tool that generates long, high-resolution videos with custom aspect ratios from text descriptions. This represents a significant advance over current industry standards, which often struggle with the length or quality of video. From conceptual animations to realistic short films.
Video editing with text
Another feature of Movie Gen is the ability to edit videos via text. Users can enter precise instructions to edit videos at different levels – from removing or adding elements to more comprehensive changes such as changing the background or applying style effects. These functions allow for the fine-tuning of video elements, all of which are controlled by simple text entries.
personalized video creation
The model also excels at creating personalized videos. By uploading their own image, users can create videos that reflect their identity and movements. Movie Gen is ideal for personal projects, avatars or interactive content. The advanced motion capture capabilities of the AI ensure that these personalized videos have a natural flow and look very realistic.
Generation of soundtracks and sound effects
Movie Gen’s audio model (with 13B parameters) also delivers very good results here. By combining video and optional text instructions, the user can create high-quality sound that perfectly matches their video. The model generates a variety of sound effects, from ambient sound and instrumental background music to Foley sound, ensuring that the soundtrack not only fits but also enhances the atmosphere of the video.
The potential of Meta Movie Gen
Developed by Meta’s AI research teams, Movie Gen could have a huge impact on a wide range of industries, particularly media, entertainment and marketing
While still in the research phase, the team is working closely with creative professionals to incorporate feedback and expand functionality before a broader release. Meta is already providing insights into future developments – it’s clear that Movie Gen could be a real game-changer for digital storytelling, offering creatives a whole new way to bring their ideas to life with the power of AI.
Source: https://x.com/aiatmeta/status/1842188252541043075?s=12
