The “Daily Economic News” reporter summarized the six major advantages of Sora from the report:
(1) Accuracy and Diversity: Sora can transform short text descriptions into high-definition videos lasting up to one minute. It accurately interprets user-provided text inputs and generates high-quality video clips with various scenes and characters. It covers a wide range of topics, from people and animals to lush landscapes, urban scenes, gardens, and even underwater views of New York City, providing diverse content according to user requirements. According to Medium, Sora can accurately interpret long prompts of up to 135 words.
(2) Powerful Language Understanding: OpenAI utilizes the recaptioning technology of the DALL·E model to generate descriptive subtitles for visual training data, improving both the accuracy of the text and the overall quality of the video. Additionally, similar to DALL·E 3, OpenAI also utilizes GPT technology to transform short user prompts into longer, detailed transcriptions, which are then sent to the video model. This enables Sora to accurately generate high-quality videos according to user prompts.
(3) Video Generation from Images/Videos: In addition to converting text into videos, Sora can also accept other types of input prompts, such as existing images or videos. This allows Sora to perform a wide range of image and video editing tasks, such as creating perfect loop videos, converting static images into animations, or extending videos forwards or backwards. OpenAI demonstrated demo videos of image generation based on DALL·E 2 and DALL·E 3 in the report. This not only demonstrates the powerful capabilities of Sora but also showcases its unlimited potential in the field of image and video editing.
(4) Video Extension Functionality: Due to its ability to accept diverse input prompts, users can create videos based on images or supplement existing videos. As a Transformer-based diffusion model, Sora can also extend videos forwards or backwards along the timeline.
(5) Excellent Device Adaptability: Sora has excellent sampling capabilities, capable of handling any video size between widescreen 1920x1080p and portrait 1080x1920 effortlessly. This means Sora can generate content perfectly matched to the original aspect ratio of various devices. Additionally, before generating high-resolution content, Sora can quickly create content prototypes in smaller sizes.
(6) Consistency and Continuity of Scenes and Objects: Sora can generate videos with dynamic changes in perspective, making the movement of characters and scene elements in three-dimensional space appear more natural. Sora handles occlusion well. One issue with existing models is that when objects leave the field of view, they may lose track of them. However, by providing multi-frame predictions at once, Sora ensures that the main subject remains unchanged even when temporarily out of view.