Text-to-video AI blossoms with new metamorphic video capabilities

While text-to-video artificial intelligence models like OpenAI's Sora are rapidly metamorphosing in front of our eyes, they have struggled to produce metamorphic videos. Simulating a tree sprouting or a flower blooming is harder for AI systems than generating other types of videos because it requires the knowledge of the physical world and can vary widely.

But now, these models have taken an evolutionary step.

Computer scientists at the University of Rochester, Peking University, University of California, Santa Cruz, and National University of Singapore developed a new AI text-to-video model that learns real-world physics knowledge from time-lapse videos. The team outlines their model, MagicTime, in a paper published in IEEE Transactions on Pattern Analysis and Machine Intelligence.

"Artificial intelligence has been developed to try to understand the real world and to simulate the activities and events ...