Text-to-video AI blossoms with new metamorphic video capabilities

Using time-lapse videos as training data, computer scientists have developed video generators that simulate the physical world more accurately.

2025-05-05

(Press-News.org) While text-to-video artificial intelligence models like OpenAI’s Sora are rapidly metamorphosing in front of our eyes, they have struggled to produce metamorphic videos. Simulating a tree sprouting or a flower blooming is harder for AI systems than generating other types of videos because it requires the knowledge of the physical world and can vary widely.

But now, these models have taken an evolutionary step.

Computer scientists at the University of Rochester, Peking University, University of California, Santa Cruz, and National University of Singapore developed a new AI text-to-video model that learns real-world physics knowledge from time-lapse videos. The team outlines their model, MagicTime, in a paper published in IEEE Transactions on Pattern Analysis and Machine Intelligence.

“Artificial intelligence has been developed to try to understand the real world and to simulate the activities and events that take place,” says Jinfa Huang, a PhD student supervised by Professor Jiebo Luo from Rochester’s Department of Computer Science, both of whom are among the paper’s authors. “MagicTime is a step toward AI that can better simulate the physical, chemical, biological, or social properties of the world around us.”

Previous models generated videos that typically have limited motion and poor variations. To train AI models to more effectively mimic metamorphic processes, the researchers developed a high-quality dataset of more than 2,000 time-lapse videos with detailed captions.

Currently, the open-source U-Net version of MagicTime generates two-second, 512 -by- 512-pixel clips (at 8 frames per second), and an accompanying diffusion-transformer architecture extends this to ten-second clips. The model can be used to simulate not only biological metamorphosis but also buildings undergoing construction or bread baking in the oven.

But while the videos generated are visually interesting and the demo can be fun to play with, the researchers view this as an important step toward more sophisticated models that could provide important tools for scientists.

“Our hope is that someday, for example, biologists could use generative video to speed up preliminary exploration of ideas,” says Huang. “While physical experiments remain indispensable for final verification, accurate simulations can shorten iteration cycles and reduce the number of live trials needed.”

END

ELSE PRESS RELEASES FROM THIS DATE:

Using age, sex, and race-specific standards could reclassify many thyroid disease diagnoses

2025-05-05

Embargoed for release until 5:00 p.m. ET on Monday 5 May 2025 Follow @Annalsofim on X, Facebook, Instagram, threads, and Linkedin Below please find summaries of new articles that will be published in the next issue of Annals of Internal Medicine. The summaries are not intended to substitute for the full articles as a source of information. This information is under strict embargo and by taking it into possession, media representatives are committing to the terms of the embargo not only on their own behalf, but also on ...

A Big Data approach for battery electrolytes

2025-05-05

Discovering new, powerful electrolytes is one of the major bottlenecks for designing next-generation batteries for electric vehicles, phones, laptops and grid-scale energy storage. The most stable electrolytes are not always the most conductive. The most efficient batteries are not always the most stable. And so on. “The electrodes have to satisfy very different properties at the same time. They always conflict with each other,” said Ritesh Kumar, an Eric and Wendy Schimdt AI in Science Postdoctoral Fellow working in the Amanchukwu Lab at the University of Chicago Pritzker School of ...

Moffitt study finds structural barriers may prevent cancer care for people living with HIV

2025-05-05

TAMPA, Fla. (May 5, 2025) — People living with HIV are less likely to receive potentially lifesaving cancer treatment if they live in communities with lower income levels and educational attainment, according to a new national study led by researchers from Moffitt Cancer Center. In the study, published in Cancer, researchers looked at cancer treatment records for more than 31,000 adults with HIV who were diagnosed with one of 14 common cancers between 2004 and 2020. They found that 16.5% of them did not receive the recommended first line curative treatment for ...

Min proteins for max efficiency during cell division

2025-05-05

The Min protein system prevents abnormal cell division in bacteria by forming oscillating patterns between the ends of a cell (“poles”). Despite decades of theoretical work, predicting the protein concentrations at which oscillations start and whether cells can maintain them under different conditions has been a challenge. Understanding these thresholds is important because they reveal how efficient this self-organizing system is in guiding division to the right place. UC San Diego researchers have engineered ...

How tiny particles coordinate energy transfer inside cells uncovered

2025-05-05

Protons are the basis of bioenergetics. We know them, in our everyday life, from the pH values we see on various soaps and lotions. But the ability to move them through biological systems is essential for life. A new study shows for the first time that proton transfer is directly influenced by the spin of electrons, when measured in chiral biological environments such as proteins. In other words, proton movement in living systems is not purely chemical; it is also a quantum process involving electron spin and molecular chirality. The quantum process directly affects ...

Gorilla study reveals complex pros and cons of friendship

2025-05-05

Friendship comes with complex pros and cons – possibly explaining why some individuals are less sociable, according to a new study of gorillas. Scientists examined over 20 years of data on 164 wild mountain gorillas, to see how their social lives affected their health. Costs and benefits changed depending on the size of gorilla groups, and differed for males and females. For example, friendly females in small groups didn’t get ill very often but had fewer offspring – while those in large groups got ill more but had higher birth rates. Meanwhile, males with strong social bonds tended to get ill more – ...

Ancient Andes society used hallucinogens to strengthen social order

2025-05-05

Two thousand years before the Inca empire dominated the Andes, a lesser-known society known as the Chavín Phenomenon shared common art, architecture, and materials throughout modern-day Peru. Through agricultural innovations, craft production, and trade, Chavín shaped a growing social order and laid the foundations for hierarchical society among the high peaks. But one of their most powerful tools wasn’t farming. It was access to altered states of consciousness. That’s according to a new study that uncovered the earliest-known direct evidence of the use of psychoactive plants in the Peruvian Andes. A team ...

Biological ‘clocks’ key to muscle health and accelerated ageing in shift workers

2025-05-05

Muscle cells contain their own circadian clocks and disrupting them with shift work can have a profound impact on ageing, according to new research. A study published today in Proceedings of the National Academy of Sciences (PNAS) contributes to the growing evidence of the damage shift work has on health. The King’s College London team revealed how muscle cells have an intrinsic timekeeping mechanism that regulates protein turnover, modulating muscle growth and function. At night, the muscle clock activates the breakdown of defective proteins, replenishing muscles while the body rests. Altering this intrinsic ...

Physical cloaking works like a disappearing act for structural defects

2025-05-05

Whether designing a window in an airliner or a cable conduit for an engine, manufacturers devote a lot of effort to reinforcing openings for structural integrity. But the reinforcement is rarely perfect and often creates structural weaknesses elsewhere. Now, engineers at Princeton and Georgia Institute of Technology have developed a technique that can maintain structural integrity by essentially hiding the opening from the surrounding forces. Rather than reinforcing the opening to protect against a few select forces, the new approach reorganizes nearly any set of forces that could affect the surrounding material to avoid the opening. In a May 5 article in the ...

New molecular label could lead to simpler, faster tuberculosis tests

2025-05-05

CAMBRIDGE, MA -- Tuberculosis, the world’s deadliest infectious disease, is estimated to infect around 10 million people each year, and kills more than 1 million annually. Once established in the lungs, the bacteria’s thick cell wall helps it to fight off the host immune system. Much of that cell wall is made from complex sugar molecules known as glycans, but it’s not well-understood how those glycans help to defend the bacteria. One reason for that is that there hasn’t been an easy way to label them inside cells. MIT chemists have now overcome that obstacle, demonstrating that they can label a ...