STAR-SNR: spatial–temporal adaptive regulation and SNR optimization for few-shot video generation
Year: 2025
Type: article
Abstract: In recent years, text-to-image generation technology based on diffusion models has made significant progress, but extending it to the field of video generation, especially under few-shot conditions, still faces huge challenges. Existing methods usually rely on a large amount of text-video pair data or consume a lot of training resources. Based on this, this paper proposes a new few-shot video generation framework, STAN-SNR , which combines spatio-temporal feature regulation, feature scrolling enhancement and dynamic signal-to-noise ratio (SNR) weighting strategies, using 8–16 videos on a single A6000 training, effectively improving the quality and efficiency of video generation and reducing the amount of calculation. Specifically, the spatio-temporal feature regulation module effectively extracts spatio-temporal features and reduces computational complexity. The feature scrolling enhancement module enhances the ability to capture local features to avoid overfitting. In addition, the dynamic SNR weighting strategy adjusts the loss calculation according to the time step, which improves the convergence speed of the model, which is 2.44 times faster compared with the baseline model. The experimental results show that the STAN-SNR framework generates videos with higher text alignment, consistency, and diversity under few-shot conditions.
Source: Complex & Intelligent Systems
Authors: Xian Yu, Jianxun Zhang, Siran Tian, Hongyu Yi
Institution: Chongqing University of Technology
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $2,490
Awards: G404886445, G5545201110, G8926056618
OpenAlex ID: W4410247772