Year: 2024
Type: article
Abstract: Large-scale text-to-video diffusion models have shown outstanding capabilities. However, their direct application to video stylization is hindered by the limited availability of text-to-video datasets and computational resources. Moreover, meeting content preservation standards for style transfer tasks is challenging due to the stochastic and destructive nature of the noise addition process. This letter introduces a succinct video stylization approach, named Style-A-Video, which leverages a generative pre-trained transformer and an image latent diffusion model for text-controlled video stylization. We improve the guidance conditions in the denoising process to maintain a balance between artistic expression and structural preservation. Additionally, by integrating sampling optimization and temporal consistency modules, we address inter-frame flickering and prevent additional artifacts. Comprehensive experimental results demonstrate superior content preservation and stylistic performance while minimizing resource consumption.
Source: IEEE Signal Processing Letters
Authors: Nisha Huang, Yuxin Zhang, Weiming Dong
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: G2203821447, G5049913043
OpenAlex ID: W4396753622