Autoregressive Diffusion with Retentive Networks for Image and Audio Generation
Year: 2024
Type: article
Abstract: This paper introduces a novel approach to generative modeling that combines autoregressive diffusion processes with Retentive Networks (RetNet) for both image and audio generation. Our method, termed Autoregressive Diffusion with Retention (ADR), leverages the dual-mode capability of RetNet to achieve efficient parallel processing during training while enabling fast autoregressive generation through recurrent computation during inference. This approach demonstrates promising results in generating high-quality images and audio samples while potentially reducing computational complexity and improving scalability. We evaluate our model on standard datasets and discuss the implications of this approach for the broader field of generative AI.
Author: Albers, Josef
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W6930661825
FWCI:
Cites:
Cited by:
Related to: