Diffusion-Cap: A diffusion model for image captioning
Year: 2024
Type: conference-paper
Abstract: While autoregressive image caption generation models have achieved remarkable success, they are still constrained in generation speed, which may become a bottleneck in practical applications. To address this challenge, our research introduces the Diffusion Image Captioning Model (Diffusion-Cap). This innovative, non-autoregressive framework conceptualizes image captioning as a process that unifies continuous and discrete diffusion. By exploring two distinct architectural designs, our research investigates optimal strategies for the image captioning diffusion process. Comprehensive assessments on the MSCOCO dataset confirm the superior performance of our Diffusion-Cap, surpassing existing non-autoregressive benchmarks in crafting accurate captions.
Authors: Tingxuan Xiao, Yizhe Yang, Yang Gao
Institution: Beijing Institute of Technology
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W4403244386