A unified deep learning model for image captioning and text-to-image synthesis
Year: 2025
Type: article
Abstract: Deep learning models have significantly advanced various artificial intelligence tasks, including text-to-image generation and image captioning. However, there remains a semantic gap between textual descriptions and visual representations, which affects the accuracy and coherence of generated images and captions. This paper proposes a novel deep learning model that integrates Stable Diffusion, Convolutional Neural Networks (CNN), and Long Short-Term Memory (LSTM) networks to enhance both text-to-image generation and image captioning tasks. The model employs CNN-LSTM architecture for feature extraction, while Stable Diffusion refines the output iteratively to improve coherence and realism. The approach generates diverse and high-quality images from text inputs and produces accurate captions for images. Experimental evaluations demonstrate the effectiveness of our approach. The image captioning model achieved a BLEU score of 0.89, highlighting its high accuracy. The text-to-image generation results also exhibit substantial improvements in visual realism and semantic alignment. The proposed model offers a robust framework for multimodal AI applications, advancing both content synthesis and understanding in multimedia tasks. These findings underscore the potential of deep learning in bridging the gap between textual and visual modalities, contributing to more effective and versatile AI-driven multimedia solutions.
DOI: 10.14419/m9njft54
Authors: Rose Mary Mathew, Sujesh P Lal, Gowri Ganesh
Institution: Federal Institute of Science and Technology
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W4410517062