Year: 2024
Type: conference-paper
Abstract: Text-to-image generation models have made rapid progress in recent years. They aim to generate vivid images after inputting text descriptions into the model. The framework of generative adversarial networks (GANs) is incorporated widely. GAN leverages techniques such as extracting fine-grained text features and designing practical loss functions to enhance the quality of generated images. Even though the generated image generally fits the text prompts in general, the shapes and spatial relationships of different parts in the image are often inconsistent with the descriptions. To address these challenges, we introduce a novel framework called Semantic-Consistent CLIP in Generative Adversarial Networks (SCC-GAN). Specifically, we present efficient Semantic-Feature Predic- tor by leveraging the pre-trained CLIP model. Due to the powerful CLIP encoder and accurate semantic-consistent transformation, the generator leverages CLIP to generate visual features from fused features generated from Semantic-Feature Predictor. COCO and CUB bird are the datasets that we conduct experiments on. The experimental results demonstrate the efficacy of SCC-GAN in comparison to recent text-to-image methods.
Authors: Boqun Li, Li Liu, Huaxiang Zhang
Institution: Shandong Normal University
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
OpenAlex ID: W4409833019