Get started
Work

Alignement texte-image des modèles génératifs d'images

Year: 2025
Type: dissertation
Abstract: Generative Text-to-Image models, particularly those based on diffusion, have achieved remarkable realism in image synthesis. However, they frequently struggle with robust text-image alignment, failing to adhere to complex prompts. This leads to common errors such as catastrophic neglect, where objects are omitted, and incorrect attribute binding, where properties like color are misassigned. This thesis addresses this core problem from two perspectives: evaluation and enhancement. First, we introduce a novel evaluation framework that uses structured prompt templates to systematically quantify specific compositional failures. This approach provides a more granular and interpretable analysis of model weaknesses, such as object omission or attribute misassignment, compared to existing general-purpose metrics. Second, we propose a principled enhancement to training-free, inference-time guidance. This method simplifies hyperparameter tuning by identifying a single optimal intervention step during the denoising process and is extended to modern transformer-based architectures that lack traditional cross-attention mechanisms. Finally, we generalize this guidance approach by introducing a novel framework that learns a distribution of well-text-aligned latent representations. By sampling from this learned distribution, our method efficiently generates multiple high-fidelity, compositionally correct images without the need for costly per-sample optimization. Together, these contributions provide a comprehensive approach to addressing compositional failures in Text-to-Image models. We establish a rigorous method for diagnosing specific alignment issues and then offer a series of inference-time interventions, from targeted single-sample corrections to efficient distributional sampling, that systematically improve the prompt fidelity and compositional accuracy of state-of-the-art generative models without requiring costly retraining.
Author: Grimal, Paul
Language: English
Open Access status: green
OpenAlex ID: W7119562440
Cites:
Cited by:
Related to: