Work

Text-to-Image Alignment in Denoising-Based Models through Step Selection

Year: 2025

Type: preprint

Abstract: Visual generative AI models often encounter challenges related to text-image alignment and reasoning limitations. This paper presents a novel method for selectively enhancing the signal at critical denoising steps, optimizing image generation based on input semantics. Our approach addresses the shortcomings of early-stage signal modifications, demonstrating that adjustments made at later stages yield superior results. We conduct extensive experiments to validate the effectiveness of our method in producing semantically aligned images on Diffusion and Flow Matching model, achieving state-of-the-art performance. Our results highlight the importance of a judicious choice of sampling stage to improve performance and overall image alignment.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2504.17525

Authors: Grimal, Paul, Borgne, Hervé Le, Ferret, Olivier

Institution: Université Paris-Saclay

Language: English

Topic: Generative Adversarial Networks and Image Synthesis

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: Agence Nationale de la Recherche, Grand Équipement National De Calcul Intensif

Awards: G1190173965, Sharp Theoretical and Algorithmic Principles for frugal ML

OpenAlex ID: W4415307641

Cites:

Cited by:

Related to: