Work

Adaptive Prompt Elicitation for Text-to-Image Generation

Year: 2026

Type: conference-paper

Abstract: Aligning text-to-image generation with user intent remains challenging, as users frequently provide ambiguous inputs and struggle with model idiosyncrasies. We propose Adaptive Prompt Elicitation (APE), a technique that adaptively poses visual queries to help users refine prompts without extensive writing. Our technical contribution is a formulation of interactive intent inference under an information-theoretic framework. APE represents latent user intent as interpretable feature requirements using language model priors, adaptively generates visual queries, and compiles elicited requirements into effective prompts. Evaluation on IDEA-Bench and DesignBench shows that APE achieves stronger alignment with improved efficiency. A user study with 128 participants on user-defined tasks demonstrates 19.8% higher perceived alignment without increased workload. Our work contributes a principled approach to prompting that offers an effective and efficient complement to the prevailing prompt-based interaction paradigm with text-to-image models.

Source: ACM International Conference on Intelligent User Interfaces (IUI)

DOI: 10.1145/3742413.3789149

Authors: Xinyi Wen, Lena Hegemann, Xiaofu Jin, Shuai Ma, Antti Oulasvirta

Institutions: University of Helsinki, Aalto University

Language: English

Topic: Generative Adversarial Networks and Image Synthesis

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funders: Finnish Center for Artificial Intelligence, HORIZON EUROPE European Research Council

Award: Artificial User