Work

Collaborative Generative AI: Integrating GPT-k for Efficient Editing in Text-to-Image Generation

Year: 2023

Type: conference-paper

Abstract: The field of text-to-image (T2I) generation has garnered significant attention both within the research community and among everyday users.Despite the advancements of T2I models, a common issue encountered by users is the need for repetitive editing of input prompts in order to receive a satisfactory image, which is time-consuming and labor-intensive.Given the demonstrated text generation power of largescale language models, such as GPT-k, we investigate the potential of utilizing such models to improve the prompt editing process for T2I generation.We conduct a series of experiments to compare the common edits made by humans and GPT-k, evaluate the performance of GPT-k in prompting T2I, and examine factors that may influence this process.We found that GPT-k models focus more on inserting modifiers while humans tend to replace words and phrases, which includes changes to the subject matter.Experimental results show that GPT-k are more effective in adjusting modifiers rather than predicting spontaneous changes in the primary subject matters.Adopting the edit suggested by GPT-k models may reduce the percentage of remaining edits by 20-30%. 1 Our experiments are conducted upon StableDiffusion since it is a wide-adopted open-source large text-to-image generative model with SoTA performance.

Source: Conference on Empirical Methods in Natural Language Processing (EMNLP)

DOI: 10.18653/v1/2023.emnlp-main.685