Year: 2024
Type: dissertation
Abstract: Text-to-image generation has seen substantial advancements in recent years, particularly with the advent of diffusion models, which have transformed how images are generated from text prompts.Despite these advancements, current methods often struggle with complex challenges such as accurately interpreting spatial and numerical relationships, unusual attributes, and logically intricate prompts.This work introduces a pioneering experimental approach specifically designed to tackle these complexities.Inspired by the layered editing techniques found in Adobe Photoshop, we propose an iterative framework that leverages Large Language Models (LLMs) along with state-of-the-art image generation models.This approach aims to create highly accurate and controllable visual representations from detailed textual descriptions.Our methodology involves decomposing text prompts into structured sub-prompts via LLMs, which are then sequentially rendered into images.To refine this process further, we integrate a dynamic feedback mechanism using a suite of foundational models.This system meticulously evaluates each generated image to ensure it aligns closely with the original prompt and maintains high visual fidelity.Our findings demonstrate the effectiveness of this approach, showing notable advancements for specific types of prompts while also revealing areas for improvement in others.This research underscores the considerable potential of our method and sets a foundation for future explorations in enhancing automated visual content generation.I extend my heartfelt gratitude to Professor Svetlana Lazebnik, my thesis advisor, for her unwavering guidance, support, and invaluable expertise throughout my graduate program.Her mentorship was pivotal in navigating the complexities of my research and in shaping my academic growth.I am also thankful to Viraj Shah, Sethuraman T V and Aiyu Cui, who have been dedicated mentors and collaborators on this project.Their insights and contributions were instrumental in bringing this research to fruition.Working alongside them has been a truly enriching experience, and I am grateful for the knowledge and perspectives they have shared.To my friends, who have provided me with endless encouragement and moments of relief amidst the rigors of graduate study, your support has been a source of strength.Your belief in my abilities and your unwavering encouragement have been my pillars of support.Special mention to Deeksha Anand, whose love, patience, and understanding have been constant sources of comfort and motivation throughout this journey. Additionally, I extend my appreciation to my uncles
Author: Sarswat, Shrey
Institution: University of Illinois Urbana-Champaign
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7220788478