Work

A Doodle-Based Control for Characters in Story Visualization

Year: 2024

Type: article

Abstract: We propose a story visualization technique that allows users to control the arrangement, poses, and styles of characters in a scene based on user-input doodle sketches. Our method utilizes a text encoder to process scene prompts and an image encoder to handle doodle sketches, generating inputs for a predefined scene generation model. Furthermore, we achieve efficient model training by fine-tuning the backbone network by applying a small dataset and employing a LoRA-based fine-tuning technique. We demonstrate that our method can generate characters with various poses and styles from doodle sketches, and it can validate the advantages of our approach by comparing it with the results from other story visualization studies.

Source: Electronics

DOI: 10.3390/electronics13234628

Authors: Hyemin Yang, Hyemin Yang, Heekyung Yang, Heekyung Yang, Kyungha Min

Institution: Sangmyung University

Language: English

Topic: Generative Adversarial Networks and Image Synthesis

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

Estimated APC paid: $2,728

OpenAlex ID: W4404695620

FWCI:

Cites: 33

Cited by:

Related to: 10