Kirjan ympäri 10 minuutissa: narratiivisen visualisaation luominen generatiivisen tekoälyn avulla
Year: 2024
Type: dissertation
Abstract: This thesis aims to present a method of visualizing a story in a way that supports user interaction. The goal is to enable the user to gain a more detailed understanding of the story. It will also investigate whether it is possible to utilize Large Language Models to extract information from a text to a format that can be computationally parsed, and whether it is possible to create images with consistent features using text to image models by giving the model prior context. The book used for the visualization is Around the world in 80 days. LLM performance is evaluated mainly by comparing the correctness of the results to manually labelled gold standard for selected chapters that attempt to indicate the worst-case scenario of many characters being present in several locations during a single chapter. The effectiveness of images created with automatically generated image descriptions is evaluated by subjectively comparing them to images created with manual descriptions that attempt to capture the key event or events in the chapter. Finally, character portraits generated with LLM-generated descriptions will be compared to illustrations in the 1873 release of the book. Furthermore, the cost of both the use of the LLMs and text to image models will be evaluated to see whether this approach would be financially viable. Finally, the visualization will be evaluated based on responsiveness and resource usage. The LLM-generated files required a total of 12 manual fixes, while comparing the data to manual data revealed shortcomings in being able to extract all characters in some chapters. Thus, fully automating the process is not possible at the moment, but it is more effective than extracting the data manually. Similarly, images generated with text to image models provided acceptable results in some cases, but manual oversight and fixes are still required for most of the images. Finally, while it is possible to understand events within the story by moving from a high-level overview to the full text with the visualization, it still requires further optimization to improve responsiveness.
Source: Aaltodoc (Aalto University)
Author: Mairue, Sami
Language: English
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7039136533