Kondensering av källdomändataset för oövervakad djup domänanpassning : Jämförelse av UDA-algoritmer med kondenserade indatamängder och analys av domäninvarians
Year: 2024
Type: dissertation
Abstract: Modern deep-learning applications require vast amounts of data. Fine-tuning allows knowledge sharing between algorithms, enabling broader access to machine learning without the need for extensive computational resources to train on large datasets. Fine-tuning methods are primarily used for supervised tasks, where knowledge is extracted from a labeled source domain and applied to a labeled target domain by sharing model weights. Labeling data is an inconvenient task, often requiring manual work by humans. Unsupervised Domain Adaptation (UDA) algorithms can classify unlabeled raw data, by adapting knowledge from a labeled source domain to an unlabeled target domain. These models, however, have to train on source and target domain data simultaneously. Distilling knowledge from a massive source domain and sharing it as reusable weights doesn’t change the fact that these algorithms still require training on large source domain datasets. This thesis explores a new framework to distill and reuse knowledge from the source domain to classify unlabeled data with UDA algorithms. By condensing source domain datasets and evaluating several UDA algorithms, this study investigates the opportunity to fine-tune UDA algorithms by sharing knowledge in the form of condensed datasets. This paper presents an empirical study aimed at benchmarking the performance of various UDA algorithms on digit classification tasks with increasing levels of difficulty. Benchmark comparisons were conducted between UDA algorithms that have access to the full source domain datasets and counterparts using condensed datasets of different sizes. Our findings indicate that integrating UDA with Dataset Condensation (DC) can offer superior accuracy in scenarios with limited difficulty, specifically in the transition from USPS to MNIST. The study applies A-distance to measure domain invariance, revealing that UDA algorithms can be more proficient at encoding condensed source domain data and unlabeled target data similarly. For the majority of algorithms studied, neither accuracy nor A-distance was impacted negatively by training on at least 100 synthetic images. Our results indicate that condensed datasets can be used to successfully train UDA algorithms on simpler tasks, providing a preliminary framework for fine-tuning deep-learning algorithms on unlabeled data.
Author: Helle, Daniel
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W6992140404