Transfer Learning Methods for Object and Action Recognition
Year: 2020
Type: dissertation
Abstract: Thanks to the advancement of machine learning and computer vision research we are observing remarkable improvements in object and action recognition. However, it can be seen that when there are differences (e.g., distributional, representations, semantic) in the training and test data, many learning algorithms fail to perform well or at times becomes infeasible. Nevertheless, problem setups dealing specifically with such differences are constantly being explored in literature. In general these problems are known as transfer learning problems. For instance, the objective of Domain Adaptation (DA) is to learn models that are invariant to distributional differences between train and test data (i.e., source and target domain data). Zero-Shot Learning (ZSL) is a framework for recognizing instances from classes never seen during training (i.e., missing classes in training data). In ZSL, the idea is to use related semantic information about data to workaround this data deficiency. In this thesis we present two solutions for Domain Adaptation and Zero-Shot Learning. First, we discuss our DA solution for learning an invariant Hilbert space for source and target domain data embedding. This is achieved by learning projections from individual domains to a latent space. We learn a Mahalanobis metric on this latent space to simultaneously minimizing domain disparity while maximizing a measure of discriminative power. We also show that the alignment of statistical properties (e.g., first and second order statistics) helps in improving our solution's performance, especially when there are no labeled samples from the target domain. We explore the effect of geometrical constraints such as orthogonality and positive definiteness for our model. Here, we use Riemannian optimization methods to seamlessly implement these constraints in our model training. Inspired by the success of statistical alignment, we further explore statistical alignment by min-max learning. Our motivation here is to avoid the trivial feature degeneration occurring when aligning two spaces by minimizing a statistical loss such as KL divergence. Here, we show that min-max statistical alignment is a promising solution for DA as well as ZSL. In fact, we show that min-max statistical alignment can reach comparable performance with state of the art adversarial solutions, including the ones using powerful Wasserstein GANs as the backbone. Lastly, we discuss our work on spatio-temporal transfer learning for human action recognition. In our survey on human action recognition from videos, we extensively study representation based and deep network based action recognition solutions. Here, we observe that one of the high-impact video action recognition solutions is the Two-Stream networks solution. However, the applicability of two-stream solution requires a temporal stream. In other words this restricts its applicability only to video data. In this work we explore use of transfer learning methods for devising two stream action recognition for still action images where there are no temporal information. We call this Temporal to Still Image Learning (or in short T2SIL). To explore the T2SIL framework we use existing still image and video datasets to form a new human action dataset of 40 classes. The data set contains still image actions and a video actions counterpart with the same set of classes. In this work, we explore three state of the art transfer learning solutions (i.e., Deep embedding learning, GANs and model distillation with Teacher-Student Networks) for T2SIL.
Author: Herath, Samitha
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W3036155538