Year: 2017
Type: dissertation
Abstract: Structured data and structured problems are common in machine learning, and they appear in many applications from computer vision, natural language understanding, information retrieval, computational biology, and many more. Compared to unstructured problems, where the input data is represented as a vector of independent feature values and output is a scalar prediction like a class label or regression value, both the input and output for structured problems may be objects with internal structure, like sequences, grids, trees or general graphs. Effectively exploiting the structure in the problems can help build efficient prediction models that significantly improve performance. The complexity of the structures requires expressive models that have enough representation capabilities. However, increased model complexity usually leads to increased inference complexity. A key challenge in building more expressive structured models is therefore to balance the model complexity and inference complexity, and explore models that are both expressive enough and have efficient inference. In this thesis, I present our work in the direction of building more expressive structured models, from developing more expressive structured output models, to semi-supervised learning of structured models, and then structured neural network models. The first technical part of the thesis describes a model that uses a new family of expressive high order pattern potentials, for which we characterized the theoretical properties and developed efficient inference and learning algorithms. Next we study semi-supervised learning algorithms for structured prediction problems that can help improve prediction performance by using unlabeled data. Motivated by our observation that standard structured models with iterative inference algorithms can be converted to neural networks, we study in particular structured neural network models for structured problems, and propose a new model that can handle prediction problems on graphs. Discussions about promising future directions are presented at the end of each technical chapter as well as at the end of the thesis.
Source: TSpace (University of Toronto)
Author: Yujia Li
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W2766322162