Year: 2005
Type: dissertation
Abstract: Clustering is one of the most common data mining tasks, used frequently for data \ncategorization and analysis in both industry and academia. The focus of our research is \non semi-supervised clustering, where we study how prior knowledge, gathered either from \nautomated information sources or human supervision, can be incorporated into clustering \nalgorithms. In this thesis, we present probabilistic models for semi-supervised clustering, \ndevelop algorithms based on these models and empirically validate their performances by \nextensive experiments on data sets from different domains, e.g., text analysis, hand-written \ncharacter recognition, and bioinformatics. \nIn many domains where clustering is applied, some prior knowledge is available \neither in the form of labeled data (specifying the category to which an instance belongs) or \npairwise constraints on some of the instances (specifying whether two instances should be in \nsame or different clusters). In this thesis, we first analyze effective methods of incorporating \nlabeled supervision into prototype-based clustering algorithms, and propose two variants of \nthe well-known KMeans algorithm that can improve their performance with limited labeled \ndata. \nWe then focus on the problem of semi-supervised clustering with constraints and \nshow how this problem can be studied in the framework of a well-defined probabilistic \ngenerative model of a Hidden Markov Random Field. We derive an efficient KMeans-type \niterative algorithm, HMRF-KMeans, for optimizing a semi-supervised clustering objective \nfunction defined on the HMRF model. We also give convergence guarantees of our algorithm \nfor a large class of clustering distortion measures (e.g., squared Euclidean distance, \nKL divergence, and cosine distance). \nFinally, we develop an active learning algorithm for acquiring maximally informative \npairwise constraints in an interactive query-driven framework, which to our knowledge \nis the first active learning algorithm for semi-supervised clustering with constraints. \nOther interesting problems of semi-supervised clustering that we discuss in this \nthesis include (1) semi-supervised graph-based clustering using kernels, (2) using prior \nknowledge to improve overlapping clustering of data, (3) integration of both constraintbased \nand distance-based semi-supervised clustering methods using the HMRF model, and \n(4) model selection techniques that use the available supervision to automatically select the \nright number of clusters.
Authors: Sugato Basu, Raymond J. Mooney
Institution: The University of Texas at Austin
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W1928593089