Year: 2015
Type: dissertation
Abstract: Recent advances in processing and networking capabilities of computers have caused an accumulation \nof immense amounts of multimodal multimedia data (image, text, video). These data \nare generally presented as high-dimensional vectors of features. The availability of these highdimensional \ndata sets has provided the input to a large variety of statistical learning applications \nincluding clustering, classification, feature selection, outlier detection and density estimation. In \nthis context, a finite mixture offers a formal approach to clustering and a powerful tool to tackle \nthe problem of data modeling. A mixture model assumes that the data is generated by a set of \nparametric probability distributions. The main learning process of a mixture model consists of the \nfollowing two parts: parameter estimation and model selection (estimation the number of components). \nIn addition, other issues may be considered during the learning process of mixture models \nsuch as the: a) feature selection and b) outlier detection. The main objective of this thesis is to \nwork with different kinds of estimation criteria and to incorporate those challenges into a single \nframework. \nThe first contribution of this thesis is to propose a statistical framework which can tackle the problem \nof parameter estimation, model selection, feature selection, and outlier rejection in a unified \nmodel. We propose to use feature saliency and introduce an expectation-maximization (EM) algorithm \nfor the estimation of the Generalized Inverted Dirichlet (GID) mixture model. By using \nthe Minimum Message Length (MML), we can identify how much each feature contributes to \nour model as well as determine the number of components. The presence of outliers is an added \nchallenge and is handled by incorporating an auxiliary outlier component, to which we associate a uniform density. Experimental results on synthetic data, as well as real world applications involving \nvisual scenes and object classification, indicates that the proposed approach was promising, \neven though low-dimensional representation of the data was applied. In addition, it showed \nthe importance of embedding an outlier component to the proposed model. EM learning suffers \nfrom significant drawbacks. In order to overcome those drawbacks, a learning approach using a \nBayesian framework is proposed as our second contribution. This learning is based on the estimation \nof the parameters posteriors and by considering the prior knowledge about these parameters. \nCalculation of the posterior distribution of each parameter in the model is done by using Markov \nchain Monte Carlo (MCMC) simulation methods - namely, the Gibbs sampling and the Metropolis- \nHastings methods. The Bayesian Information Criterion (BIC) was used for model selection. The \nproposed model was validated on object classification and forgery detection applications. For the \nfirst two contributions, we developed a finite GID mixture. However, in the third contribution, \nwe propose an infinite GID mixture model. The proposed model simutaneously tackles the clustering \nand feature selection problems. The proposed learning model is based on Gibbs sampling. \nThe effectiveness of the proposed method is shown using image categorization application. Our \nlast contribution in this thesis is another fully Bayesian approach for a finite GID mixture learning \nmodel using the Reversible Jump Markov Chain Monte Carlo (RJMCMC) technique. The \nproposed algorithm allows for the simultaneously handling of the model selection and parameter estimation for high dimensional data. The merits of this approach are investigated using synthetic \ndata, and data generated from a challenging namely object detection.
Author: Mohamed Al Mashrgy
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W2201705342