Unifying clustering and representation learning for unsupervised text analysis: A Bayesian knowledge-enhanced approach
Year: 2024
Type: article
Abstract: Clustering and representation learning are foundational tasks in natural language processing and text mining, aiming to structure and semantically encode text documents, respectively. While clustering organizes documents into cohesive groups based on similarity, representation learning generates low-dimensional embeddings that capture the nuances of document semantics. This paper presents a novel knowledge-enhanced approach that synergistically integrates these two tasks, improving performance in both areas. Our method employs a latent-factor Bayesian generative model, named MINING ( docuMent clusterINg and embeddING ), along with a specialized collapsed Gibbs sampling algorithm. We enrich the learned representations by incorporating external knowledge from word and entity embeddings, enhancing their semantic and syntactic richness. Our approach treats clustering and representation learning as interdependent tasks, allowing them to inform and refine one another. Extensive experiments on benchmark datasets demonstrate that our integrated approach outperforms traditional methods that carry out clustering and representation learning as separate tasks. • New approach to text analysis unifies clustering and representation learning. • Novel mathematical foundation laid for the principled integration of both tasks. • Innovative Bayesian latent-factor model of text corpora formalizes their interplay. • Document representations enhanced via external knowledge graph and text collections. • MCMC inference allows for both tasks to be performed jointly and cooperatively.
Source: Information Fusion
Authors: Gianni Costa, Riccardo Ortale
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $4,840
OpenAlex ID: W4405865437