Choosing LSI Dimensions by Document Linear Association Analysis.
Year: 2003
Type: conference-paper
Abstract: Latent Semantic Indexing (LSI) has proven to be a valuable analysis tool with a wide range of applications, however the crucial question, choosing an appropriate number of dimensions for LSI, is still unsolved. In this paper, a new method which is to deal with this problem is described. It finds that a sum of total dot products between all document vectors reaches the maximum value at a specific number of dimensions for a given dataset. With this reduced dimensions LSI achieves the best performance. The performance evaluations have demonstrated that this method can choose an appropriate number of dimensions for LSI and effective detect the data structure for a dataset.
Authors: Chenggen Shi, Jie Lü
Institution: University of Technology Sydney
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W1520082443