Improving Performance of the k-Nearest Neighbor Classifier by Combining Feature Selection with Feature Weighting.
Year: 2002
Type: article
Abstract: The k-nearest neighbor (k-NN) classification is a simple and effective classification approach. However, it suffers from over-sensitivity problem due to irrelevant and noisy features. There are two ways to relax such sensitivity. One is to assign each feature a weight, and the other way is to select a subset of relevant features. Existing researches showed that both approaches can improve generalization accuracy, but it is impossible to predict which one is better for a specific dataset. In this paper, we propose an algorithm to improve the effectiveness of k-NN by combining these two approaches. Specifically, we select all relevant features firstly, and then assign a weight to each relevant feature. Experiments have been conducted on 14 datasets from the UCI Machine Learning Repository, and the results show that our algorithm achieves the highest accuracy or near to the highest accuracy on all test datasets. It increases generalization accuracy 8.68% on the average. It also achieves higher generalization accuracy compared with well-known machine learning algorithm IB1-4 and C4.5.
DOI: 10.1527/tjsai.17.209
Authors: Yongguang Bao, Xiaoyong Du, Naohiro Ishii
Institutions: Nagoya Institute of Technology, Renmin University of China
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W2084597679