Year: 2005
Type: conference-paper
Abstract: Although non-parametric tests have already been proposed for that pur-pose, statistical significance tests for non-standard measures (different from the classification error) are less often used in the literature. This paper is an attempt at empirically verifying how these tests compare with more classical tests, on various conditions. More precisely, using a very large dataset to estimate the whole “population”, we analyzed the behav-ior of several statistical test, varying the class unbalance, the compared models, the performance measure, and the sample size. The main re-sult is that providing big enough evaluation sets non-parametric tests are relatively reliable in all conditions. 1
Authors: Mikaela Keller, Samy Bengio, Siew Y. Wong
Institution: Idiap Research Institute
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Funder: National Science Foundation
OpenAlex ID: W2101995396