Work

Perturbed Model Validation: A New Framework to Validate Model Relevance

Year: 2019
Type: preprint
Abstract: This paper introduces Perturbed Model Validation (PMV), a new technique to validate model relevance and detect overfitting or underfitting. PMV operates by injecting noise to the training data, re-training the model against the perturbed data, then using the training accuracy decrease rate to assess model relevance. A larger decrease rate indicates better concept-hypothesis fit. We realise PMV by perturbing labels to inject noise, and evaluate PMV on four real-world datasets (breast cancer, adult, connect-4, and MNIST) and nine synthetic datasets in the classification setting. The results reveal that PMV selects models more precisely and in a more stable way than cross-validation, and effectively detects both overfitting and underfitting.
Source: HAL
Language: English
Open Access status: green
OpenAlex ID: W2945097090
Cites: 3
Cited by: 21
Related to: 9