A Seamless and High-Performance Out-of-Distribution Detection Approach Simply Replacing the SoftMax Loss
Year: 2021
Type: preprint
Abstract: Current out-of-distribution detection approaches usually present special requirements (e.g., collecting outlier data and hyperparameter validation) and produce side effects (classification accuracy drop and slow/inefficient inferences). Recently, entropic out-of-distribution detection has been proposed as a seamless approach (i.e., a solution that avoids all the previously mentioned drawbacks). The entropic out-of-distribution detection solution comprises the IsoMax loss for training and the entropic score for out-of-distribution detection. The IsoMax loss works as a SoftMax loss drop-in replacement because swapping the SoftMax loss with the IsoMax loss requires no changes in the model's architecture or training procedures/hyperparameters. In this paper, we propose to perform what we call an isometrization of the distances used in the IsoMax loss. Additionally, we propose to replace the entropic score with the minimum distance score. Our experiments showed that these simple modifications increase out-of-distribution detection performance while keeping the solution seamless. Besides being competitive with or outperforming all major current approaches, our solution avoids all their current limitations in addition to being much easier to use, as just a simple loss replacement for training the neural network is required. Code available at this https URL.
Source: arXiv (Cornell University)
Authors: David Macêdo, Teresa B. Ludermir
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W3202083798