Year: 2023
Type: conference-paper
Abstract: Auto-evaluation aims to automatically evaluate a trained model on any test dataset without human annotations. Most existing methods utilize global statistics of features extracted by the model as the representation of a dataset. This ignores the influence of the classification head and loses category-wise confusion information of the model. However, ratios of instances assigned to different categories together with their confidence scores reflect how many instances in which categories are difficult for the model to classify, which contain significant indicators for both overall and category-wise performances. In this paper, we propose a Confidence-based Category Relation-aware Regression (C2R2) method. C2R2divides all instances in a meta-set into different categories according to their confidence scores and extracts the global representation from them. For each category, C2R2encodes its local confusion relations to other categories into a local representation. The overall and category-wise performances are regressed from global and local representations, respectively. Extensive experiments show the effectiveness of our method.
Authors: Jiexin Wang, Jiahao Chen, Bing Su
Institution: Renmin University of China
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: G3634295602, G8415418894
OpenAlex ID: W4372346215