Year: 2025
Type: article
Abstract: Robustness to distributional shifts remains a critical limitation for deploying deep neural networks (DNNs) in real-world applications. While DNNs excel in standard benchmarks, their performance often deteriorates under unseen or perturbed conditions. Understanding how internal information representations relate to such robustness remains underexplored. In this work, we propose an interpretable framework for robustness assessment based on partial information decomposition (PID), which quantifies how neurons redundantly, uniquely, or synergistically encode task-relevant information. Analysis of PID measures computed from clean inputs reveals that models characterized by higher redundancy rates and lower synergy rates tend to maintain more stable performance under various natural corruptions. Additionally, a higher rate of unique information is positively associated with improved classification accuracy on the data from which the measure is computed. These findings provide new insights for understanding and comparing model behavior through internal information analysis, and highlight the feasibility of lightweight robustness assessment without requiring extensive access to corrupted data.
Source: Entropy
DOI: 10.3390/e28010050
Institution: Beihang University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $3,137
Award: G8655508424
OpenAlex ID: W7117733640