Year: 2023
Type: dissertation
Abstract: The limitation of DNNs to explain their black box behavior has led to a recent surge of explainability methods. However, there are growing concerns that these explainability methods are not robust and trustworthy. In this paper, we perform the first robustness analysis of Neuron Explanation methods under a unified pipeline and show that these explanations can be significantly corrupted by random noises and well-designed perturbations added to the probing dataset. We find that even adding small random noise with a standard deviation of 0.02 can already change the assigned concepts of up to 28% neurons in the deeper layers. Furthermore, we devise a novel corruption algorithm and show that our algorithm can manipulate the concept of more than 80% neurons by poisoning less than 10\\% of probing data. This raises the concern of trusting Neuron Explanation methods in real-life safety and fairness critical applications.
Author: Divyansh, FNU
Institution: University of California San Diego
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7008878296