Year: 2021
Type: article
Abstract: The efficient and accurate characterization of the robustness of neural networks to \ninput perturbation is an important open problem. Many approaches exist including \nheuristic and exact (or complete) methods. Complete methods are expensive \nbut their mathematical formulation guarantees that they provide exact robustness \nmetrics. However, this guarantee is valid only if we assume that the verified network \napplies arbitrary-precision arithmetic and the verifier is reliable. In practice, \nhowever, both the networks and the verifiers apply limited-precision floating point \narithmetic. In this paper, we show that numerical roundoff errors can be exploited \nto craft adversarial networks, in which the actual robustness and the robustness \ncomputed by a state-of-the-art complete verifier radically differ. We also show \nthat such adversarial networks can be used to insert a backdoor into any network \nin such a way that the backdoor is completely missed by the verifier. The attack \nis easy to detect in its naive form but, as we show, the adversarial network can be \ntransformed to make its detection less trivial. We offer a simple defense against \nour particular attack based on adding a very small perturbation to the network \nweights. However, our conjecture is that other numerical attacks are possible, \nand exact verification has to take into account all the details of the computation \nexecuted by the verified networks, which makes the problem significantly harder.
Institution: University of Szeged
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W3127395865