Year: 2023
Type: preprint
Abstract: To advance the understanding of robust deep learning, we delve into the effects of adversarial training on self-supervised and supervised contrastive learning alongside supervised learning. Our analysis uncovers significant disparities between adversarial and clean representations in standard-trained networks across various learning algorithms. Remarkably, adversarial training mitigates these disparities and fosters the convergence of representations toward a universal set, regardless of the learning scheme used. Additionally, increasing the similarity between adversarial and clean representations, particularly near the end of the network, enhances network robustness. These findings offer valuable insights for designing and training effective and robust deep learning networks. Our code is released at \textcolor{magenta}{\url{https://github.com/softsys4ai/CL-Robustness}}.
Source: arXiv (Cornell University)
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: Collaborative Research: CNS Core: Small: SmartSight: an AI-Based Computing Platform to Assist Blind and Visually Impaired People, Collaborative Research: SHF: Medium: Causal Performance Debugging for Highly-Configurable Systems, Collaborative Research: EAGER: Towards a Design Methodology for Software-Driven Sustainability
OpenAlex ID: W4319453097