Work

Understanding and Guarding against Natural Language Adversarial Examples

Year: 2024
Type: dissertation
Abstract: Despite their success, machine learning models have been shown to be \nsusceptible to adversarial examples: carefully constructed perturbations of \nmodel inputs that are intended to lead a model into misclassifying those inputs. While this phenomenon was discovered in the context of computer \nvision, an increasing body of work focuses on adversarial examples in natural language processing (NLP). This PhD thesis presents an investigation \ninto such adversarial examples in the context of text classification, focusing \non studies to characterize them through both computational analyses and \nbehavioral studies. \nAs computational analysis, we present results showing that the effectiveness of adversarial word-level perturbations is due to the replacement \nof input words with low-frequency synonyms. Based on these insights, we \npropose an effective detection method for adversarial examples (Study 1). \nAs behavioral analysis, we present (Study 2) a data collection effort \ncomprising human-written word-level adversarial examples, and conduct \nstatistical comparisons between human- and machine-generated adversarial examples with respect to their preservation of sentiment, naturalness, \nand grammaticality. We find that human- and machine-authored adversarial examples are of similar quality across most comparisons, yet humans can \ngenerate adversarial examples with much greater efficiency. \nIn Study 3, we investigate the patterns of human behavior when authoring adversarial examples, and provide “human strategies” for generating \nadversarial examples that have the potential to advance automated attacks. \n \nStudy 4 discusses the NLP-related scientific safety and security literature with respect to more recent large language models (LLMs). We provide \na taxonomy of existing efforts related to that topic that are categorized into \nthreats arising from the generative capabilities of LLMs, prevention measures developed to safeguard models against misuse, and vulnerabilities \nstemming from imperfect prevention measures. \n \nWe conclude the thesis by discussing this work’s contributions and impact on the research community as well as potential future work arising from \nthe obtained insights.
Language: English
Open Access status: green
OpenAlex ID: W6987825075
Cites: 295
Cited by: 2
Related to: 10