Design and implementation of robust systems for secure malware detection
Year: 2016
Type: dissertation
Abstract: Malicious software (malware) have significantly increased in terms of number and effectiveness \nduring the past years. Until 2006, such software were mostly used to disrupt \nnetwork infrastructures or to show coders’ skills. Nowadays, malware constitute a very \nimportant source of economical profit, and are very difficult to detect. Thousands of \nnovel variants are released every day, and modern obfuscation techniques are used to \nensure that signature-based anti-malware systems are not able to detect such threats. \nThis tendency has also appeared on mobile devices, with Android being the most targeted \nplatform. To counteract this phenomenon, a lot of approaches have been developed \nby the scientific community that attempt to increase the resilience of anti-malware systems. \nMost of these approaches rely on machine learning, and have become very popular \nalso in commercial applications. However, attackers are now knowledgeable about these \nsystems, and have started preparing their countermeasures. This has lead to an arms \nrace between attackers and developers. Novel systems are progressively built to tackle \nthe attacks that get more and more sophisticated. For this reason, a necessity grows \nfor the developers to anticipate the attackers’ moves. This means that defense systems \nshould be built proactively, i.e., by introducing some security design principles in their \ndevelopment. The main goal of this work is showing that such proactive approach can \nbe employed on a number of case studies. To do so, I adopted a global methodology that \ncan be divided in two steps. First, understanding what are the vulnerabilities of current \nstate-of-the-art systems (this anticipates the attacker’s moves). Then, developing novel \nsystems that are robust to these attacks, or suggesting research guidelines with which \ncurrent systems can be improved. This work presents two main case studies, concerning \nthe detection of PDF and Android malware. The idea is showing that a proactive approach \ncan be applied both on the X86 and mobile world. The contributions provided on \nthis two case studies are multifolded. With respect to PDF files, I first develop novel attacks \nthat can empirically and optimally evade current state-of-the-art detectors. Then, \nI propose possible solutions with which it is possible to increase the robustness of such \ndetectors against known and novel attacks. With respect to the Android case study, \nI first show how current signature-based tools and academically developed systems are \nweak against empirical obfuscation attacks, which can be easily employed without particular \nknowledge of the targeted systems. Then, I examine a possible strategy to build a \nmachine learning detector that is robust against both empirical obfuscation and optimal \nattacks. Finally, I will show how proactive approaches can be also employed to develop \nsystems that are not aimed at detecting malware, such as mobile fingerprinting systems. \nIn particular, I propose a methodology to build a powerful mobile fingerprinting system, \nand examine possible attacks with which users might be able to evade it, thus preserving \ntheir privacy. To provide the aforementioned contributions, I co-developed (with the cooperation \nof the researchers at PRALab and Ruhr-Universität Bochum) various systems: \na library to perform optimal attacks against machine learning systems (AdversariaLib), \na framework for automatically obfuscating Android applications, a system to the robust \ndetection of Javascript malware inside PDF files (LuxOR), a robust machine learning system \nto the detection of Android malware, and a system to fingerprint mobile devices. I \nalso contributed to develop Android PRAGuard, a dataset containing a lot of empirical \nobfuscation attacks against the Android platform. Finally, I entirely developed Slayer \nNEO, an evolution of a previous system to the detection of PDF malware. The results \nattained by using the aforementioned tools show that it is possible to proactively build \nsystems that predict possible evasion attacks. This suggests that a proactive approach \nis crucial to build systems that provide concrete security against general and evasion \nattacks.
Author: Maiorca, Davide
Institution: University of Cagliari
Language: English
Subfield: Software
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W2369633396
